Your First Workflow

Learn how to build multi-agent workflows that orchestrate OSSA-compliant agents to automate complex tasks.

Overview

This tutorial guides you through creating a complete CI/CD workflow that: - Validates code quality on every commit - Runs automated tests - Performs security scanning - Deploys to staging environment - Coordinates multiple OSSA agents

Time to complete: 30-40 minutes

Prerequisites

• Completed Your First Agent
• BuildKit CLI installed and configured
• At least 2 OSSA agents available
• GitLab project with CI/CD access

Understanding Workflows

What is an OSSA Workflow?

A workflow orchestrates multiple agents to accomplish a complex goal. Think of it as a pipeline where each step is handled by a specialized agent.

Workflow characteristics: - Sequential or Parallel - Steps run in order or concurrently - Event-Driven - Triggered by webhooks, schedules, or manual invocation - Conditional - Steps execute based on previous results - Recoverable - Failed steps can retry or trigger fallback agents

Workflow vs CI/CD Pipeline

Step 1: Design Your Workflow

Identify Requirements

Goal: Automate code review and deployment for a Drupal module.

Steps needed: 1. Code Quality Check - Validate Drupal coding standards 2. Security Scan - Check for vulnerabilities 3. Automated Tests - Run PHPUnit tests 4. Build Artifact - Package module for deployment 5. Deploy to Staging - Deploy if all checks pass

Agents required: - code-quality-enforcer (Governor) - security-scanner (Critic) - test-runner (Worker) - artifact-builder (Worker) - deployment-orchestrator (Worker)

Map Agent Capabilities

# List available agents and their capabilities
buildkit agents list --capabilities

# Example output:
# Agent                      Role      Capabilities
# code-quality-enforcer      governor  validate_code, post_review
# security-scanner           critic    scan_vulnerabilities, audit_dependencies
# test-runner                worker    run_tests, generate_coverage
# artifact-builder           worker    build_package, sign_artifact
# deployment-orchestrator    worker    deploy_staging, deploy_production

Step 2: Create Workflow Manifest

Generate Scaffold

# Create workflow directory
mkdir -p ~/Sites/LLM/workflows
cd ~/Sites/LLM/workflows

# Generate workflow scaffold
buildkit workflows create drupal-ci-cd \
  --type ci-cd \
  --trigger gitlab.push

# This creates:
# drupal-ci-cd/
# ├── workflow.yaml           # Workflow definition
# ├── config/
# │   └── environment.yaml    # Environment-specific settings
# └── README.md

Define Workflow

Edit workflow.yaml:

ossaVersion: "0.2.4"

workflow:
  id: drupal-ci-cd
  name: Drupal CI/CD Pipeline
  version: "1.0.0"
  description: Automated code quality, testing, and deployment for Drupal modules

  trigger:
    type: webhook
    event: gitlab.push
    filters:
      branches:
        - main
        - develop
        - /^feature\/.*/
      paths:
        - "*.php"
        - "*.module"
        - "*.install"
        - "*.info.yml"

  variables:
    PROJECT_PATH: "{{ event.project.path_with_namespace }}"
    COMMIT_SHA: "{{ event.checkout_sha }}"
    BRANCH_NAME: "{{ event.ref }}"
    CHANGED_FILES: "{{ event.commits[*].added + event.commits[*].modified }}"

  stages:
    - name: validate
      parallel: true
      steps:
        - name: code_quality
          agent: code-quality-enforcer
          capability: validate_code
          input:
            files: "{{ CHANGED_FILES }}"
            standards: "drupal"
          output:
            violations: validation_result
          on_failure: fail_workflow

        - name: security_scan
          agent: security-scanner
          capability: scan_vulnerabilities
          input:
            files: "{{ CHANGED_FILES }}"
            include_dependencies: true
          output:
            vulnerabilities: security_result
          on_failure: fail_workflow

    - name: test
      depends_on: [validate]
      steps:
        - name: run_unit_tests
          agent: test-runner
          capability: run_tests
          input:
            test_suite: unit
            coverage_threshold: 80
            phpunit_config: phpunit.xml
          output:
            test_results: unit_test_results
            coverage: code_coverage
          on_failure: fail_workflow

        - name: run_integration_tests
          agent: test-runner
          capability: run_tests
          input:
            test_suite: integration
            database: postgres
          output:
            test_results: integration_test_results
          on_failure: warn

    - name: build
      depends_on: [test]
      condition: "{{ stages.test.status == 'passed' }}"
      steps:
        - name: build_artifact
          agent: artifact-builder
          capability: build_package
          input:
            source_path: "."
            output_format: tar.gz
            include_dependencies: true
            sign: true
          output:
            artifact_path: package_artifact
            checksum: package_checksum

    - name: deploy
      depends_on: [build]
      condition: "{{ BRANCH_NAME == 'main' || BRANCH_NAME == 'develop' }}"
      steps:
        - name: deploy_to_staging
          agent: deployment-orchestrator
          capability: deploy_staging
          input:
            artifact: "{{ package_artifact }}"
            environment: staging
            namespace: drupal-staging
            rollback_on_failure: true
          output:
            deployment_url: staging_url
            deployment_status: deploy_status

  notifications:
    - type: gitlab_comment
      on: [success, failure]
      template: |
        ## Workflow Results

        **Status:** {{ workflow.status }}
        **Duration:** {{ workflow.duration }}

        ### Code Quality
        - Violations: {{ validation_result.violations | length }}
        - Status: {{ stages.validate.steps.code_quality.status }}

        ### Security
        - Vulnerabilities: {{ security_result.vulnerabilities | length }}
        - Critical: {{ security_result.vulnerabilities | selectattr('severity', 'equalto', 'critical') | list | length }}

        ### Tests
        - Unit Tests: {{ unit_test_results.passed }}/{{ unit_test_results.total }}
        - Coverage: {{ code_coverage }}%

        {% if deploy_status %}
        ### Deployment
        - Environment: Staging
        - URL: {{ staging_url }}
        - Status: {{ deploy_status }}
        {% endif %}

    - type: slack
      on: [failure]
      webhook_url: "{{ env.SLACK_WEBHOOK_URL }}"
      channel: "#deployments"

  observability:
    tracing:
      enabled: true
      sampler: always
    metrics:
      enabled: true
      labels:
        project: "{{ PROJECT_PATH }}"
        workflow: drupal-ci-cd
    logging:
      level: info
      include_agent_logs: true

metadata:
  author: Your Team
  tags: [drupal, ci-cd, automation]
  documentation: https://docs.yourcompany.com/workflows/drupal-ci-cd

Step 3: Configure Environment Variables

Create config/environment.yaml:

environments:
  development:
    DRUPAL_DB_HOST: localhost
    DRUPAL_DB_NAME: drupal_dev
    SLACK_WEBHOOK_URL: https://hooks.slack.com/services/DEV/WEBHOOK

  staging:
    DRUPAL_DB_HOST: postgres-staging
    DRUPAL_DB_NAME: drupal_staging
    KUBERNETES_NAMESPACE: drupal-staging
    SLACK_WEBHOOK_URL: https://hooks.slack.com/services/STAGING/WEBHOOK

  production:
    DRUPAL_DB_HOST: postgres-prod
    DRUPAL_DB_NAME: drupal_prod
    KUBERNETES_NAMESPACE: drupal-prod
    SLACK_WEBHOOK_URL: https://hooks.slack.com/services/PROD/WEBHOOK

Step 4: Validate Workflow

# Validate workflow syntax
buildkit workflows validate workflow.yaml

# Expected output:
# ✓ Workflow syntax valid
# ✓ All referenced agents exist
# ✓ Agent capabilities match
# ✓ Variable references valid
# ✓ Conditional expressions valid

# Check for best practices
buildkit workflows lint workflow.yaml

# This checks:
# - Parallel execution opportunities
# - Dependency cycles
# - Timeout configurations
# - Error handling coverage

Step 5: Test Workflow Locally

Dry Run

# Simulate workflow execution without running agents
buildkit workflows run drupal-ci-cd \
  --dry-run \
  --event-file test-fixtures/gitlab-push-event.json

# Output shows:
# - Execution plan (stages, steps, dependencies)
# - Agent assignments
# - Variable interpolation
# - Condition evaluation

Local Execution

# Run workflow locally with mock data
buildkit workflows run drupal-ci-cd \
  --local \
  --event-file test-fixtures/gitlab-push-event.json \
  --env development

# Watch execution
buildkit workflows watch drupal-ci-cd

# View logs
buildkit workflows logs drupal-ci-cd --follow

Step 6: Deploy Workflow

Register with BuildKit

# Register workflow
buildkit workflows register workflow.yaml

# Verify registration
buildkit workflows list

# Expected output:
# ID              NAME                  VERSION  STATUS   TRIGGERS
# drupal-ci-cd    Drupal CI/CD Pipeline 1.0.0    active   gitlab.push

Configure GitLab Webhook

# Create webhook in GitLab
buildkit gitlab webhooks create \
  --project llm/modules/custom/my-drupal-module \
  --workflow drupal-ci-cd \
  --events push,merge_request

# This creates webhook pointing to:
# https://agents.yourcompany.com/workflows/drupal-ci-cd/trigger

Step 7: Execute Workflow

Manual Trigger

# Trigger workflow manually
buildkit workflows trigger drupal-ci-cd \
  --branch main \
  --commit HEAD

# Get execution ID
EXECUTION_ID=$(buildkit workflows executions list --workflow drupal-ci-cd --limit 1 --format json | jq -r '.[0].id')

echo "Execution ID: $EXECUTION_ID"

Monitor Execution

# Watch execution progress
buildkit workflows executions watch $EXECUTION_ID

# Example output:
# Stage: validate [RUNNING]
#   ├─ code_quality [RUNNING] → code-quality-enforcer
#   └─ security_scan [RUNNING] → security-scanner
#
# Stage: test [PENDING]
#   ├─ run_unit_tests [PENDING]
#   └─ run_integration_tests [PENDING]

View Detailed Logs

# Get logs for specific step
buildkit workflows executions logs $EXECUTION_ID \
  --step code_quality

# Get all logs
buildkit workflows executions logs $EXECUTION_ID --all

# Filter by level
buildkit workflows executions logs $EXECUTION_ID --level error

Step 8: Handle Failures and Retries

Configure Retry Logic

Update workflow.yaml:

stages:
  - name: test
    steps:
      - name: run_unit_tests
        agent: test-runner
        capability: run_tests
        retry:
          max_attempts: 3
          backoff: exponential
          initial_delay: 5s
          max_delay: 60s
        timeout: 10m
        on_failure: retry

Add Fallback Agents

stages:
  - name: deploy
    steps:
      - name: deploy_to_staging
        agent: deployment-orchestrator
        capability: deploy_staging
        fallback_agents:
          - deployment-orchestrator-backup
          - manual-deployment-trigger
        on_failure: fallback

Define Failure Workflows

on_workflow_failure:
  - name: rollback_deployment
    agent: deployment-orchestrator
    capability: rollback
    input:
      environment: staging

  - name: notify_team
    agent: notification-agent
    capability: send_alert
    input:
      channel: "#incidents"
      severity: high

Step 9: Add Advanced Features

Conditional Execution

stages:
  - name: deploy_production
    condition: |
      {{
        BRANCH_NAME == 'main' &&
        stages.test.status == 'passed' &&
        code_coverage >= 80 &&
        security_result.critical_vulnerabilities == 0
      }}
    steps:
      - name: deploy_to_production
        agent: deployment-orchestrator
        capability: deploy_production

Parallel Execution with Fan-out

stages:
  - name: multi_environment_test
    parallel: true
    matrix:
      environment: [php81, php82, php83]
      database: [mysql, postgres]
    steps:
      - name: test_compatibility
        agent: test-runner
        capability: run_tests
        input:
          php_version: "{{ matrix.environment }}"
          database: "{{ matrix.database }}"

Agent-to-Agent Data Passing

stages:
  - name: analyze
    steps:
      - name: complexity_analysis
        agent: code-analyzer
        capability: analyze_complexity
        output:
          complexity_score: code_complexity

  - name: review
    depends_on: [analyze]
    steps:
      - name: ai_review
        agent: ai-reviewer
        capability: review_code
        input:
          files: "{{ CHANGED_FILES }}"
          complexity_context: "{{ code_complexity }}"
          focus_areas: "{{ code_complexity.high_complexity_files }}"

Step 10: Monitoring and Observability

View Workflow Metrics

# Get workflow execution statistics
buildkit workflows metrics drupal-ci-cd

# Output:
# Workflow: drupal-ci-cd
# Total Executions: 245
# Success Rate: 94.7%
# Average Duration: 4m 32s
# P95 Duration: 8m 15s
# Failed Stages:
#   - test: 8 failures
#   - deploy: 5 failures

Set Up Alerts

# Create alert rule
buildkit workflows alerts create \
  --workflow drupal-ci-cd \
  --condition "success_rate < 90" \
  --action "notify:slack:#alerts"

# Create SLO (Service Level Objective)
buildkit workflows slo create \
  --workflow drupal-ci-cd \
  --metric duration \
  --threshold 10m \
  --percentile 95

Visualize in Grafana

# Generate Grafana dashboard
buildkit workflows dashboard generate drupal-ci-cd \
  --output grafana-dashboard.json

# Import to Grafana
curl -X POST https://grafana.yourcompany.com/api/dashboards/db \
  -H "Authorization: Bearer $GRAFANA_API_KEY" \
  -H "Content-Type: application/json" \
  -d @grafana-dashboard.json

Step 11: Optimize Performance

Enable Caching

stages:
  - name: build
    steps:
      - name: build_artifact
        agent: artifact-builder
        capability: build_package
        cache:
          key: "{{ COMMIT_SHA }}"
          paths:
            - vendor/
            - node_modules/
          policy: pull-push

Resource Limits

stages:
  - name: test
    steps:
      - name: run_unit_tests
        agent: test-runner
        capability: run_tests
        resources:
          requests:
            cpu: 500m
            memory: 1Gi
          limits:
            cpu: 2000m
            memory: 4Gi

Real-World Example: Complete Workflow

See full implementation at: ~/Sites/LLM/workflows/drupal-ci-cd/workflow.yaml

Troubleshooting

Workflow Won't Trigger

# Check webhook configuration
buildkit gitlab webhooks list --project llm/modules/custom/my-drupal-module

# Test webhook delivery
buildkit gitlab webhooks test <webhook-id>

# View webhook delivery logs in GitLab UI

Agent Not Responding

# Check agent health
buildkit agents status code-quality-enforcer

# View agent logs
buildkit agents logs code-quality-enforcer --tail 100

# Restart agent
buildkit agents restart code-quality-enforcer

Workflow Stuck

# Check execution status
buildkit workflows executions status $EXECUTION_ID

# Cancel stuck execution
buildkit workflows executions cancel $EXECUTION_ID

# Force retry from specific stage
buildkit workflows executions retry $EXECUTION_ID --from-stage test

Next Steps

You've built a production-ready workflow! Next:

1. Deploy to Kubernetes: Deploy to K8s Tutorial
2. Monitor agents: Monitor Agents
3. Learn best practices: Common Pitfalls

Additional Resources

• BuildKit Workflow Reference
• OSSA Workflow Patterns
• Workflow Examples