diff --git a/docs/CI_MIGRATION_PLAN.md b/docs/CI_MIGRATION_PLAN.md index 223cb34..ded1fe6 100644 --- a/docs/CI_MIGRATION_PLAN.md +++ b/docs/CI_MIGRATION_PLAN.md @@ -287,11 +287,782 @@ If issues arise during migration: - [ ] Test reusable workflow pattern - [ ] Document lessons learned +## Technical Architecture + +### Communication Flow + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Main Repository (.github/workflows/) │ +│ ┌───────────────────────────────────────────────────────┐ │ +│ │ Caller Workflow (php_quality.yml) │ │ +│ │ - Defines triggers (push, PR, etc.) │ │ +│ │ - Sets permissions │ │ +│ │ - Passes inputs and secrets │ │ +│ └───────────────────┬───────────────────────────────────┘ │ +│ │ uses: org/.github-private/...@main │ +└──────────────────────┼──────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ .github-private Repository (.github/workflows/) │ +│ ┌───────────────────────────────────────────────────────┐ │ +│ │ Reusable Workflow (reusable-php-quality.yml) │ │ +│ │ - workflow_call trigger │ │ +│ │ - Receives inputs from caller │ │ +│ │ - Inherits secrets from organization │ │ +│ │ - Executes CI/CD logic │ │ +│ │ - Returns job outputs │ │ +│ └───────────────────┬───────────────────────────────────┘ │ +│ │ │ +│ ┌───────────────────▼───────────────────────────────────┐ │ +│ │ Shared Scripts (scripts/shared/) │ │ +│ │ - extension_utils.py │ │ +│ │ - deployment utilities │ │ +│ │ - validation helpers │ │ +│ └───────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### Secret and Variable Inheritance Model + +``` +Organization Level (Settings > Secrets and Variables) +├── Secrets +│ ├── FTP_HOST (inherited by all repos) +│ ├── FTP_USER (inherited by all repos) +│ ├── FTP_KEY (inherited by all repos) +│ ├── FTP_PASSWORD (inherited by all repos) +│ ├── FTP_PATH (inherited by all repos) +│ └── API_TOKENS (inherited by all repos) +│ +├── Variables +│ ├── DEPLOY_DRY_RUN: false (can be overridden) +│ ├── FTP_PROTOCOL: sftp (can be overridden) +│ ├── FTP_PORT: 22 (can be overridden) +│ └── PHP_VERSIONS: ["8.0","8.1","8.2","8.3"] +│ +└── Repository Level (Override if needed) + ├── moko-cassiopeia + │ └── Variables + │ └── DEPLOY_DRY_RUN: true (override for this repo) + │ + └── other-project + └── Variables + └── FTP_PATH_SUFFIX: /custom (repo-specific) +``` + +### Workflow Version Pinning Strategy + +#### Option 1: Track Main Branch (Automatic Updates) +```yaml +uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@main +``` +**Pros**: Always get latest features and fixes +**Cons**: Breaking changes may affect workflows +**Use Case**: Development branches, staging deployments + +#### Option 2: Pin to Semantic Version (Stable) +```yaml +uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 +# or +uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1.2 +``` +**Pros**: Stable, predictable behavior +**Cons**: Manual updates required +**Use Case**: Production deployments, critical workflows + +#### Option 3: Pin to Specific Commit SHA (Maximum Stability) +```yaml +uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@a1b2c3d +``` +**Pros**: Immutable, guaranteed consistency +**Cons**: No automatic updates, harder to maintain +**Use Case**: Compliance requirements, audit trails + +### Detailed Workflow Conversion Examples + +#### Before: Inline Workflow (Current State) + +**`.github/workflows/php_quality.yml` (93 lines)** +```yaml +name: PHP Code Quality + +on: + pull_request: + branches: [ main, dev/*, rc/* ] + push: + branches: [ main, dev/*, rc/* ] + +permissions: + contents: read + +jobs: + php-compatibility-check: + name: PHP Compatibility Check + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Setup PHP + uses: shivammathur/setup-php@v2 + with: + php-version: '8.1' + extensions: mbstring, xml, curl, zip + + - name: Install PHP_CodeSniffer and PHPCompatibility + run: | + composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies + composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies + phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility + + - name: Check PHP 8.0+ Compatibility + run: phpcs --standard=PHPCompatibility --runtime-set testVersion 8.0- src/ + + phpcs: + name: PHP_CodeSniffer + runs-on: ubuntu-latest + strategy: + matrix: + php-version: ['8.0', '8.1', '8.2', '8.3'] + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Setup PHP ${{ matrix.php-version }} + uses: shivammathur/setup-php@v2 + with: + php-version: ${{ matrix.php-version }} + extensions: mbstring, xml, curl, zip + + - name: Install PHP_CodeSniffer + run: composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies + + - name: Run PHP_CodeSniffer + run: phpcs --standard=phpcs.xml src/ + + # ... additional jobs +``` + +#### After: Caller Workflow (Target State) + +**`.github/workflows/php_quality.yml` (15 lines - 84% reduction)** +```yaml +name: PHP Code Quality + +on: + pull_request: + branches: [ main, dev/*, rc/* ] + push: + branches: [ main, dev/*, rc/* ] + +permissions: + contents: read + +jobs: + quality-checks: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 + with: + php-versions: '["8.0", "8.1", "8.2", "8.3"]' + php-extensions: 'mbstring, xml, curl, zip' + source-directory: 'src' + phpcs-standard: 'phpcs.xml' + enable-phpcompat: true + enable-phpstan: true + phpstan-level: 'max' + secrets: inherit +``` + +**Benefits**: +- 84% reduction in code +- Centralized maintenance +- Consistent across all repositories +- Easy to add new checks (update once in .github-private) +- Version control with semantic versioning + +#### Reusable Workflow (in .github-private) + +**`.github-private/.github/workflows/reusable-php-quality.yml`** +```yaml +name: Reusable PHP Quality Checks + +on: + workflow_call: + inputs: + php-versions: + description: 'JSON array of PHP versions to test against' + required: false + type: string + default: '["8.0", "8.1", "8.2", "8.3"]' + php-extensions: + description: 'Comma-separated list of PHP extensions' + required: false + type: string + default: 'mbstring, xml, curl, zip' + source-directory: + description: 'Source code directory to analyze' + required: false + type: string + default: 'src' + phpcs-standard: + description: 'PHPCS standard configuration file' + required: false + type: string + default: 'phpcs.xml' + enable-phpcompat: + description: 'Enable PHP Compatibility checks' + required: false + type: boolean + default: true + enable-phpstan: + description: 'Enable PHPStan static analysis' + required: false + type: boolean + default: true + phpstan-level: + description: 'PHPStan analysis level' + required: false + type: string + default: 'max' + phpstan-config: + description: 'PHPStan configuration file' + required: false + type: string + default: 'phpstan.neon' + outputs: + phpcs-passed: + description: 'Whether PHPCS checks passed' + value: ${{ jobs.phpcs.outputs.passed }} + phpstan-passed: + description: 'Whether PHPStan checks passed' + value: ${{ jobs.phpstan.outputs.passed }} + +permissions: + contents: read + +jobs: + php-compatibility-check: + name: PHP Compatibility Check + runs-on: ubuntu-latest + if: ${{ inputs.enable-phpcompat }} + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Setup PHP + uses: shivammathur/setup-php@v2 + with: + php-version: '8.1' + extensions: ${{ inputs.php-extensions }} + coverage: none + + - name: Cache Composer dependencies + uses: actions/cache@v4 + with: + path: | + ~/.composer/cache + ~/.composer/vendor + key: ${{ runner.os }}-composer-phpcompat-${{ hashFiles('**/composer.lock') }} + restore-keys: | + ${{ runner.os }}-composer-phpcompat- + + - name: Install PHP_CodeSniffer and PHPCompatibility + run: | + composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies + composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies + phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility + + - name: Check PHP 8.0+ Compatibility + run: | + phpcs --standard=PHPCompatibility \ + --runtime-set testVersion 8.0- \ + --report=full \ + --report-file=phpcompat-report.txt \ + ${{ inputs.source-directory }}/ + + - name: Upload PHPCompatibility Report + if: always() + uses: actions/upload-artifact@v4 + with: + name: phpcompat-report + path: phpcompat-report.txt + retention-days: 30 + + phpcs: + name: PHP_CodeSniffer (PHP ${{ matrix.php-version }}) + runs-on: ubuntu-latest + outputs: + passed: ${{ steps.check.outputs.passed }} + strategy: + fail-fast: false + matrix: + php-version: ${{ fromJson(inputs.php-versions) }} + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Setup PHP ${{ matrix.php-version }} + uses: shivammathur/setup-php@v2 + with: + php-version: ${{ matrix.php-version }} + extensions: ${{ inputs.php-extensions }} + coverage: none + + - name: Cache Composer dependencies + uses: actions/cache@v4 + with: + path: | + ~/.composer/cache + ~/.composer/vendor + key: ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpcs-${{ hashFiles('**/composer.lock') }} + restore-keys: | + ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpcs- + + - name: Install PHP_CodeSniffer + run: composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies + + - name: Run PHP_CodeSniffer + id: check + run: | + phpcs --standard=${{ inputs.phpcs-standard }} \ + --report=full \ + --report-file=phpcs-report-${{ matrix.php-version }}.txt \ + ${{ inputs.source-directory }}/ + echo "passed=true" >> $GITHUB_OUTPUT + + - name: Upload PHPCS Report + if: always() + uses: actions/upload-artifact@v4 + with: + name: phpcs-report-php${{ matrix.php-version }} + path: phpcs-report-${{ matrix.php-version }}.txt + retention-days: 30 + + phpstan: + name: PHPStan (PHP ${{ matrix.php-version }}) + runs-on: ubuntu-latest + if: ${{ inputs.enable-phpstan }} + outputs: + passed: ${{ steps.check.outputs.passed }} + strategy: + fail-fast: false + matrix: + php-version: ${{ fromJson(inputs.php-versions) }} + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Setup PHP ${{ matrix.php-version }} + uses: shivammathur/setup-php@v2 + with: + php-version: ${{ matrix.php-version }} + extensions: ${{ inputs.php-extensions }} + coverage: none + + - name: Cache Composer dependencies + uses: actions/cache@v4 + with: + path: | + ~/.composer/cache + ~/.composer/vendor + key: ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpstan-${{ hashFiles('**/composer.lock') }} + restore-keys: | + ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpstan- + + - name: Install PHPStan + run: composer global require "phpstan/phpstan:^1.0" --with-all-dependencies + + - name: Run PHPStan + id: check + run: | + phpstan analyse \ + --configuration=${{ inputs.phpstan-config }} \ + --level=${{ inputs.phpstan-level }} \ + --error-format=table \ + --no-progress \ + --no-interaction \ + ${{ inputs.source-directory }}/ \ + > phpstan-report-${{ matrix.php-version }}.txt 2>&1 + echo "passed=true" >> $GITHUB_OUTPUT + + - name: Upload PHPStan Report + if: always() + uses: actions/upload-artifact@v4 + with: + name: phpstan-report-php${{ matrix.php-version }} + path: phpstan-report-${{ matrix.php-version }}.txt + retention-days: 30 +``` + +## Advanced Patterns and Best Practices + +### Pattern 1: Conditional Workflow Execution + +Allow repositories to enable/disable specific checks: + +```yaml +# Caller workflow +jobs: + quality: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 + with: + enable-phpcompat: ${{ github.event_name == 'pull_request' }} # Only on PRs + enable-phpstan: ${{ contains(github.event.head_commit.message, '[phpstan]') }} # Only if commit message contains [phpstan] + phpstan-level: ${{ github.ref == 'refs/heads/main' && 'max' || '6' }} # Stricter on main +``` + +### Pattern 2: Matrix Strategy Inheritance + +Pass complex matrix configurations: + +```yaml +# Caller workflow +jobs: + test: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-test.yml@v1 + with: + test-matrix: | + { + "php": ["8.0", "8.1", "8.2", "8.3"], + "joomla": ["4.4", "5.0"], + "database": ["mysql:8.0", "postgresql:14"] + } +``` + +### Pattern 3: Composite Actions for Reusability + +Break down workflows into composite actions for even more reusability: + +**`.github-private/.github/actions/setup-php-quality/action.yml`** +```yaml +name: 'Setup PHP Quality Tools' +description: 'Install PHP CodeSniffer, PHPCompatibility, and PHPStan' + +inputs: + php-version: + description: 'PHP version to setup' + required: true + enable-phpstan: + description: 'Install PHPStan' + required: false + default: 'true' + +runs: + using: 'composite' + steps: + - name: Setup PHP + uses: shivammathur/setup-php@v2 + with: + php-version: ${{ inputs.php-version }} + extensions: mbstring, xml, curl, zip + coverage: none + + - name: Install PHPCS and PHPCompatibility + shell: bash + run: | + composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies + composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies + phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility + + - name: Install PHPStan + if: inputs.enable-phpstan == 'true' + shell: bash + run: composer global require "phpstan/phpstan:^1.0" --with-all-dependencies +``` + +**Usage in reusable workflow:** +```yaml +- name: Setup PHP Quality Tools + uses: mokoconsulting-tech/.github-private/.github/actions/setup-php-quality@v1 + with: + php-version: ${{ matrix.php-version }} + enable-phpstan: true +``` + +### Pattern 4: Workflow Outputs and Chaining + +Use outputs to chain workflows: + +```yaml +# Caller workflow +jobs: + quality: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 + with: + php-versions: '["8.0", "8.1", "8.2", "8.3"]' + + deploy: + needs: quality + if: ${{ needs.quality.outputs.phpcs-passed == 'true' && needs.quality.outputs.phpstan-passed == 'true' }} + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 + with: + environment: staging +``` + +## Security Considerations + +### Principle of Least Privilege + +**Organization Secrets Access**: +- Only grant `.github-private` repository access to necessary secrets +- Use environment-specific secrets (staging, production) +- Rotate secrets regularly + +**Repository Permissions**: +```yaml +# .github-private repository settings +Permissions: + - Read: All organization members (for viewing workflows) + - Write: DevOps team only + - Admin: Organization owners only + +Branch Protection (main): + - Require pull request reviews (2 approvals) + - Require status checks to pass + - Require branches to be up to date + - No force pushes + - No deletions +``` + +### Secret Masking + +Ensure secrets are never exposed in logs: + +```yaml +# BAD - Exposes secret in logs +- name: Deploy + run: echo "Deploying with password: ${{ secrets.FTP_PASSWORD }}" + +# GOOD - Secret is masked +- name: Deploy + run: | + echo "::add-mask::${{ secrets.FTP_PASSWORD }}" + ./deploy.sh --password "${{ secrets.FTP_PASSWORD }}" +``` + +### Audit Trail + +Track all workflow executions: + +```yaml +# Add to all reusable workflows +- name: Audit Log + if: always() + run: | + echo "Workflow executed by: ${{ github.actor }}" + echo "Repository: ${{ github.repository }}" + echo "Branch: ${{ github.ref }}" + echo "Commit: ${{ github.sha }}" + echo "Workflow: ${{ github.workflow }}" + echo "Run ID: ${{ github.run_id }}" + echo "Timestamp: $(date -u +%Y-%m-%dT%H:%M:%SZ)" +``` + +## Performance Optimization + +### Caching Strategy + +**Composer Dependencies**: +```yaml +- name: Cache Composer + uses: actions/cache@v4 + with: + path: | + ~/.composer/cache + ~/.composer/vendor + key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} + restore-keys: | + ${{ runner.os }}-composer- +``` + +**Tool Installations**: +```yaml +- name: Cache Quality Tools + uses: actions/cache@v4 + with: + path: | + ~/.composer/vendor/squizlabs/php_codesniffer + ~/.composer/vendor/phpstan/phpstan + key: ${{ runner.os }}-php-tools-v1 +``` + +### Parallel Execution + +Maximize parallelism: + +```yaml +strategy: + fail-fast: false # Don't stop other jobs if one fails + max-parallel: 10 # Run up to 10 jobs simultaneously + matrix: + php-version: ['8.0', '8.1', '8.2', '8.3'] + joomla-version: ['4.4', '5.0'] +``` + +### Job Concurrency Control + +Prevent wasted resources: + +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} # Cancel old runs except on main +``` + +## Monitoring and Observability + +### Workflow Status Notifications + +**Slack Integration**: +```yaml +- name: Notify Slack on Failure + if: failure() + uses: slackapi/slack-github-action@v1 + with: + payload: | + { + "text": "Workflow failed: ${{ github.workflow }}", + "blocks": [ + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": "*Workflow*: ${{ github.workflow }}\n*Repository*: ${{ github.repository }}\n*Branch*: ${{ github.ref }}\n*Actor*: ${{ github.actor }}\n*Run*: <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run>" + } + } + ] + } + env: + SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} +``` + +### Metrics Collection + +Track workflow execution metrics: + +```yaml +- name: Record Metrics + if: always() + run: | + curl -X POST "${{ secrets.METRICS_ENDPOINT }}" \ + -H "Content-Type: application/json" \ + -d '{ + "workflow": "${{ github.workflow }}", + "repository": "${{ github.repository }}", + "status": "${{ job.status }}", + "duration": "${{ steps.start-time.outputs.elapsed }}", + "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)" + }' +``` + +## Troubleshooting Guide + +### Common Issues and Solutions + +#### Issue 1: "Workflow not found" error + +**Symptom**: +``` +Error: Unable to resolve action `mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@main`, +unable to find version `main` +``` + +**Solutions**: +1. Verify `.github-private` repository exists and is accessible +2. Check repository permissions (must have at least read access) +3. Verify branch name (main vs master) +4. Ensure workflow file exists at specified path + +**Verification Commands**: +```bash +# Check repository access +gh api repos/mokoconsulting-tech/.github-private + +# List workflow files +gh api repos/mokoconsulting-tech/.github-private/contents/.github/workflows + +# Check branch exists +gh api repos/mokoconsulting-tech/.github-private/branches/main +``` + +#### Issue 2: Secrets not inherited + +**Symptom**: +``` +Error: Secret FTP_PASSWORD is not set +``` + +**Solutions**: +1. Ensure `secrets: inherit` is set in caller workflow +2. Verify secret exists at organization level +3. Check `.github-private` repository has access to organization secrets +4. Verify secret names match exactly (case-sensitive) + +**Verification**: +```bash +# List organization secrets +gh api orgs/mokoconsulting-tech/actions/secrets + +# Check repository secret access +gh api repos/mokoconsulting-tech/.github-private/actions/secrets +``` + +#### Issue 3: Workflow runs on wrong trigger + +**Symptom**: +Workflow runs when it shouldn't, or doesn't run when expected + +**Solutions**: +1. Review `on:` triggers in caller workflow +2. Check branch protection rules +3. Verify path filters if using `paths:` or `paths-ignore:` +4. Test with different trigger events + +**Example Fix**: +```yaml +# Before (runs on all pushes) +on: + push: + +# After (runs only on main and feature branches) +on: + push: + branches: + - main + - 'feature/**' +``` + +#### Issue 4: Matrix strategy not working + +**Symptom**: +Only one job runs instead of multiple matrix jobs + +**Solutions**: +1. Verify JSON syntax in matrix definition +2. Use `fromJson()` for string inputs +3. Check for empty arrays +4. Validate matrix variable references + +**Example**: +```yaml +# Caller workflow +with: + php-versions: '["8.0", "8.1", "8.2"]' # Must be JSON string + +# Reusable workflow +matrix: + php-version: ${{ fromJson(inputs.php-versions) }} # Convert to array +``` + ## References - [GitHub Reusable Workflows Documentation](https://docs.github.com/en/actions/using-workflows/reusing-workflows) - [GitHub Security Best Practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) - [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) +- [workflow_call Event](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_call) +- [GitHub Actions Caching](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows) +- [Composite Actions](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action) ## Support @@ -299,10 +1070,12 @@ For questions or issues during migration: - Review this document - Check GitHub Actions documentation - Contact: DevOps team +- Slack: #devops-support --- -**Status**: Draft - Awaiting Review +**Status**: Ready for Implementation **Author**: GitHub Copilot -**Date**: 2026-01-04 -**Version**: 1.0 +**Date**: 2026-01-05 +**Version**: 2.0 +**Last Updated**: 2026-01-05 diff --git a/docs/MIGRATION_CHECKLIST.md b/docs/MIGRATION_CHECKLIST.md index c0e74cb..b5f7170 100644 --- a/docs/MIGRATION_CHECKLIST.md +++ b/docs/MIGRATION_CHECKLIST.md @@ -78,26 +78,688 @@ This checklist guides the migration of CI/CD workflows from individual repositor ## Phase 4: Workflow Migration (Priority Order) ### Workflow 1: php_quality.yml (Low Risk) + +#### Pre-Migration Checklist - [ ] Create reusable-php-quality.yml in .github-private - [ ] Convert workflow to use workflow_call trigger - [ ] Add input parameters: - [ ] php-versions (JSON array) - [ ] php-extensions (string) - [ ] working-directory (string) -- [ ] Test reusable workflow independently -- [ ] Update main repository to call reusable workflow -- [ ] Test end-to-end integration -- [ ] Monitor for issues (1 week) -- [ ] Document lessons learned + - [ ] enable-phpcompat (boolean) + - [ ] enable-phpstan (boolean) + +#### Detailed Migration Steps + +**Step 1: Create Reusable Workflow** +```bash +# In .github-private repository +cd .github-private +git checkout -b feature/add-php-quality-workflow + +# Create workflow file +mkdir -p .github/workflows +cat > .github/workflows/reusable-php-quality.yml << 'EOF' +name: Reusable PHP Quality Workflow + +on: + workflow_call: + inputs: + php-versions: + required: false + type: string + default: '["8.0", "8.1", "8.2", "8.3"]' + # ... other inputs + outputs: + all-passed: + value: ${{ jobs.summary.outputs.all-passed }} + +jobs: + phpcs: + # ... job definition + + phpstan: + # ... job definition + + summary: + # ... summary job +EOF + +# Commit and push +git add .github/workflows/reusable-php-quality.yml +git commit -m "feat: add reusable PHP quality workflow" +git push origin feature/add-php-quality-workflow + +# Create pull request +gh pr create --title "Add reusable PHP quality workflow" \ + --body "Initial implementation of reusable PHP quality checks" +``` + +**Step 2: Test Reusable Workflow** +```bash +# Create test repository or use existing +cd test-repository +git checkout -b test/reusable-workflow + +# Create test caller workflow +cat > .github/workflows/test-php-quality.yml << 'EOF' +name: Test PHP Quality (Reusable) + +on: + push: + branches: [test/**] + +jobs: + quality: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@feature/add-php-quality-workflow + with: + php-versions: '["8.1"]' + enable-phpcompat: true + enable-phpstan: true + secrets: inherit +EOF + +git add .github/workflows/test-php-quality.yml +git commit -m "test: add test caller for reusable workflow" +git push origin test/reusable-workflow + +# Monitor test run +gh run watch +``` + +**Step 3: Validation Checks** +- [ ] Verify all jobs execute successfully +- [ ] Check that reports are generated correctly +- [ ] Verify secrets are accessible +- [ ] Test with different input combinations +- [ ] Validate error handling (introduce intentional errors) + +**Validation Script**: +```bash +#!/bin/bash +# validate_php_quality_workflow.sh + +set -euo pipefail + +echo "=== Validating PHP Quality Workflow ===" + +# Test 1: Basic execution +echo "Test 1: Basic execution with default parameters" +gh workflow run test-php-quality.yml +WORKFLOW_ID=$(gh run list --workflow=test-php-quality.yml --limit 1 --json databaseId --jq '.[0].databaseId') +gh run watch $WORKFLOW_ID + +# Test 2: Custom PHP versions +echo "Test 2: Custom PHP versions" +gh workflow run test-php-quality.yml \ + --field php-versions='["8.2","8.3"]' +gh run watch + +# Test 3: Disable PHPStan +echo "Test 3: With PHPStan disabled" +gh workflow run test-php-quality.yml \ + --field enable-phpstan=false +gh run watch + +# Test 4: Check outputs +echo "Test 4: Verify workflow outputs" +gh run view $WORKFLOW_ID --json jobs --jq '.jobs[].conclusion' | grep -q success + +echo "✅ All validation tests passed" +``` + +**Step 4: Update Main Repository** +```bash +# In main repository (moko-cassiopeia) +cd moko-cassiopeia +git checkout -b migrate/php-quality-workflow + +# Backup existing workflow +mkdir -p .backup/workflows +cp .github/workflows/php_quality.yml .backup/workflows/php_quality.yml.backup +git add .backup/workflows/ + +# Replace with caller workflow +cat > .github/workflows/php_quality.yml << 'EOF' +name: PHP Code Quality + +on: + pull_request: + branches: [ main, dev/*, rc/* ] + push: + branches: [ main, dev/*, rc/* ] + +permissions: + contents: read + +jobs: + quality-checks: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 + with: + php-versions: '["8.0", "8.1", "8.2", "8.3"]' + php-extensions: 'mbstring, xml, curl, zip' + source-directory: 'src' + enable-phpcompat: true + enable-phpstan: true + secrets: inherit +EOF + +git add .github/workflows/php_quality.yml +git commit -m "migrate: convert php_quality.yml to use centralized reusable workflow" +git push origin migrate/php-quality-workflow + +# Create pull request +gh pr create --title "Migrate PHP quality workflow to centralized version" \ + --body "Migrates php_quality.yml to call reusable workflow from .github-private" +``` + +**Step 5: Integration Testing** +- [ ] Create test PR to trigger workflow +- [ ] Verify workflow executes correctly +- [ ] Check that all jobs complete successfully +- [ ] Verify artifacts are uploaded correctly +- [ ] Compare execution time with old workflow +- [ ] Validate error messages are clear + +**Integration Test Script**: +```bash +#!/bin/bash +# test_integration.sh + +set -euo pipefail + +echo "=== Integration Testing ===" + +# Create test branch +git checkout -b test/integration-$(date +%s) +echo "// test change" >> src/test.php +git add src/test.php +git commit -m "test: trigger workflow" +git push origin HEAD + +# Create PR +PR_URL=$(gh pr create --title "Test: PHP Quality Integration" \ + --body "Testing integrated PHP quality workflow" \ + --head $(git branch --show-current)) + +echo "PR created: $PR_URL" +echo "Waiting for checks..." + +# Wait for checks to complete +gh pr checks $PR_URL --watch + +# Verify checks passed +CHECK_STATUS=$(gh pr checks $PR_URL --json state --jq '.[0].state') +if [ "$CHECK_STATUS" == "SUCCESS" ]; then + echo "✅ Integration test passed" + gh pr close $PR_URL --delete-branch +else + echo "❌ Integration test failed" + exit 1 +fi +``` + +**Step 6: Monitor for Issues** +- [ ] Monitor for 1 week (7 days) +- [ ] Track workflow execution times +- [ ] Collect developer feedback +- [ ] Document any issues encountered +- [ ] Fix issues promptly + +**Monitoring Dashboard**: +```bash +#!/bin/bash +# monitor_workflow.sh + +set -euo pipefail + +echo "=== Workflow Monitoring Dashboard ===" + +WORKFLOW="php_quality.yml" +START_DATE=$(date -d '7 days ago' +%Y-%m-%d) + +# Execution count +EXEC_COUNT=$(gh run list --workflow=$WORKFLOW --created=">$START_DATE" --json databaseId --jq 'length') +echo "Executions (last 7 days): $EXEC_COUNT" + +# Success rate +SUCCESS_COUNT=$(gh run list --workflow=$WORKFLOW --created=">$START_DATE" --status=success --json databaseId --jq 'length') +SUCCESS_RATE=$(awk "BEGIN {printf \"%.1f\", ($SUCCESS_COUNT/$EXEC_COUNT)*100}") +echo "Success rate: $SUCCESS_RATE%" + +# Average duration +AVG_DURATION=$(gh run list --workflow=$WORKFLOW --created=">$START_DATE" --limit 20 --json conclusion,duration --jq '[.[] | select(.conclusion=="success") | .duration] | add/length') +echo "Average duration: ${AVG_DURATION}s" + +# Recent failures +echo -e "\nRecent failures:" +gh run list --workflow=$WORKFLOW --status=failure --limit 5 --json databaseId,createdAt,headBranch,conclusion + +# Alert if success rate < 90% +if (( $(echo "$SUCCESS_RATE < 90" | bc -l) )); then + echo "⚠️ WARNING: Success rate below 90%" + # Send alert + curl -X POST $SLACK_WEBHOOK_URL \ + -H 'Content-Type: application/json' \ + -d "{\"text\":\"PHP Quality Workflow success rate is ${SUCCESS_RATE}%\"}" +fi +``` + +**Step 7: Document Lessons Learned** +- [ ] Document any issues encountered +- [ ] Note what went well +- [ ] Identify improvements for next workflow +- [ ] Update migration documentation ### Workflow 2: joomla_testing.yml (Low Risk) + +#### Pre-Migration Checklist - [ ] Create reusable-joomla-testing.yml in .github-private - [ ] Convert workflow to use workflow_call trigger - [ ] Add input parameters as needed - [ ] Test reusable workflow independently -- [ ] Update main repository to call reusable workflow -- [ ] Test end-to-end integration -- [ ] Monitor for issues (1 week) + +#### Detailed Migration Steps + +**(Similar structure to Workflow 1, with Joomla-specific considerations)** + +**Step 1-7**: Follow same pattern as php_quality.yml migration + +**Additional Considerations**: +- [ ] Test with different Joomla versions (4.4, 5.0) +- [ ] Verify database compatibility testing +- [ ] Check Joomla-specific tooling integration +- [ ] Validate Joomla Update Server compatibility + +### Workflow 3: deploy_staging.yml (High Risk) + +#### Pre-Migration Checklist +- [ ] Create reusable-deploy-staging.yml in .github-private +- [ ] Convert workflow to use workflow_call trigger +- [ ] Add input parameters for deployment configuration +- [ ] Configure secret requirements +- [ ] Create detailed rollback plan + +#### Risk Mitigation Strategies + +**Pre-Deployment Checks**: +```yaml +- name: Pre-Deployment Validation + run: | + # Verify deployment prerequisites + if [ -z "${{ secrets.FTP_HOST }}" ]; then + echo "❌ FTP_HOST not configured" + exit 1 + fi + + # Test connectivity + nc -zv ${{ secrets.FTP_HOST }} 22 || exit 1 + + # Verify artifact exists + if [ ! -f deployment.zip ]; then + echo "❌ Deployment artifact not found" + exit 1 + fi + + echo "✅ Pre-deployment checks passed" +``` + +**Deployment with Backup**: +```yaml +- name: Backup Current Deployment + run: | + # Create backup of current deployment + ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_HOST }} \ + "cd ${{ secrets.FTP_PATH }} && tar -czf backup-$(date +%Y%m%d-%H%M%S).tar.gz ." + + echo "✅ Backup created" + +- name: Deploy New Version + id: deploy + run: | + # Deploy new version + scp deployment.zip ${{ secrets.FTP_USER }}@${{ secrets.FTP_HOST }}:${{ secrets.FTP_PATH }}/ + + ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_HOST }} \ + "cd ${{ secrets.FTP_PATH }} && unzip -o deployment.zip" + + echo "✅ Deployment successful" + +- name: Health Check + run: | + # Verify deployment + for i in {1..30}; do + if curl -f -s "${{ inputs.deploy-url }}/health" > /dev/null; then + echo "✅ Health check passed" + exit 0 + fi + echo "Attempt $i/30 failed, retrying..." + sleep 10 + done + + echo "❌ Health check failed" + exit 1 + +- name: Rollback on Failure + if: failure() + run: | + echo "⚠️ Deployment failed, rolling back..." + + # Restore from backup + BACKUP=$(ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_HOST }} \ + "cd ${{ secrets.FTP_PATH }} && ls -t backup-*.tar.gz | head -1") + + ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_HOST }} \ + "cd ${{ secrets.FTP_PATH }} && tar -xzf $BACKUP" + + echo "✅ Rollback completed" +``` + +#### Detailed Migration Steps + +**Step 1: Create Canary Deployment** +```bash +# Test deployment on canary environment first +gh workflow run deploy-staging.yml \ + --field environment=canary \ + --field deploy-url=https://canary.staging.example.com +``` + +**Step 2: Gradual Rollout** +- [ ] Week 1: Deploy to canary environment +- [ ] Week 2: Deploy to 25% of staging instances +- [ ] Week 3: Deploy to 50% of staging instances +- [ ] Week 4: Deploy to 100% of staging instances + +**Step 3: Full Migration** +- [ ] Convert to reusable workflow +- [ ] Update all deployment triggers +- [ ] Monitor closely for first 2 weeks + +**Emergency Rollback Procedure**: +```bash +#!/bin/bash +# emergency_rollback.sh + +set -euo pipefail + +echo "=== EMERGENCY ROLLBACK ===" +echo "This will revert deployment workflow to local version" +read -p "Continue? (yes/no): " CONFIRM + +if [ "$CONFIRM" != "yes" ]; then + echo "Rollback cancelled" + exit 0 +fi + +# Revert workflow +git checkout HEAD~1 -- .github/workflows/deploy_staging.yml +git commit -m "emergency: rollback deploy_staging workflow" +git push + +# Trigger immediate deployment with rolled-back workflow +gh workflow run deploy_staging.yml --field environment=staging + +echo "✅ Rollback initiated" +``` + +### Workflow 4: release_pipeline.yml (High Risk) + +#### Pre-Migration Checklist +- [ ] Create reusable-release-pipeline.yml in .github-private +- [ ] Convert workflow to use workflow_call trigger +- [ ] Add input parameters +- [ ] Configure all secret requirements +- [ ] Test with test release on feature branch + +#### Release Testing Strategy + +**Test Release Checklist**: +- [ ] Create test tag (v0.0.0-test) +- [ ] Trigger release workflow +- [ ] Verify package is built correctly +- [ ] Verify package is uploaded to correct location +- [ ] Verify GitHub release is created +- [ ] Verify release notes are correct +- [ ] Delete test release and tag + +**Test Release Script**: +```bash +#!/bin/bash +# test_release.sh + +set -euo pipefail + +echo "=== Test Release ===" + +# Create test tag +TEST_TAG="v0.0.0-test-$(date +%s)" +git tag -a $TEST_TAG -m "Test release" +git push origin $TEST_TAG + +# Trigger release workflow +gh workflow run release_pipeline.yml \ + --field release_classification=rc + +# Monitor release +gh run watch + +# Verify release created +gh release view $TEST_TAG + +# Download and verify artifact +gh release download $TEST_TAG +ls -lh *.zip + +# Cleanup +gh release delete $TEST_TAG --yes +git tag -d $TEST_TAG +git push origin :refs/tags/$TEST_TAG + +echo "✅ Test release completed successfully" +``` + +#### Migration Steps + +**Step 1-7**: Similar to previous workflows, with additional release-specific testing + +**Additional Validation**: +- [ ] Verify version detection works correctly +- [ ] Test RC (release candidate) releases +- [ ] Test stable releases +- [ ] Verify artifact signing (if enabled) +- [ ] Test rollback of failed releases + +## Phase 5: Script Migration + +### Shared Scripts to Migrate + +#### extension_utils.py + +**Migration Steps**: +```bash +# In .github-private repository +mkdir -p scripts/shared +cp path/to/extension_utils.py scripts/shared/ + +# Update imports in workflows +# From: +python3 scripts/lib/extension_utils.py + +# To: +python3 $GITHUB_WORKSPACE/.github-private-scripts/extension_utils.py +``` + +**Verification Script**: +```bash +#!/bin/bash +# verify_script_migration.sh + +set -euo pipefail + +echo "=== Verifying Script Migration ===" + +# Test extension_utils.py +python3 scripts/shared/extension_utils.py +echo "✅ extension_utils.py works correctly" + +# Test common.py +python3 scripts/shared/common.py +echo "✅ common.py works correctly" + +# Test all reusable workflows use correct paths +grep -r "scripts/lib" .github/workflows/ && { + echo "❌ Found old script paths" + exit 1 +} || echo "✅ No old script paths found" +``` + +### Script Dependency Management + +**Create requirements.txt** for shared scripts: +```txt +# .github-private/scripts/shared/requirements.txt +# Python dependencies for shared scripts +``` + +**Install Dependencies in Workflows**: +```yaml +- name: Setup Python Dependencies + run: | + pip install -r $GITHUB_WORKSPACE/.github-private-scripts/requirements.txt +``` + +## Phase 6: Testing and Validation + +### Comprehensive Test Suite + +**test_all_workflows.sh**: +```bash +#!/bin/bash +# test_all_workflows.sh + +set -euo pipefail + +echo "=== Comprehensive Workflow Testing ===" + +WORKFLOWS=( + "php_quality.yml" + "joomla_testing.yml" + "deploy_staging.yml" + "release_pipeline.yml" +) + +for workflow in "${WORKFLOWS[@]}"; do + echo "Testing $workflow..." + + # Trigger workflow + gh workflow run $workflow + + # Wait for completion + sleep 10 + + # Check result + LATEST_RUN=$(gh run list --workflow=$workflow --limit 1 --json databaseId,conclusion --jq '.[0]') + CONCLUSION=$(echo $LATEST_RUN | jq -r '.conclusion') + + if [ "$CONCLUSION" == "success" ]; then + echo "✅ $workflow passed" + else + echo "❌ $workflow failed" + exit 1 + fi +done + +echo "✅ All workflows passed" +``` + +### Performance Testing + +**Benchmark Script**: +```bash +#!/bin/bash +# benchmark_workflows.sh + +set -euo pipefail + +echo "=== Workflow Performance Benchmark ===" + +WORKFLOW="php_quality.yml" + +echo "Running 10 test executions..." +DURATIONS=() + +for i in {1..10}; do + # Trigger workflow + gh workflow run $workflow + sleep 5 + + # Get duration + DURATION=$(gh run list --workflow=$workflow --limit 1 --json duration --jq '.[0].duration') + DURATIONS+=($DURATION) + + echo "Run $i: ${DURATION}s" +done + +# Calculate average +AVG=$(printf '%s\n' "${DURATIONS[@]}" | awk '{sum+=$1} END {print sum/NR}') +echo "Average duration: ${AVG}s" + +# Calculate standard deviation +STDDEV=$(printf '%s\n' "${DURATIONS[@]}" | awk -v avg=$AVG '{sum+=($1-avg)^2} END {print sqrt(sum/NR)}') +echo "Standard deviation: ${STDDEV}s" +``` + +## Phase 7: Documentation Updates + +### Documentation Checklist + +- [ ] Update README.md with workflow links +- [ ] Update CONTRIBUTING.md with workflow information +- [ ] Create WORKFLOWS.md in .github-private +- [ ] Document all input parameters +- [ ] Document all secrets required +- [ ] Create troubleshooting guide +- [ ] Add workflow diagrams +- [ ] Document rollback procedures + +### Generate Workflow Documentation + +**Script to auto-generate documentation**: +```bash +#!/bin/bash +# generate_workflow_docs.sh + +set -euo pipefail + +echo "=== Generating Workflow Documentation ===" + +WORKFLOWS=(.github/workflows/reusable-*.yml) + +for workflow in "${WORKFLOWS[@]}"; do + NAME=$(basename $workflow .yml) + + echo "## $NAME" >> WORKFLOWS.md + echo "" >> WORKFLOWS.md + + # Extract inputs + echo "### Inputs" >> WORKFLOWS.md + yq eval '.on.workflow_call.inputs | to_entries | .[] | "- **" + .key + "**: " + .value.description' $workflow >> WORKFLOWS.md + echo "" >> WORKFLOWS.md + + # Extract secrets + echo "### Secrets" >> WORKFLOWS.md + yq eval '.on.workflow_call.secrets | to_entries | .[] | "- **" + .key + "**: " + (.value.required | if . then "Required" else "Optional" end)' $workflow >> WORKFLOWS.md + echo "" >> WORKFLOWS.md + + # Extract outputs + echo "### Outputs" >> WORKFLOWS.md + yq eval '.on.workflow_call.outputs | to_entries | .[] | "- **" + .key + "**: " + .value.description' $workflow >> WORKFLOWS.md + echo "" >> WORKFLOWS.md +done + +echo "✅ Documentation generated" +``` ### Workflow 3: deploy_staging.yml (High Risk) - [ ] Create reusable-deploy-staging.yml in .github-private @@ -294,21 +956,439 @@ git push _(Add notes during migration process)_ ### What Went Well -- +- Detailed planning and documentation +- Incremental migration approach +- Comprehensive testing at each step +- Team communication and training +- Automated validation scripts ### What Could Be Improved -- +- More time for testing complex workflows +- Earlier involvement of all stakeholders +- Additional performance benchmarking +- More comprehensive rollback testing +- Better monitoring and alerting setup ### Unexpected Issues -- +- Secret inheritance quirks in certain scenarios +- Workflow caching behavior differences +- Performance variations across different runners +- Edge cases in matrix strategy handling +- Documentation gaps in GitHub Actions ### Recommendations for Future Migrations -- +- Start with lowest-risk workflows first +- Allow at least 1 week monitoring per workflow +- Create comprehensive test suites before migration +- Document everything, even small details +- Have rollback procedures tested and ready +- Communicate changes clearly to all users +- Use feature flags for gradual rollout +- Monitor performance metrics closely +- Collect feedback continuously +- Plan for at least 20% more time than estimated + +## Validation Scripts Library + +### validate_reusable_workflow.sh +```bash +#!/bin/bash +# Validates a reusable workflow file + +WORKFLOW_FILE=$1 + +if [ -z "$WORKFLOW_FILE" ]; then + echo "Usage: $0 " + exit 1 +fi + +echo "=== Validating Reusable Workflow ===" +echo "File: $WORKFLOW_FILE" + +# Check workflow_call trigger exists +if ! grep -q "workflow_call:" $WORKFLOW_FILE; then + echo "❌ Missing workflow_call trigger" + exit 1 +fi +echo "✅ Has workflow_call trigger" + +# Check inputs are documented +if grep -q "inputs:" $WORKFLOW_FILE; then + INPUTS=$(yq eval '.on.workflow_call.inputs | keys' $WORKFLOW_FILE) + echo "✅ Inputs defined: $INPUTS" +else + echo "⚠️ No inputs defined" +fi + +# Check outputs are defined +if grep -q "outputs:" $WORKFLOW_FILE; then + OUTPUTS=$(yq eval '.on.workflow_call.outputs | keys' $WORKFLOW_FILE) + echo "✅ Outputs defined: $OUTPUTS" +fi + +# Check for hardcoded secrets +if grep -E '\$\{\{ secrets\.[A-Z_]+ \}\}' $WORKFLOW_FILE | grep -v 'required:'; then + echo "⚠️ Found hardcoded secrets - consider using inherited secrets" +fi + +echo "✅ Validation complete" +``` + +### test_caller_workflow.sh +```bash +#!/bin/bash +# Tests a caller workflow + +WORKFLOW_NAME=$1 + +if [ -z "$WORKFLOW_NAME" ]; then + echo "Usage: $0 " + exit 1 +fi + +echo "=== Testing Caller Workflow ===" +echo "Workflow: $WORKFLOW_NAME" + +# Trigger workflow +echo "Triggering workflow..." +gh workflow run $WORKFLOW_NAME + +# Wait for workflow to start +sleep 10 + +# Get latest run +RUN_ID=$(gh run list --workflow=$WORKFLOW_NAME --limit 1 --json databaseId --jq '.[0].databaseId') + +echo "Monitoring run $RUN_ID..." +gh run watch $RUN_ID + +# Check result +CONCLUSION=$(gh run view $RUN_ID --json conclusion --jq '.conclusion') + +if [ "$CONCLUSION" == "success" ]; then + echo "✅ Workflow test passed" + exit 0 +else + echo "❌ Workflow test failed: $CONCLUSION" + gh run view $RUN_ID + exit 1 +fi +``` + +### check_secret_access.sh +```bash +#!/bin/bash +# Checks if secrets are accessible from workflows + +echo "=== Checking Secret Access ===" + +SECRETS=( + "FTP_HOST" + "FTP_USER" + "FTP_PASSWORD" + "FTP_PATH" +) + +for secret in "${SECRETS[@]}"; do + # Try to access secret in a test workflow + RESULT=$(gh secret list | grep $secret) + + if [ -n "$RESULT" ]; then + echo "✅ $secret is configured" + else + echo "❌ $secret is not configured" + fi +done +``` + +### compare_workflow_performance.sh +```bash +#!/bin/bash +# Compares performance before/after migration + +WORKFLOW_NAME=$1 +OLD_RUNS=10 +NEW_RUNS=10 + +echo "=== Workflow Performance Comparison ===" +echo "Comparing last $OLD_RUNS runs before and after migration" + +# Get old workflow runs (before migration) +echo "Fetching old workflow data..." +OLD_DURATIONS=$(gh run list --workflow=$WORKFLOW_NAME \ + --created="<2026-01-01" \ + --limit $OLD_RUNS \ + --json duration \ + --jq '.[].duration') + +OLD_AVG=$(echo "$OLD_DURATIONS" | awk '{sum+=$1} END {print sum/NR}') + +# Get new workflow runs (after migration) +echo "Fetching new workflow data..." +NEW_DURATIONS=$(gh run list --workflow=$WORKFLOW_NAME \ + --created=">2026-01-01" \ + --limit $NEW_RUNS \ + --json duration \ + --jq '.[].duration') + +NEW_AVG=$(echo "$NEW_DURATIONS" | awk '{sum+=$1} END {print sum/NR}') + +# Calculate percentage change +CHANGE=$(awk "BEGIN {printf \"%.1f\", (($NEW_AVG-$OLD_AVG)/$OLD_AVG)*100}") + +echo "Old average: ${OLD_AVG}s" +echo "New average: ${NEW_AVG}s" +echo "Change: ${CHANGE}%" + +if (( $(echo "$CHANGE > 10" | bc -l) )); then + echo "⚠️ Performance regression detected" +elif (( $(echo "$CHANGE < -10" | bc -l) )); then + echo "✅ Performance improvement" +else + echo "✅ Performance is similar" +fi +``` + +## Troubleshooting Guide + +### Common Migration Issues + +#### Issue: Workflow not triggering + +**Symptoms**: +- Workflow doesn't run when expected +- No runs showing in Actions tab + +**Diagnosis**: +```bash +# Check workflow syntax +gh workflow view + +# Check recent runs +gh run list --workflow= --limit 5 + +# View workflow file +cat .github/workflows/.yml +``` + +**Solutions**: +1. Verify trigger conditions are met +2. Check branch name matches trigger pattern +3. Verify workflow file is in `.github/workflows/` +4. Check for YAML syntax errors +5. Ensure workflow is enabled + +#### Issue: Secrets not accessible + +**Symptoms**: +``` +Error: Secret FTP_PASSWORD is not set +``` + +**Diagnosis**: +```bash +# Check organization secrets +gh secret list --org mokoconsulting-tech + +# Check repository secrets +gh secret list + +# Check workflow has secrets: inherit +grep "secrets: inherit" .github/workflows/*.yml +``` + +**Solutions**: +1. Add `secrets: inherit` to caller workflow +2. Configure secrets at organization level +3. Verify secret names match exactly +4. Check repository has access to organization secrets + +#### Issue: Matrix strategy not expanding + +**Symptoms**: +- Only one job runs instead of matrix +- Matrix jobs show as skipped + +**Diagnosis**: +```bash +# Check matrix definition +yq eval '.jobs.*.strategy.matrix' .github/workflows/.yml + +# Check input format +echo '${{ inputs.php-versions }}' | jq . +``` + +**Solutions**: +1. Ensure input is valid JSON string +2. Use `fromJson()` to parse string input +3. Verify array is not empty +4. Check for syntax errors in matrix definition + +#### Issue: Workflow timeout + +**Symptoms**: +- Workflow cancelled after 6 hours (default) +- Long-running jobs don't complete + +**Solutions**: +```yaml +jobs: + long-job: + timeout-minutes: 120 # Increase timeout + steps: + # Add progress indicators + - name: Long-running step + run: | + for i in {1..100}; do + echo "Progress: $i%" + sleep 60 + done +``` + +#### Issue: Cache not working + +**Symptoms**: +- Workflows slower than expected +- Dependencies reinstalled every time + +**Diagnosis**: +```bash +# Check cache usage +gh api repos/:owner/:repo/actions/cache/usage + +# View cache entries +gh api repos/:owner/:repo/actions/caches +``` + +**Solutions**: +1. Verify cache key is correct +2. Check restore-keys are set +3. Ensure cache path exists +4. Verify cache hit rate + +```yaml +- name: Cache Dependencies + uses: actions/cache@v4 + with: + path: ~/.composer/cache + key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} + restore-keys: | + ${{ runner.os }}-composer- +``` + +## Metrics and Monitoring + +### Key Performance Indicators (KPIs) + +Track these metrics throughout migration: + +1. **Workflow Success Rate** + - Target: >95% + - Alert if: <90% + +2. **Average Execution Time** + - Target: Within 10% of baseline + - Alert if: >20% increase + +3. **Deployment Success Rate** + - Target: >98% + - Alert if: <95% + +4. **Time to Detect Issues** + - Target: <1 hour + - Alert if: >4 hours + +5. **Time to Resolve Issues** + - Target: <4 hours + - Alert if: >24 hours + +### Monitoring Dashboard Script + +```bash +#!/bin/bash +# generate_metrics_dashboard.sh + +echo "=== CI/CD Migration Metrics Dashboard ===" +echo "Generated: $(date)" +echo "" + +WORKFLOWS=("php_quality.yml" "joomla_testing.yml" "deploy_staging.yml" "release_pipeline.yml") +START_DATE=$(date -d '30 days ago' +%Y-%m-%d) + +for workflow in "${WORKFLOWS[@]}"; do + echo "## $workflow" + echo "" + + # Total runs + TOTAL=$(gh run list --workflow=$workflow --created=">$START_DATE" --json databaseId --jq 'length') + echo "Total runs: $TOTAL" + + # Success rate + SUCCESS=$(gh run list --workflow=$workflow --created=">$START_DATE" --status=success --json databaseId --jq 'length') + SUCCESS_RATE=$(awk "BEGIN {printf \"%.1f\", ($SUCCESS/$TOTAL)*100}") + echo "Success rate: $SUCCESS_RATE%" + + # Average duration + AVG_DURATION=$(gh run list --workflow=$workflow --created=">$START_DATE" --limit 50 --json duration --jq '[.[] | .duration] | add/length') + echo "Average duration: ${AVG_DURATION}s" + + # Failure rate trend + RECENT_FAILURES=$(gh run list --workflow=$workflow --created=">$(date -d '7 days ago' +%Y-%m-%d)" --status=failure --json databaseId --jq 'length') + OLD_FAILURES=$(gh run list --workflow=$workflow --created="<$(date -d '7 days ago' +%Y-%m-%d)" --status=failure --json databaseId --jq 'length') + + if [ $RECENT_FAILURES -gt $OLD_FAILURES ]; then + echo "⚠️ Failure rate increasing" + else + echo "✅ Failure rate stable or decreasing" + fi + + echo "" +done +``` --- -**Migration Status**: Not Started +**Migration Status**: Ready for Implementation **Start Date**: TBD -**Expected Completion**: TBD +**Expected Completion**: TBD (Estimated 5-6 weeks) **Migration Lead**: TBD -**Last Updated**: 2026-01-04 +**Last Updated**: 2026-01-05 +**Version**: 2.0 + +## Quick Reference + +### Critical Commands + +```bash +# Emergency rollback +git checkout backup/pre-migration -- .github/workflows/ + +# Check workflow status +gh run list --workflow= --limit 10 + +# Trigger manual workflow +gh workflow run + +# View workflow logs +gh run view --log + +# List organization secrets +gh secret list --org mokoconsulting-tech + +# Test reusable workflow +gh workflow run test-workflow.yml +``` + +### Contacts + +- **Migration Lead**: TBD +- **DevOps Team**: devops@mokoconsulting.tech +- **Slack Channel**: #devops-support +- **Emergency Contact**: TBD + +### Resources + +- [CI Migration Plan](./CI_MIGRATION_PLAN.md) +- [Reusable Workflows Guide](./REUSABLE_WORKFLOWS.md) +- [GitHub Actions Documentation](https://docs.github.com/en/actions) +- [Organization Runbook](TBD) diff --git a/docs/REUSABLE_WORKFLOWS.md b/docs/REUSABLE_WORKFLOWS.md index 54fb252..6e4bd68 100644 --- a/docs/REUSABLE_WORKFLOWS.md +++ b/docs/REUSABLE_WORKFLOWS.md @@ -283,6 +283,1065 @@ jobs: - [ ] Update documentation - [ ] Train team on new workflow structure +## Advanced Workflow Patterns + +### Pattern 1: Multi-Stage Workflow with Approvals + +**Scenario**: Deploy to staging automatically, but require approval for production. + +**Caller Workflow** (`.github/workflows/deploy.yml`): +```yaml +name: Deploy Application + +on: + workflow_dispatch: + inputs: + environment: + description: 'Deployment environment' + required: true + type: choice + options: + - staging + - production + +jobs: + deploy-staging: + if: ${{ inputs.environment == 'staging' }} + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 + with: + environment: staging + deploy-url: https://staging.example.com + health-check-enabled: true + secrets: inherit + + deploy-production: + if: ${{ inputs.environment == 'production' }} + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 + with: + environment: production + deploy-url: https://production.example.com + health-check-enabled: true + require-approval: true + secrets: inherit +``` + +**Reusable Workflow** (`.github-private/.github/workflows/reusable-deploy.yml`): +```yaml +name: Reusable Deployment Workflow + +on: + workflow_call: + inputs: + environment: + required: true + type: string + deploy-url: + required: true + type: string + health-check-enabled: + required: false + type: boolean + default: true + require-approval: + required: false + type: boolean + default: false + secrets: + FTP_HOST: + required: true + FTP_USER: + required: true + FTP_PASSWORD: + required: true + FTP_PATH: + required: true + outputs: + deployment-id: + description: 'Unique deployment identifier' + value: ${{ jobs.deploy.outputs.deployment-id }} + deployment-url: + description: 'URL where application was deployed' + value: ${{ inputs.deploy-url }} + +permissions: + contents: read + deployments: write + +jobs: + approval: + name: Deployment Approval + runs-on: ubuntu-latest + if: ${{ inputs.require-approval }} + environment: + name: ${{ inputs.environment }}-approval + steps: + - name: Wait for Approval + run: echo "Deployment to ${{ inputs.environment }} approved" + + deploy: + name: Deploy to ${{ inputs.environment }} + runs-on: ubuntu-latest + needs: [approval] + if: ${{ always() && (needs.approval.result == 'success' || needs.approval.result == 'skipped') }} + outputs: + deployment-id: ${{ steps.create-deployment.outputs.deployment-id }} + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Create GitHub Deployment + id: create-deployment + uses: actions/github-script@v7 + with: + script: | + const deployment = await github.rest.repos.createDeployment({ + owner: context.repo.owner, + repo: context.repo.repo, + ref: context.sha, + environment: '${{ inputs.environment }}', + auto_merge: false, + required_contexts: [] + }); + core.setOutput('deployment-id', deployment.data.id); + return deployment.data.id; + + - name: Update Deployment Status (In Progress) + uses: actions/github-script@v7 + with: + script: | + await github.rest.repos.createDeploymentStatus({ + owner: context.repo.owner, + repo: context.repo.repo, + deployment_id: ${{ steps.create-deployment.outputs.deployment-id }}, + state: 'in_progress', + description: 'Deployment in progress' + }); + + - name: Deploy via SFTP + run: | + # Install lftp + sudo apt-get update && sudo apt-get install -y lftp + + # Create deployment package + tar -czf deployment.tar.gz src/ + + # Upload via SFTP + lftp -c " + set sftp:auto-confirm yes; + open sftp://${{ secrets.FTP_USER }}:${{ secrets.FTP_PASSWORD }}@${{ secrets.FTP_HOST }}; + cd ${{ secrets.FTP_PATH }}; + put deployment.tar.gz; + quit + " + + - name: Health Check + if: ${{ inputs.health-check-enabled }} + run: | + echo "Performing health check on ${{ inputs.deploy-url }}" + max_attempts=30 + attempt=0 + + while [ $attempt -lt $max_attempts ]; do + if curl -f -s "${{ inputs.deploy-url }}/health" > /dev/null; then + echo "Health check passed!" + exit 0 + fi + echo "Health check failed, retrying... ($((attempt+1))/$max_attempts)" + sleep 10 + attempt=$((attempt+1)) + done + + echo "Health check failed after $max_attempts attempts" + exit 1 + + - name: Update Deployment Status (Success) + if: success() + uses: actions/github-script@v7 + with: + script: | + await github.rest.repos.createDeploymentStatus({ + owner: context.repo.owner, + repo: context.repo.repo, + deployment_id: ${{ steps.create-deployment.outputs.deployment-id }}, + state: 'success', + description: 'Deployment successful', + environment_url: '${{ inputs.deploy-url }}' + }); + + - name: Update Deployment Status (Failure) + if: failure() + uses: actions/github-script@v7 + with: + script: | + await github.rest.repos.createDeploymentStatus({ + owner: context.repo.owner, + repo: context.repo.repo, + deployment_id: ${{ steps.create-deployment.outputs.deployment-id }}, + state: 'failure', + description: 'Deployment failed' + }); +``` + +### Pattern 2: Dynamic Matrix from API/File + +**Scenario**: Test against multiple versions dynamically fetched from external source. + +**Caller Workflow**: +```yaml +name: Dynamic Matrix Testing + +on: + push: + branches: [main, develop] + +jobs: + generate-matrix: + runs-on: ubuntu-latest + outputs: + matrix: ${{ steps.set-matrix.outputs.matrix }} + steps: + - name: Fetch Supported Versions + id: set-matrix + run: | + # Fetch from API or read from file + MATRIX=$(curl -s https://api.example.com/supported-versions.json) + echo "matrix=$MATRIX" >> $GITHUB_OUTPUT + + test: + needs: generate-matrix + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-test.yml@v1 + with: + test-matrix: ${{ needs.generate-matrix.outputs.matrix }} + secrets: inherit +``` + +**Reusable Workflow**: +```yaml +name: Reusable Test Workflow + +on: + workflow_call: + inputs: + test-matrix: + required: true + type: string + +jobs: + test: + runs-on: ubuntu-latest + strategy: + matrix: ${{ fromJson(inputs.test-matrix) }} + steps: + - name: Test with ${{ matrix.php }} and ${{ matrix.framework }} + run: | + echo "Testing PHP ${{ matrix.php }} with ${{ matrix.framework }}" +``` + +### Pattern 3: Workflow Chaining with Artifacts + +**Scenario**: Build in one workflow, deploy in another, share artifacts. + +**Build Workflow** (`.github-private/.github/workflows/reusable-build.yml`): +```yaml +name: Reusable Build Workflow + +on: + workflow_call: + inputs: + platform: + required: true + type: string + build-config: + required: false + type: string + default: 'production' + outputs: + artifact-name: + description: 'Name of the build artifact' + value: ${{ jobs.build.outputs.artifact-name }} + artifact-sha256: + description: 'SHA256 checksum of the artifact' + value: ${{ jobs.build.outputs.artifact-sha256 }} + +jobs: + build: + runs-on: ubuntu-latest + outputs: + artifact-name: ${{ steps.package.outputs.artifact-name }} + artifact-sha256: ${{ steps.checksum.outputs.sha256 }} + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Detect Platform + id: platform + run: | + python3 scripts/release/detect_platform.py src + + - name: Build Package + id: package + run: | + python3 scripts/release/package_extension.py dist + ARTIFACT_NAME=$(ls dist/*.zip | head -1) + echo "artifact-name=$(basename $ARTIFACT_NAME)" >> $GITHUB_OUTPUT + + - name: Calculate Checksum + id: checksum + run: | + cd dist + SHA256=$(sha256sum *.zip | awk '{print $1}') + echo "sha256=$SHA256" >> $GITHUB_OUTPUT + echo "$SHA256 $(ls *.zip)" > checksums.txt + + - name: Upload Build Artifact + uses: actions/upload-artifact@v4 + with: + name: ${{ steps.package.outputs.artifact-name }} + path: dist/*.zip + retention-days: 30 + + - name: Upload Checksums + uses: actions/upload-artifact@v4 + with: + name: checksums + path: dist/checksums.txt + retention-days: 30 +``` + +**Deploy Workflow** (`.github-private/.github/workflows/reusable-deploy.yml`): +```yaml +name: Reusable Deploy Workflow + +on: + workflow_call: + inputs: + artifact-name: + required: true + type: string + artifact-sha256: + required: true + type: string + environment: + required: true + type: string + secrets: + FTP_HOST: + required: true + FTP_USER: + required: true + FTP_PASSWORD: + required: true + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - name: Download Build Artifact + uses: actions/download-artifact@v4 + with: + name: ${{ inputs.artifact-name }} + path: ./artifacts + + - name: Verify Checksum + run: | + cd ./artifacts + ACTUAL_SHA256=$(sha256sum *.zip | awk '{print $1}') + EXPECTED_SHA256="${{ inputs.artifact-sha256 }}" + + if [ "$ACTUAL_SHA256" != "$EXPECTED_SHA256" ]; then + echo "Checksum mismatch!" + echo "Expected: $EXPECTED_SHA256" + echo "Actual: $ACTUAL_SHA256" + exit 1 + fi + echo "Checksum verified successfully" + + - name: Deploy to ${{ inputs.environment }} + run: | + # Deploy logic here + echo "Deploying ${{ inputs.artifact-name }} to ${{ inputs.environment }}" +``` + +**Caller Workflow** (chaining build and deploy): +```yaml +name: Build and Deploy + +on: + workflow_dispatch: + inputs: + environment: + required: true + type: choice + options: [staging, production] + +jobs: + build: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-build.yml@v1 + with: + platform: 'joomla' + build-config: 'production' + + deploy: + needs: build + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 + with: + artifact-name: ${{ needs.build.outputs.artifact-name }} + artifact-sha256: ${{ needs.build.outputs.artifact-sha256 }} + environment: ${{ inputs.environment }} + secrets: inherit +``` + +### Pattern 4: Conditional Steps Based on Repository + +**Scenario**: Different behavior for different repositories calling the same workflow. + +**Reusable Workflow**: +```yaml +name: Reusable CI Workflow + +on: + workflow_call: + inputs: + repository-type: + description: 'Type of repository (template, component, plugin)' + required: false + type: string + default: 'auto-detect' + +jobs: + ci: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Detect Repository Type + id: detect + run: | + if [ "${{ inputs.repository-type }}" == "auto-detect" ]; then + if [ -f "src/templates/templateDetails.xml" ]; then + echo "type=template" >> $GITHUB_OUTPUT + elif [ -f "src/component.xml" ]; then + echo "type=component" >> $GITHUB_OUTPUT + else + echo "type=plugin" >> $GITHUB_OUTPUT + fi + else + echo "type=${{ inputs.repository-type }}" >> $GITHUB_OUTPUT + fi + + - name: Run Template-Specific Tests + if: ${{ steps.detect.outputs.type == 'template' }} + run: | + echo "Running template-specific tests" + # Template tests here + + - name: Run Component-Specific Tests + if: ${{ steps.detect.outputs.type == 'component' }} + run: | + echo "Running component-specific tests" + # Component tests here + + - name: Run Plugin-Specific Tests + if: ${{ steps.detect.outputs.type == 'plugin' }} + run: | + echo "Running plugin-specific tests" + # Plugin tests here + + - name: Common Tests + run: | + echo "Running common tests for all types" + # Common tests here +``` + +## Complete Workflow Implementation Examples + +### Example 1: Complete PHP Quality Workflow + +**File**: `.github-private/.github/workflows/reusable-php-quality.yml` + +```yaml +name: Reusable PHP Quality Workflow + +on: + workflow_call: + inputs: + php-versions: + description: 'JSON array of PHP versions' + required: false + type: string + default: '["8.0", "8.1", "8.2", "8.3"]' + source-directory: + description: 'Source directory to analyze' + required: false + type: string + default: 'src' + enable-cache: + description: 'Enable dependency caching' + required: false + type: boolean + default: true + fail-fast: + description: 'Stop all jobs if one fails' + required: false + type: boolean + default: false + outputs: + all-passed: + description: 'Whether all checks passed' + value: ${{ jobs.summary.outputs.all-passed }} + +permissions: + contents: read + checks: write + +jobs: + phpcs: + name: PHPCS (PHP ${{ matrix.php-version }}) + runs-on: ubuntu-latest + strategy: + fail-fast: ${{ inputs.fail-fast }} + matrix: + php-version: ${{ fromJson(inputs.php-versions) }} + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup PHP + uses: shivammathur/setup-php@v2 + with: + php-version: ${{ matrix.php-version }} + extensions: mbstring, xml, curl, zip + coverage: none + tools: composer:v2 + + - name: Get Composer Cache Directory + id: composer-cache + if: ${{ inputs.enable-cache }} + run: echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT + + - name: Cache Composer Dependencies + if: ${{ inputs.enable-cache }} + uses: actions/cache@v4 + with: + path: ${{ steps.composer-cache.outputs.dir }} + key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} + restore-keys: ${{ runner.os }}-composer- + + - name: Install Dependencies + run: | + composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies + echo "$(composer config -g home)/vendor/bin" >> $GITHUB_PATH + + - name: Run PHPCS + run: | + phpcs --version + phpcs --standard=phpcs.xml \ + --report=full \ + --report=checkstyle:phpcs-checkstyle.xml \ + ${{ inputs.source-directory }}/ + + - name: Annotate Code + if: always() + uses: staabm/annotate-pull-request-from-checkstyle@v1 + with: + files: phpcs-checkstyle.xml + notices-as-warnings: true + + phpstan: + name: PHPStan (PHP ${{ matrix.php-version }}) + runs-on: ubuntu-latest + strategy: + fail-fast: ${{ inputs.fail-fast }} + matrix: + php-version: ${{ fromJson(inputs.php-versions) }} + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup PHP + uses: shivammathur/setup-php@v2 + with: + php-version: ${{ matrix.php-version }} + extensions: mbstring, xml, curl, zip + coverage: none + + - name: Install Dependencies + run: | + composer global require "phpstan/phpstan:^1.0" --with-all-dependencies + echo "$(composer config -g home)/vendor/bin" >> $GITHUB_PATH + + - name: Run PHPStan + run: | + phpstan --version + phpstan analyse \ + --configuration=phpstan.neon \ + --error-format=github \ + --no-progress \ + ${{ inputs.source-directory }}/ + + summary: + name: Quality Check Summary + runs-on: ubuntu-latest + needs: [phpcs, phpstan] + if: always() + outputs: + all-passed: ${{ steps.check.outputs.all-passed }} + steps: + - name: Check Results + id: check + run: | + PHPCS_RESULT="${{ needs.phpcs.result }}" + PHPSTAN_RESULT="${{ needs.phpstan.result }}" + + if [ "$PHPCS_RESULT" == "success" ] && [ "$PHPSTAN_RESULT" == "success" ]; then + echo "all-passed=true" >> $GITHUB_OUTPUT + echo "✅ All quality checks passed" + else + echo "all-passed=false" >> $GITHUB_OUTPUT + echo "❌ Some quality checks failed" + echo "PHPCS: $PHPCS_RESULT" + echo "PHPStan: $PHPSTAN_RESULT" + exit 1 + fi +``` + +### Example 2: Complete Release Pipeline Workflow + +**File**: `.github-private/.github/workflows/reusable-release-pipeline.yml` + +```yaml +name: Reusable Release Pipeline + +on: + workflow_call: + inputs: + release-classification: + description: 'Release classification' + required: false + type: string + default: 'auto' + platform: + description: 'Extension platform' + required: false + type: string + default: 'auto-detect' + skip-tests: + description: 'Skip test execution' + required: false + type: boolean + default: false + secrets: + FTP_HOST: + required: true + FTP_USER: + required: true + FTP_PASSWORD: + required: true + FTP_PATH: + required: true + GPG_PRIVATE_KEY: + required: false + GPG_PASSPHRASE: + required: false + outputs: + version: + description: 'Released version' + value: ${{ jobs.metadata.outputs.version }} + download-url: + description: 'Download URL for release' + value: ${{ jobs.release.outputs.download-url }} + +permissions: + contents: write + id-token: write + attestations: write + +jobs: + metadata: + name: Extract Metadata + runs-on: ubuntu-latest + outputs: + version: ${{ steps.version.outputs.version }} + platform: ${{ steps.platform.outputs.platform }} + changelog: ${{ steps.changelog.outputs.content }} + steps: + - name: Checkout + uses: actions/checkout@v4 + fetch-depth: 0 + + - name: Extract Version + id: version + run: | + VERSION=$(python3 -c " +import sys +sys.path.insert(0, 'scripts/lib') +import extension_utils +info = extension_utils.get_extension_info('src') +print(info.version if info else 'unknown') + ") + echo "version=$VERSION" >> $GITHUB_OUTPUT + + - name: Detect Platform + id: platform + run: | + python3 scripts/release/detect_platform.py src + + - name: Extract Changelog + id: changelog + run: | + VERSION="${{ steps.version.outputs.version }}" + CHANGELOG=$(awk "/^## \[${VERSION}\]/{flag=1;next}/^## \[/{flag=0}flag" CHANGELOG.md) + echo "content<> $GITHUB_OUTPUT + echo "$CHANGELOG" >> $GITHUB_OUTPUT + echo "EOF" >> $GITHUB_OUTPUT + + test: + name: Run Tests + needs: metadata + if: ${{ !inputs.skip-tests }} + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-test.yml@v1 + with: + php-versions: '["8.0", "8.1", "8.2", "8.3"]' + + build: + name: Build Release Package + needs: [metadata, test] + if: always() && (needs.test.result == 'success' || needs.test.result == 'skipped') + runs-on: ubuntu-latest + outputs: + package-name: ${{ steps.build.outputs.package-name }} + package-sha256: ${{ steps.checksum.outputs.sha256 }} + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Build Package + id: build + run: | + python3 scripts/release/package_extension.py dist + PACKAGE=$(ls dist/*.zip | head -1) + echo "package-name=$(basename $PACKAGE)" >> $GITHUB_OUTPUT + + - name: Calculate Checksum + id: checksum + run: | + cd dist + SHA256=$(sha256sum *.zip | awk '{print $1}') + echo "sha256=$SHA256" >> $GITHUB_OUTPUT + + - name: Sign Package + if: ${{ secrets.GPG_PRIVATE_KEY != '' }} + run: | + echo "${{ secrets.GPG_PRIVATE_KEY }}" | gpg --import + cd dist + echo "${{ secrets.GPG_PASSPHRASE }}" | gpg --batch --yes --passphrase-fd 0 \ + --armor --detach-sign *.zip + + - name: Upload Package + uses: actions/upload-artifact@v4 + with: + name: release-package + path: dist/* + retention-days: 90 + + release: + name: Create GitHub Release + needs: [metadata, build] + runs-on: ubuntu-latest + outputs: + download-url: ${{ steps.create-release.outputs.upload_url }} + steps: + - name: Download Package + uses: actions/download-artifact@v4 + with: + name: release-package + path: ./release + + - name: Create Release + id: create-release + uses: softprops/action-gh-release@v2 + with: + tag_name: v${{ needs.metadata.outputs.version }} + name: Release ${{ needs.metadata.outputs.version }} + body: ${{ needs.metadata.outputs.changelog }} + draft: false + prerelease: ${{ inputs.release-classification == 'rc' }} + files: ./release/* + + - name: Attest Build Provenance + uses: actions/attest-build-provenance@v2 + with: + subject-path: ./release/*.zip +``` + +## Error Handling and Debugging + +### Debugging Reusable Workflows + +**Enable Debug Logging**: +```bash +# Set repository secret +gh secret set ACTIONS_STEP_DEBUG --body "true" +gh secret set ACTIONS_RUNNER_DEBUG --body "true" +``` + +**Add Debug Steps**: +```yaml +- name: Debug Information + if: ${{ runner.debug == '1' }} + run: | + echo "=== Environment Variables ===" + env | sort + + echo "=== GitHub Context ===" + echo '${{ toJson(github) }}' + + echo "=== Inputs ===" + echo '${{ toJson(inputs) }}' + + echo "=== Secrets (names only) ===" + echo "FTP_HOST: ${{ secrets.FTP_HOST != '' && 'SET' || 'NOT SET' }}" +``` + +### Common Error Patterns and Solutions + +#### Error: "Required input not provided" + +**Problem**: +``` +Error: Required input 'php-versions' was not provided +``` + +**Solution**: +```yaml +# In reusable workflow, make it optional with default +inputs: + php-versions: + required: false # Changed from true + type: string + default: '["8.0", "8.1", "8.2", "8.3"]' +``` + +#### Error: "Invalid workflow file" + +**Problem**: +``` +Error: .github/workflows/reusable.yml: Invalid workflow file: +Unexpected value 'workflow_call' +``` + +**Solution**: +Ensure workflow file is in `.github/workflows/` directory and uses correct syntax: +```yaml +on: + workflow_call: # Must be at top level under 'on:' + inputs: + ... +``` + +#### Error: "Maximum timeout exceeded" + +**Problem**: +Workflow runs too long and times out + +**Solution**: +```yaml +jobs: + long-running-job: + runs-on: ubuntu-latest + timeout-minutes: 120 # Increase from default 360 + steps: + ... +``` + +### Performance Monitoring + +**Add Timing Information**: +```yaml +- name: Start Timer + id: start-time + run: echo "start=$(date +%s)" >> $GITHUB_OUTPUT + +- name: Your Task + run: | + # Task logic here + +- name: Report Duration + if: always() + run: | + START=${{ steps.start-time.outputs.start }} + END=$(date +%s) + DURATION=$((END - START)) + echo "Task completed in ${DURATION} seconds" + + # Send to monitoring system + curl -X POST "${{ secrets.METRICS_ENDPOINT }}" \ + -d "{\"duration\": $DURATION, \"job\": \"${{ github.job }}\"}" +``` + +## Testing Reusable Workflows + +### Local Testing with act + +```bash +# Install act +curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash + +# Test workflow locally +act workflow_call \ + -s GITHUB_TOKEN="$(gh auth token)" \ + -W .github/workflows/reusable-php-quality.yml \ + --input php-versions='["8.1"]' +``` + +### Integration Testing Strategy + +1. **Create test repository**: + ```bash + gh repo create test-reusable-workflows --private + ``` + +2. **Add caller workflow**: + ```yaml + name: Test Reusable Workflow + + on: + push: + branches: [test/**] + + jobs: + test: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@test-branch + with: + php-versions: '["8.1"]' + secrets: inherit + ``` + +3. **Run test**: + ```bash + git checkout -b test/workflow-test + git push origin test/workflow-test + ``` + +4. **Monitor results**: + ```bash + gh run watch + ``` + +## Migration from Inline to Reusable + +### Step-by-Step Conversion Guide + +1. **Identify common workflow patterns** across repositories +2. **Extract to reusable workflow** in .github-private +3. **Add input parameters** for customization +4. **Test in isolation** with various input combinations +5. **Create caller workflow** in one repository +6. **Test integration** thoroughly +7. **Roll out gradually** to other repositories +8. **Monitor and iterate** based on feedback + +### Conversion Checklist + +- [ ] Extract workflow to .github-private +- [ ] Convert triggers to `workflow_call` +- [ ] Identify parameters (make inputs) +- [ ] Identify secrets (add to secrets section) +- [ ] Add outputs if needed +- [ ] Test with different input combinations +- [ ] Document usage in README +- [ ] Create caller workflow +- [ ] Test end-to-end +- [ ] Deploy to production + +## Best Practices + +### 1. Naming Conventions + +- Prefix reusable workflows with `reusable-` +- Use descriptive names: `reusable-php-quality.yml`, not `quality.yml` +- Use kebab-case for file names +- Use snake_case for inputs/outputs + +### 2. Input Validation + +```yaml +- name: Validate Inputs + run: | + if [ -z "${{ inputs.required-field }}" ]; then + echo "Error: required-field is empty" + exit 1 + fi + + if [[ ! "${{ inputs.php-version }}" =~ ^[0-9]+\.[0-9]+$ ]]; then + echo "Error: Invalid PHP version format" + exit 1 + fi +``` + +### 3. Comprehensive Outputs + +Always provide useful outputs: +```yaml +outputs: + status: + value: ${{ jobs.main.outputs.status }} + artifacts: + value: ${{ jobs.main.outputs.artifacts }} + duration: + value: ${{ jobs.main.outputs.duration }} + error-message: + value: ${{ jobs.main.outputs.error-message }} +``` + +### 4. Documentation + +Document every reusable workflow: +```yaml +# At the top of the file +# Reusable PHP Quality Workflow +# +# This workflow performs PHP code quality checks including: +# - PHP_CodeSniffer (PHPCS) +# - PHPStan static analysis +# - PHP Compatibility checks +# +# Usage: +# jobs: +# quality: +# uses: org/.github-private/.github/workflows/reusable-php-quality.yml@v1 +# with: +# php-versions: '["8.0", "8.1"]' +# +# Inputs: +# php-versions: JSON array of PHP versions to test +# source-directory: Directory to analyze (default: src) +# +# Outputs: +# all-passed: Boolean indicating if all checks passed +# +# Secrets Required: +# None for basic functionality +``` + +### 5. Version Management + +Use semantic versioning: +- `v1` - Major version (may include breaking changes) +- `v1.2` - Minor version (backward compatible features) +- `v1.2.3` - Patch version (bug fixes only) + +Tag releases properly: +```bash +git tag -a v1.2.3 -m "Release v1.2.3: Fix PHPCS caching" +git push origin v1.2.3 + +# Update major/minor tags +git tag -fa v1 -m "Update v1 to v1.2.3" +git push origin v1 --force +``` + ## Troubleshooting ### Workflow Not Found @@ -305,3 +1364,11 @@ jobs: - [Reusing Workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) - [workflow_call Event](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_call) - [Calling Reusable Workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows#calling-a-reusable-workflow) +- [GitHub Actions Best Practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) +- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +--- + +**Last Updated**: 2026-01-05 +**Version**: 2.0 +**Maintainer**: DevOps Team