diff --git a/docs/CI_MIGRATION_PLAN.md b/docs/CI_MIGRATION_PLAN.md deleted file mode 100644 index 1b2152e..0000000 --- a/docs/CI_MIGRATION_PLAN.md +++ /dev/null @@ -1,1233 +0,0 @@ -# CI/CD Migration to Centralized Repositories - -## Purpose - -This document outlines the plan and preparation steps for migrating CI/CD workflows to centralized repositories. The organization uses a **dual-repository architecture**: - -### Repository Architecture - -**`.github-private`** (Private & Secure Centralization) -- **Purpose**: Sensitive/proprietary workflows and deployment logic -- **Visibility**: Private (organization members only) -- **Content**: Deployment workflows, release pipelines, sensitive scripts -- **Use Case**: Production deployments, security-sensitive operations - -**`MokoStandards`** (Public Centralization) -- **Purpose**: Public/shareable workflows and best practices -- **Visibility**: Public (community accessible) -- **Content**: Quality checks, testing workflows, public templates -- **Use Case**: Open-source projects, community contributions, standardization - -This dual approach provides: - -- **Security**: Keep sensitive CI/CD logic and configurations private in `.github-private` -- **Community**: Share public workflows and standards via `MokoStandards` -- **Reusability**: Share common workflows across multiple repositories -- **Maintainability**: Centralize CI/CD updates in one location per type -- **Organization**: Separate CI/CD infrastructure from application code -- **Flexibility**: Choose appropriate repository based on workflow sensitivity - -## Current State - -### Workflows in `.github/workflows/` -1. `php_quality.yml` - PHP code quality checks (PHPCS, PHPStan, PHP Compatibility) -2. `release_pipeline.yml` - Release and build pipeline -3. `ci.yml` - Continuous integration checks -4. `repo_health.yml` - Repository health monitoring -5. `version_branch.yml` - Version branch management -6. `joomla_testing.yml` - Joomla-specific testing -7. `deploy_staging.yml` - Staging deployment - -### Scripts in `scripts/` -- `scripts/lib/` - Shared Python libraries (common.py, extension_utils.py, joomla_manifest.py) -- `scripts/release/` - Release automation scripts -- `scripts/validate/` - Validation scripts -- `scripts/run/` - Runtime scripts - -## Migration Strategy - -### Phase 1: Preparation (Current) - -**Files to Keep in Main Repository:** -- Simple trigger workflows that call reusable workflows -- Repository-specific configuration files -- Application scripts that are part of the product - -**Files to Move to Centralized Repositories:** - -**To `.github-private` (Private):** -- Deployment workflows (deploy_staging.yml, production deployments) -- Release pipelines with sensitive logic (release_pipeline.yml) -- Workflows containing proprietary business logic -- Scripts with deployment credentials handling - -**To `MokoStandards` (Public):** -- Quality check workflows (php_quality.yml) -- Testing workflows (joomla_testing.yml) -- Public CI/CD templates and examples -- Shared utility scripts (extension_utils.py, common.py) - -### Phase 2: Structure Setup - -Create both centralized repositories with appropriate structure: - -**`.github-private` Repository Structure:** -``` -.github-private/ -├── .github/ -│ └── workflows/ -│ ├── reusable-release-pipeline.yml (sensitive) -│ ├── reusable-deploy-staging.yml (sensitive) -│ └── reusable-deploy-production.yml (sensitive) -├── scripts/ -│ ├── deployment/ -│ │ ├── deploy.sh -│ │ └── rollback.sh -│ └── release/ -│ └── publish.py -└── docs/ - ├── WORKFLOWS.md - └── DEPLOYMENT.md -``` - -**`MokoStandards` Repository Structure:** -``` -MokoStandards/ -├── .github/ -│ └── workflows/ -│ ├── reusable-php-quality.yml (public) -│ ├── reusable-joomla-testing.yml (public) -│ ├── reusable-dolibarr-testing.yml (public) -│ └── reusable-security-scan.yml (public) -├── scripts/ -│ ├── shared/ -│ │ ├── extension_utils.py -│ │ ├── common.py -│ │ └── validators/ -│ └── templates/ -│ ├── workflow-templates/ -│ └── action-templates/ -└── docs/ - ├── STANDARDS.md - ├── WORKFLOWS.md - └── CONTRIBUTING.md -``` - -### Phase 3: Conversion - -**Main Repository Workflows (Simplified Callers):** - -**Example 1: Public Quality Workflow** (calls `MokoStandards`) - -`php_quality.yml`: -```yaml -name: PHP Code Quality - -on: - pull_request: - branches: [ main, dev/*, rc/* ] - push: - branches: [ main, dev/*, rc/* ] - -jobs: - quality: - uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - secrets: inherit -``` - -**Example 2: Private Deployment Workflow** (calls `.github-private`) - -`deploy.yml`: -```yaml -name: Deploy to Staging - -on: - workflow_dispatch: - inputs: - environment: - required: true - type: choice - options: [staging, production] - -jobs: - deploy: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@main - with: - environment: ${{ inputs.environment }} - platform: 'joomla' - secrets: inherit -``` - -**Centralized Reusable Workflow Examples:** - -**In `MokoStandards` (Public):** - -Located in `MokoStandards/.github/workflows/reusable-php-quality.yml`: -```yaml -name: Reusable PHP Quality Workflow - -on: - workflow_call: - inputs: - php-versions: - required: false - type: string - default: '["8.0", "8.1", "8.2", "8.3"]' - -jobs: - # Full implementation here -``` - -**In `.github-private` (Private):** - -Located in `.github-private/.github/workflows/reusable-deploy.yml`: -```yaml -name: Reusable Deployment Workflow - -on: - workflow_call: - inputs: - environment: - required: true - type: string - secrets: - FTP_SERVER: - required: true - FTP_USER: - required: true - -jobs: - # Deployment logic here -``` - -### Phase 4: Migration Steps - -1. **Create both centralized repositories** - - **`.github-private`**: Private repository (mokoconsulting-tech/.github-private) - - **`MokoStandards`**: Public repository (mokoconsulting-tech/MokoStandards) - - Initialize each with README and LICENSE - - Set up appropriate branch protection rules - - Configure access: private (team only) and public (community) - -2. **Categorize and move workflows** - - **Sensitive workflows → `.github-private`**: - - deployment workflows, release pipelines - - Convert to reusable workflows with `workflow_call` triggers - - Add security controls and audit logging - - - **Public workflows → `MokoStandards`**: - - quality checks, testing workflows - - Add comprehensive documentation and examples - - Enable community contributions - - - Test each workflow in isolation - - Add proper input parameters and secrets handling - -3. **Update main repository workflows** - - Replace with simplified caller workflows pointing to appropriate repository - - Update documentation with new workflow references - - Test integration end-to-end - -4. **Migrate shared scripts** - - **Deployment/sensitive scripts → `.github-private/scripts/`** - - **Public utilities → `MokoStandards/scripts/shared/`** - - Keep product-specific scripts in main repo - - Update import paths and references - -5. **Update documentation** - - Document workflow calling conventions - - Update development guides - - Create troubleshooting guides - -## Configuration Requirements - -### Secrets to Configure - -**In .github-private repository:** -- Deployment credentials (FTP_SERVER, FTP_USER, FTP_KEY, etc.) -- API tokens for external services -- Signing keys - -**In main repository:** -- Inherit secrets from organization level -- Repository-specific overrides only - -### Variables to Configure - -**Organization-level variables:** -- DEPLOY_DRY_RUN -- FTP_PATH_SUFFIX -- PHP_VERSIONS (default) - -**Repository-level variables:** -- Repository-specific configurations -- Feature flags - -## Workflow Categorization - -### Workflows to Centralize - -#### To `MokoStandards` (Public Repository) - -1. **php_quality.yml** ✓ - - Reason: Standardized quality checks across all PHP projects - - Type: Reusable workflow - - Sensitivity: Low (no secrets, publicly shareable) - - Target: `MokoStandards/.github/workflows/reusable-php-quality.yml` - -2. **joomla_testing.yml** ✓ - - Reason: Shared across Joomla projects, community benefit - - Type: Reusable workflow - - Sensitivity: Low (testing patterns, no sensitive data) - - Target: `MokoStandards/.github/workflows/reusable-joomla-testing.yml` - -3. **ci.yml** (partially) - - Reason: Generic CI patterns can be shared - - Type: Reusable workflow template - - Sensitivity: Low (standard CI practices) - - Target: `MokoStandards/.github/workflows/reusable-ci-base.yml` - -#### To `.github-private` (Private Repository) - -1. **release_pipeline.yml** ✓ - - Reason: Complex release logic, contains sensitive deployment steps - - Type: Reusable workflow - - Sensitivity: High (deployment credentials, business logic) - - Target: `.github-private/.github/workflows/reusable-release-pipeline.yml` - -2. **deploy_staging.yml** ✓ - - Reason: Contains deployment credentials and proprietary logic - - Type: Reusable workflow - - Sensitivity: High (FTP credentials, server details) - - Target: `.github-private/.github/workflows/reusable-deploy-staging.yml` - -3. **deploy_production.yml** ✓ - - Reason: Production deployment with strict security requirements - - Type: Reusable workflow - - Sensitivity: Critical (production access) - - Target: `.github-private/.github/workflows/reusable-deploy-production.yml` - -### Workflows to Keep Local (Main Repository) - -1. **ci.yml** (project-specific parts) - - Reason: Repository-specific CI steps - - Can call centralized workflows from both `MokoStandards` and `.github-private` - -2. **repo_health.yml** - - Reason: Repository-specific health checks and metrics - - Keep local with option to extend from `MokoStandards` base - -3. **version_branch.yml** - - Reason: Project-specific versioning strategy - - Keep local - -## Scripts Categorization - -### Scripts to Centralize - -#### To `MokoStandards` (Public) - -1. **scripts/lib/extension_utils.py** ✓ - - Shared across all extension projects - - Platform detection logic (Joomla/Dolibarr) - - Target: `MokoStandards/scripts/shared/extension_utils.py` - -2. **scripts/lib/common.py** ✓ - - Universal utility functions - - No project-specific or sensitive logic - - Target: `MokoStandards/scripts/shared/common.py` - -3. **scripts/release/detect_platform.py** ✓ - - Platform detection helper - - Publicly useful for other projects - - Target: `MokoStandards/scripts/shared/detect_platform.py` - -#### To `.github-private` (Private) - -1. **scripts/release/deployment/** ✓ - - Deployment scripts with credential handling - - Target: `.github-private/scripts/deployment/` - -2. **scripts/release/publish.py** (if sensitive) - - Release publishing with proprietary logic - - Target: `.github-private/scripts/release/publish.py` - -### Scripts to Keep Local - -1. **scripts/lib/joomla_manifest.py** - - Joomla-specific, but project may have customizations - - Evaluate based on actual usage - -2. **scripts/validate/** (most) - - Project-specific validation rules - - Keep local unless truly generic - -3. **scripts/release/package_extension.py** - - Uses shared libraries but has project-specific logic - - Keep local, depend on centralized libs - -## Benefits After Migration - -### For Development Team -- ✅ Simplified workflow files in main repository -- ✅ Easier to understand and maintain -- ✅ Consistent CI/CD across all projects -- ✅ Faster updates (update once, applies everywhere) - -### For Security -- ✅ Sensitive credentials isolated in private repository -- ✅ Controlled access to deployment logic -- ✅ Audit trail for CI/CD changes - -### For Organization -- ✅ Centralized CI/CD governance -- ✅ Standardized processes across projects -- ✅ Reduced duplication -- ✅ Easier onboarding for new projects - -## Testing Plan - -### Pre-Migration Testing -1. ✅ Document all current workflows and their triggers -2. ✅ Identify all secrets and variables used -3. ✅ Create inventory of external dependencies - -### During Migration -1. Create .github-private repository in test organization first -2. Convert one workflow at a time -3. Test with feature branches -4. Validate all trigger conditions work -5. Verify secret inheritance - -### Post-Migration Validation -1. Run full CI/CD pipeline on test branch -2. Verify all workflows execute correctly -3. Check deployment to staging -4. Monitor for any broken integrations -5. Update runbooks and documentation - -## Rollback Plan - -If issues arise during migration: - -1. **Immediate Rollback**: Revert caller workflow to inline implementation -2. **Keep Both**: Maintain both local and centralized workflows temporarily -3. **Gradual Migration**: Move workflows one at a time with validation periods - -## Timeline - -- **Week 1**: Create .github-private repository, set up structure -- **Week 2**: Convert and test php_quality.yml -- **Week 3**: Convert and test release_pipeline.yml and deploy_staging.yml -- **Week 4**: Convert remaining workflows, finalize documentation -- **Week 5**: Complete migration, monitor, and optimize - -## Action Items - -### Immediate (This PR) -- [x] Create migration plan document -- [ ] Review and approve migration strategy -- [ ] Identify team members responsible for migration - -### Next Steps -- [ ] Create .github-private repository -- [ ] Set up repository structure -- [ ] Configure secrets and variables at organization level -- [ ] Begin workflow conversion (starting with php_quality.yml) -- [ ] Test reusable workflow pattern -- [ ] Document lessons learned - -## Technical Architecture - -### Communication Flow - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Main Repository (.github/workflows/) │ -│ ┌───────────────────────────────────────────────────────┐ │ -│ │ Caller Workflow (php_quality.yml) │ │ -│ │ - Defines triggers (push, PR, etc.) │ │ -│ │ - Sets permissions │ │ -│ │ - Passes inputs and secrets │ │ -│ └───────────────────┬───────────────────────────────────┘ │ -│ │ uses: org/.github-private/...@main │ -└──────────────────────┼──────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────┐ -│ .github-private Repository (.github/workflows/) │ -│ ┌───────────────────────────────────────────────────────┐ │ -│ │ Reusable Workflow (reusable-php-quality.yml) │ │ -│ │ - workflow_call trigger │ │ -│ │ - Receives inputs from caller │ │ -│ │ - Inherits secrets from organization │ │ -│ │ - Executes CI/CD logic │ │ -│ │ - Returns job outputs │ │ -│ └───────────────────┬───────────────────────────────────┘ │ -│ │ │ -│ ┌───────────────────▼───────────────────────────────────┐ │ -│ │ Shared Scripts (scripts/shared/) │ │ -│ │ - extension_utils.py │ │ -│ │ - deployment utilities │ │ -│ │ - validation helpers │ │ -│ └───────────────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────────────┘ -``` - -### Secret and Variable Inheritance Model - -``` -Organization Level (Settings > Secrets and Variables) -├── Secrets -│ ├── FTP_SERVER (inherited by all repos) -│ ├── FTP_USER (inherited by all repos) -│ ├── FTP_KEY (inherited by all repos) -│ ├── FTP_PASSWORD (inherited by all repos) -│ ├── FTP_PATH (inherited by all repos) -│ └── API_TOKENS (inherited by all repos) -│ -├── Variables -│ ├── DEPLOY_DRY_RUN: false (can be overridden) -│ ├── FTP_PROTOCOL: sftp (can be overridden) -│ ├── FTP_PORT: 22 (can be overridden) -│ └── PHP_VERSIONS: ["8.0","8.1","8.2","8.3"] -│ -└── Repository Level (Override if needed) - ├── moko-cassiopeia - │ └── Variables - │ └── DEPLOY_DRY_RUN: true (override for this repo) - │ - └── other-project - └── Variables - └── FTP_PATH_SUFFIX: /custom (repo-specific) -``` - -### Workflow Version Pinning Strategy - -#### Option 1: Track Main Branch (Automatic Updates) -```yaml -uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@main -``` -**Pros**: Always get latest features and fixes -**Cons**: Breaking changes may affect workflows -**Use Case**: Development branches, staging deployments - -#### Option 2: Pin to Semantic Version (Stable) -```yaml -uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 -# or -uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1.2 -``` -**Pros**: Stable, predictable behavior -**Cons**: Manual updates required -**Use Case**: Production deployments, critical workflows - -#### Option 3: Pin to Specific Commit SHA (Maximum Stability) -```yaml -uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@a1b2c3d -``` -**Pros**: Immutable, guaranteed consistency -**Cons**: No automatic updates, harder to maintain -**Use Case**: Compliance requirements, audit trails - -### Detailed Workflow Conversion Examples - -#### Before: Inline Workflow (Current State) - -**`.github/workflows/php_quality.yml` (93 lines)** -```yaml -name: PHP Code Quality - -on: - pull_request: - branches: [ main, dev/*, rc/* ] - push: - branches: [ main, dev/*, rc/* ] - -permissions: - contents: read - -jobs: - php-compatibility-check: - name: PHP Compatibility Check - runs-on: ubuntu-latest - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: '8.1' - extensions: mbstring, xml, curl, zip - - - name: Install PHP_CodeSniffer and PHPCompatibility - run: | - composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies - phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility - - - name: Check PHP 8.0+ Compatibility - run: phpcs --standard=PHPCompatibility --runtime-set testVersion 8.0- src/ - - phpcs: - name: PHP_CodeSniffer - runs-on: ubuntu-latest - strategy: - matrix: - php-version: ['8.0', '8.1', '8.2', '8.3'] - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP ${{ matrix.php-version }} - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: mbstring, xml, curl, zip - - - name: Install PHP_CodeSniffer - run: composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - - - name: Run PHP_CodeSniffer - run: phpcs --standard=phpcs.xml src/ - - # ... additional jobs -``` - -#### After: Caller Workflow (Target State) - -**`.github/workflows/php_quality.yml` (15 lines - 84% reduction)** -```yaml -name: PHP Code Quality - -on: - pull_request: - branches: [ main, dev/*, rc/* ] - push: - branches: [ main, dev/*, rc/* ] - -permissions: - contents: read - -jobs: - quality-checks: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - php-extensions: 'mbstring, xml, curl, zip' - source-directory: 'src' - phpcs-standard: 'phpcs.xml' - enable-phpcompat: true - enable-phpstan: true - phpstan-level: 'max' - secrets: inherit -``` - -**Benefits**: -- 84% reduction in code -- Centralized maintenance -- Consistent across all repositories -- Easy to add new checks (update once in .github-private) -- Version control with semantic versioning - -#### Reusable Workflow (in .github-private) - -**`.github-private/.github/workflows/reusable-php-quality.yml`** -```yaml -name: Reusable PHP Quality Checks - -on: - workflow_call: - inputs: - php-versions: - description: 'JSON array of PHP versions to test against' - required: false - type: string - default: '["8.0", "8.1", "8.2", "8.3"]' - php-extensions: - description: 'Comma-separated list of PHP extensions' - required: false - type: string - default: 'mbstring, xml, curl, zip' - source-directory: - description: 'Source code directory to analyze' - required: false - type: string - default: 'src' - phpcs-standard: - description: 'PHPCS standard configuration file' - required: false - type: string - default: 'phpcs.xml' - enable-phpcompat: - description: 'Enable PHP Compatibility checks' - required: false - type: boolean - default: true - enable-phpstan: - description: 'Enable PHPStan static analysis' - required: false - type: boolean - default: true - phpstan-level: - description: 'PHPStan analysis level' - required: false - type: string - default: 'max' - phpstan-config: - description: 'PHPStan configuration file' - required: false - type: string - default: 'phpstan.neon' - outputs: - phpcs-passed: - description: 'Whether PHPCS checks passed' - value: ${{ jobs.phpcs.outputs.passed }} - phpstan-passed: - description: 'Whether PHPStan checks passed' - value: ${{ jobs.phpstan.outputs.passed }} - -permissions: - contents: read - -jobs: - php-compatibility-check: - name: PHP Compatibility Check - runs-on: ubuntu-latest - if: ${{ inputs.enable-phpcompat }} - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: '8.1' - extensions: ${{ inputs.php-extensions }} - coverage: none - - - name: Cache Composer dependencies - uses: actions/cache@v4 - with: - path: | - ~/.composer/cache - ~/.composer/vendor - key: ${{ runner.os }}-composer-phpcompat-${{ hashFiles('**/composer.lock') }} - restore-keys: | - ${{ runner.os }}-composer-phpcompat- - - - name: Install PHP_CodeSniffer and PHPCompatibility - run: | - composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies - phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility - - - name: Check PHP 8.0+ Compatibility - run: | - phpcs --standard=PHPCompatibility \ - --runtime-set testVersion 8.0- \ - --report=full \ - --report-file=phpcompat-report.txt \ - ${{ inputs.source-directory }}/ - - - name: Upload PHPCompatibility Report - if: always() - uses: actions/upload-artifact@v4 - with: - name: phpcompat-report - path: phpcompat-report.txt - retention-days: 30 - - phpcs: - name: PHP_CodeSniffer (PHP ${{ matrix.php-version }}) - runs-on: ubuntu-latest - outputs: - passed: ${{ steps.check.outputs.passed }} - strategy: - fail-fast: false - matrix: - php-version: ${{ fromJson(inputs.php-versions) }} - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP ${{ matrix.php-version }} - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: ${{ inputs.php-extensions }} - coverage: none - - - name: Cache Composer dependencies - uses: actions/cache@v4 - with: - path: | - ~/.composer/cache - ~/.composer/vendor - key: ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpcs-${{ hashFiles('**/composer.lock') }} - restore-keys: | - ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpcs- - - - name: Install PHP_CodeSniffer - run: composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - - - name: Run PHP_CodeSniffer - id: check - run: | - phpcs --standard=${{ inputs.phpcs-standard }} \ - --report=full \ - --report-file=phpcs-report-${{ matrix.php-version }}.txt \ - ${{ inputs.source-directory }}/ - echo "passed=true" >> $GITHUB_OUTPUT - - - name: Upload PHPCS Report - if: always() - uses: actions/upload-artifact@v4 - with: - name: phpcs-report-php${{ matrix.php-version }} - path: phpcs-report-${{ matrix.php-version }}.txt - retention-days: 30 - - phpstan: - name: PHPStan (PHP ${{ matrix.php-version }}) - runs-on: ubuntu-latest - if: ${{ inputs.enable-phpstan }} - outputs: - passed: ${{ steps.check.outputs.passed }} - strategy: - fail-fast: false - matrix: - php-version: ${{ fromJson(inputs.php-versions) }} - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP ${{ matrix.php-version }} - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: ${{ inputs.php-extensions }} - coverage: none - - - name: Cache Composer dependencies - uses: actions/cache@v4 - with: - path: | - ~/.composer/cache - ~/.composer/vendor - key: ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpstan-${{ hashFiles('**/composer.lock') }} - restore-keys: | - ${{ runner.os }}-php${{ matrix.php-version }}-composer-phpstan- - - - name: Install PHPStan - run: composer global require "phpstan/phpstan:^1.0" --with-all-dependencies - - - name: Run PHPStan - id: check - run: | - phpstan analyse \ - --configuration=${{ inputs.phpstan-config }} \ - --level=${{ inputs.phpstan-level }} \ - --error-format=table \ - --no-progress \ - --no-interaction \ - ${{ inputs.source-directory }}/ \ - > phpstan-report-${{ matrix.php-version }}.txt 2>&1 - echo "passed=true" >> $GITHUB_OUTPUT - - - name: Upload PHPStan Report - if: always() - uses: actions/upload-artifact@v4 - with: - name: phpstan-report-php${{ matrix.php-version }} - path: phpstan-report-${{ matrix.php-version }}.txt - retention-days: 30 -``` - -## Advanced Patterns and Best Practices - -### Pattern 1: Conditional Workflow Execution - -Allow repositories to enable/disable specific checks: - -```yaml -# Caller workflow -jobs: - quality: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 - with: - enable-phpcompat: ${{ github.event_name == 'pull_request' }} # Only on PRs - enable-phpstan: ${{ contains(github.event.head_commit.message, '[phpstan]') }} # Only if commit message contains [phpstan] - phpstan-level: ${{ github.ref == 'refs/heads/main' && 'max' || '6' }} # Stricter on main -``` - -### Pattern 2: Matrix Strategy Inheritance - -Pass complex matrix configurations: - -```yaml -# Caller workflow -jobs: - test: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-test.yml@v1 - with: - test-matrix: | - { - "php": ["8.0", "8.1", "8.2", "8.3"], - "joomla": ["4.4", "5.0"], - "database": ["mysql:8.0", "postgresql:14"] - } -``` - -### Pattern 3: Composite Actions for Reusability - -Break down workflows into composite actions for even more reusability: - -**`.github-private/.github/actions/setup-php-quality/action.yml`** -```yaml -name: 'Setup PHP Quality Tools' -description: 'Install PHP CodeSniffer, PHPCompatibility, and PHPStan' - -inputs: - php-version: - description: 'PHP version to setup' - required: true - enable-phpstan: - description: 'Install PHPStan' - required: false - default: 'true' - -runs: - using: 'composite' - steps: - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ inputs.php-version }} - extensions: mbstring, xml, curl, zip - coverage: none - - - name: Install PHPCS and PHPCompatibility - shell: bash - run: | - composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies - phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility - - - name: Install PHPStan - if: inputs.enable-phpstan == 'true' - shell: bash - run: composer global require "phpstan/phpstan:^1.0" --with-all-dependencies -``` - -**Usage in reusable workflow:** -```yaml -- name: Setup PHP Quality Tools - uses: mokoconsulting-tech/.github-private/.github/actions/setup-php-quality@v1 - with: - php-version: ${{ matrix.php-version }} - enable-phpstan: true -``` - -### Pattern 4: Workflow Outputs and Chaining - -Use outputs to chain workflows: - -```yaml -# Caller workflow -jobs: - quality: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - - deploy: - needs: quality - if: ${{ needs.quality.outputs.phpcs-passed == 'true' && needs.quality.outputs.phpstan-passed == 'true' }} - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 - with: - environment: staging -``` - -## Security Considerations - -### Principle of Least Privilege - -**Organization Secrets Access**: -- Only grant `.github-private` repository access to necessary secrets -- Use environment-specific secrets (staging, production) -- Rotate secrets regularly - -**Repository Permissions**: -```yaml -# .github-private repository settings -Permissions: - - Read: All organization members (for viewing workflows) - - Write: DevOps team only - - Admin: Organization owners only - -Branch Protection (main): - - Require pull request reviews (2 approvals) - - Require status checks to pass - - Require branches to be up to date - - No force pushes - - No deletions -``` - -### Secret Masking - -Ensure secrets are never exposed in logs: - -```yaml -# BAD - Exposes secret in logs -- name: Deploy - run: echo "Deploying with password: ${{ secrets.FTP_PASSWORD }}" - -# GOOD - Secret is masked -- name: Deploy - run: | - echo "::add-mask::${{ secrets.FTP_PASSWORD }}" - ./deploy.sh --password "${{ secrets.FTP_PASSWORD }}" -``` - -### Audit Trail - -Track all workflow executions: - -```yaml -# Add to all reusable workflows -- name: Audit Log - if: always() - run: | - echo "Workflow executed by: ${{ github.actor }}" - echo "Repository: ${{ github.repository }}" - echo "Branch: ${{ github.ref }}" - echo "Commit: ${{ github.sha }}" - echo "Workflow: ${{ github.workflow }}" - echo "Run ID: ${{ github.run_id }}" - echo "Timestamp: $(date -u +%Y-%m-%dT%H:%M:%SZ)" -``` - -## Performance Optimization - -### Caching Strategy - -**Composer Dependencies**: -```yaml -- name: Cache Composer - uses: actions/cache@v4 - with: - path: | - ~/.composer/cache - ~/.composer/vendor - key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} - restore-keys: | - ${{ runner.os }}-composer- -``` - -**Tool Installations**: -```yaml -- name: Cache Quality Tools - uses: actions/cache@v4 - with: - path: | - ~/.composer/vendor/squizlabs/php_codesniffer - ~/.composer/vendor/phpstan/phpstan - key: ${{ runner.os }}-php-tools-v1 -``` - -### Parallel Execution - -Maximize parallelism: - -```yaml -strategy: - fail-fast: false # Don't stop other jobs if one fails - max-parallel: 10 # Run up to 10 jobs simultaneously - matrix: - php-version: ['8.0', '8.1', '8.2', '8.3'] - joomla-version: ['4.4', '5.0'] -``` - -### Job Concurrency Control - -Prevent wasted resources: - -```yaml -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} # Cancel old runs except on main -``` - -## Monitoring and Observability - -### Workflow Status Notifications - -**Slack Integration**: -```yaml -- name: Notify Slack on Failure - if: failure() - uses: slackapi/slack-github-action@v1 - with: - payload: | - { - "text": "Workflow failed: ${{ github.workflow }}", - "blocks": [ - { - "type": "section", - "text": { - "type": "mrkdwn", - "text": "*Workflow*: ${{ github.workflow }}\n*Repository*: ${{ github.repository }}\n*Branch*: ${{ github.ref }}\n*Actor*: ${{ github.actor }}\n*Run*: <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run>" - } - } - ] - } - env: - SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} -``` - -### Metrics Collection - -Track workflow execution metrics: - -```yaml -- name: Record Metrics - if: always() - run: | - curl -X POST "${{ secrets.METRICS_ENDPOINT }}" \ - -H "Content-Type: application/json" \ - -d '{ - "workflow": "${{ github.workflow }}", - "repository": "${{ github.repository }}", - "status": "${{ job.status }}", - "duration": "${{ steps.start-time.outputs.elapsed }}", - "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)" - }' -``` - -## Troubleshooting Guide - -### Common Issues and Solutions - -#### Issue 1: "Workflow not found" error - -**Symptom**: -``` -Error: Unable to resolve action `mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@main`, -unable to find version `main` -``` - -**Solutions**: -1. Verify `.github-private` repository exists and is accessible -2. Check repository permissions (must have at least read access) -3. Verify branch name (main vs master) -4. Ensure workflow file exists at specified path - -**Verification Commands**: -```bash -# Check repository access -gh api repos/mokoconsulting-tech/.github-private - -# List workflow files -gh api repos/mokoconsulting-tech/.github-private/contents/.github/workflows - -# Check branch exists -gh api repos/mokoconsulting-tech/.github-private/branches/main -``` - -#### Issue 2: Secrets not inherited - -**Symptom**: -``` -Error: Secret FTP_PASSWORD is not set -``` - -**Solutions**: -1. Ensure `secrets: inherit` is set in caller workflow -2. Verify secret exists at organization level -3. Check `.github-private` repository has access to organization secrets -4. Verify secret names match exactly (case-sensitive) - -**Verification**: -```bash -# List organization secrets -gh api orgs/mokoconsulting-tech/actions/secrets - -# Check repository secret access -gh api repos/mokoconsulting-tech/.github-private/actions/secrets -``` - -#### Issue 3: Workflow runs on wrong trigger - -**Symptom**: -Workflow runs when it shouldn't, or doesn't run when expected - -**Solutions**: -1. Review `on:` triggers in caller workflow -2. Check branch protection rules -3. Verify path filters if using `paths:` or `paths-ignore:` -4. Test with different trigger events - -**Example Fix**: -```yaml -# Before (runs on all pushes) -on: - push: - -# After (runs only on main and feature branches) -on: - push: - branches: - - main - - 'feature/**' -``` - -#### Issue 4: Matrix strategy not working - -**Symptom**: -Only one job runs instead of multiple matrix jobs - -**Solutions**: -1. Verify JSON syntax in matrix definition -2. Use `fromJson()` for string inputs -3. Check for empty arrays -4. Validate matrix variable references - -**Example**: -```yaml -# Caller workflow -with: - php-versions: '["8.0", "8.1", "8.2"]' # Must be JSON string - -# Reusable workflow -matrix: - php-version: ${{ fromJson(inputs.php-versions) }} # Convert to array -``` - -## References - -- [GitHub Reusable Workflows Documentation](https://docs.github.com/en/actions/using-workflows/reusing-workflows) -- [GitHub Security Best Practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) -- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) -- [workflow_call Event](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_call) -- [GitHub Actions Caching](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows) -- [Composite Actions](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action) - -## Support - -For questions or issues during migration: -- Review this document -- Check GitHub Actions documentation -- Contact: DevOps team -- Slack: #devops-support - ---- - -**Status**: Ready for Implementation -**Author**: GitHub Copilot -**Date**: 2026-01-05 -**Version**: 2.0 -**Last Updated**: 2026-01-05 diff --git a/docs/MIGRATION_CHECKLIST.md b/docs/MIGRATION_CHECKLIST.md deleted file mode 100644 index 50c9fdf..0000000 --- a/docs/MIGRATION_CHECKLIST.md +++ /dev/null @@ -1,1413 +0,0 @@ -# Centralized CI/CD Migration Checklist - -This checklist guides the migration of CI/CD workflows from individual repositories to centralized repositories using a dual-repository architecture. - -## Architecture Overview - -**Two Centralized Repositories:** -1. **`MokoStandards`** (Public) - Community workflows, quality checks, testing -2. **`.github-private`** (Private) - Sensitive workflows, deployments, proprietary logic - -## Phase 1: Planning and Preparation - -### Documentation -- [x] Create CI_MIGRATION_PLAN.md -- [x] Create REUSABLE_WORKFLOWS.md -- [x] Create migration checklist -- [x] Define dual-repository architecture (MokoStandards + .github-private) -- [ ] Review and approve migration plan with team -- [ ] Identify workflow owners and stakeholders -- [ ] Schedule migration windows - -### Repository Inventory -- [x] List all workflows in current repository -- [x] Identify workflows to centralize (categorized by sensitivity) -- [x] Identify public workflows for MokoStandards -- [x] Identify sensitive workflows for .github-private -- [x] Identify workflows to keep local -- [x] Document workflow dependencies -- [x] List all secrets used by workflows -- [x] List all variables used by workflows - -### Risk Assessment -- [ ] Identify critical workflows that cannot have downtime -- [ ] Create rollback procedures for both repositories -- [ ] Set up monitoring for workflow failures -- [ ] Communicate migration plan to team -- [ ] Plan staged rollout strategy - -## Phase 2: Centralized Repository Setup - -### MokoStandards Repository Creation (Public) -- [ ] Create `MokoStandards` repository in organization -- [ ] Set repository to Public -- [ ] Initialize with README explaining public standards -- [ ] Add LICENSE file (appropriate for public repository) -- [ ] Create initial branch structure (main, develop) -- [ ] Enable GitHub Pages for documentation -- [ ] Set up community contribution guidelines - -### .github-private Repository Creation (Private) -- [ ] Create `.github-private` repository in organization -- [ ] Set repository to Private -- [ ] Initialize with README explaining private workflows -- [ ] Add LICENSE file -- [ ] Create initial branch structure (main, develop) - -### Repository Configuration -- [ ] Configure branch protection rules for main -- [ ] Set up team access and permissions -- [ ] Enable GitHub Actions for repository -- [ ] Configure repository settings (issues, wiki, etc.) - -### Directory Structure -- [ ] Create `.github/workflows/` directory -- [ ] Create `scripts/` directory for shared scripts -- [ ] Create `docs/` directory for documentation -- [ ] Create `templates/` directory for workflow templates - -### Documentation -- [ ] Add README.md explaining repository purpose -- [ ] Add USAGE.md with workflow calling examples -- [ ] Add CONTRIBUTING.md for workflow maintenance -- [ ] Document secret and variable requirements - -## Phase 3: Secrets and Variables Setup - -### Organization-Level Secrets -- [ ] Migrate FTP_SERVER to organization secrets -- [ ] Migrate FTP_USER to organization secrets -- [ ] Migrate FTP_KEY to organization secrets (if used) -- [ ] Migrate FTP_PASSWORD to organization secrets (if used) -- [ ] Migrate FTP_PATH to organization secrets -- [ ] Review and migrate other deployment credentials - -### Organization-Level Variables -- [ ] Create DEPLOY_DRY_RUN variable -- [ ] Create FTP_PATH_SUFFIX variable -- [ ] Create FTP_PROTOCOL variable (default: sftp) -- [ ] Create FTP_PORT variable (default: 22) -- [ ] Document all organization variables - -### Access Configuration -- [ ] Grant .github-private repository access to organization secrets -- [ ] Configure repository-level secret overrides if needed -- [ ] Test secret accessibility from workflows - -## Phase 4: Workflow Migration (Priority Order) - -### Workflow 1: php_quality.yml (Low Risk) - -#### Pre-Migration Checklist -- [ ] Create reusable-php-quality.yml in .github-private -- [ ] Convert workflow to use workflow_call trigger -- [ ] Add input parameters: - - [ ] php-versions (JSON array) - - [ ] php-extensions (string) - - [ ] working-directory (string) - - [ ] enable-phpcompat (boolean) - - [ ] enable-phpstan (boolean) - -#### Detailed Migration Steps - -**Step 1: Create Reusable Workflow** -```bash -# In .github-private repository -cd .github-private -git checkout -b feature/add-php-quality-workflow - -# Create workflow file -mkdir -p .github/workflows -cat > .github/workflows/reusable-php-quality.yml << 'EOF' -name: Reusable PHP Quality Workflow - -on: - workflow_call: - inputs: - php-versions: - required: false - type: string - default: '["8.0", "8.1", "8.2", "8.3"]' - # ... other inputs - outputs: - all-passed: - value: ${{ jobs.summary.outputs.all-passed }} - -jobs: - phpcs: - # ... job definition - - phpstan: - # ... job definition - - summary: - # ... summary job -EOF - -# Commit and push -git add .github/workflows/reusable-php-quality.yml -git commit -m "feat: add reusable PHP quality workflow" -git push origin feature/add-php-quality-workflow - -# Create pull request -gh pr create --title "Add reusable PHP quality workflow" \ - --body "Initial implementation of reusable PHP quality checks" -``` - -**Step 2: Test Reusable Workflow** -```bash -# Create test repository or use existing -cd test-repository -git checkout -b test/reusable-workflow - -# Create test caller workflow -cat > .github/workflows/test-php-quality.yml << 'EOF' -name: Test PHP Quality (Reusable) - -on: - push: - branches: [test/**] - -jobs: - quality: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@feature/add-php-quality-workflow - with: - php-versions: '["8.1"]' - enable-phpcompat: true - enable-phpstan: true - secrets: inherit -EOF - -git add .github/workflows/test-php-quality.yml -git commit -m "test: add test caller for reusable workflow" -git push origin test/reusable-workflow - -# Monitor test run -gh run watch -``` - -**Step 3: Validation Checks** -- [ ] Verify all jobs execute successfully -- [ ] Check that reports are generated correctly -- [ ] Verify secrets are accessible -- [ ] Test with different input combinations -- [ ] Validate error handling (introduce intentional errors) - -**Validation Script**: -```bash -#!/bin/bash -# validate_php_quality_workflow.sh - -set -euo pipefail - -echo "=== Validating PHP Quality Workflow ===" - -# Test 1: Basic execution -echo "Test 1: Basic execution with default parameters" -gh workflow run test-php-quality.yml -WORKFLOW_ID=$(gh run list --workflow=test-php-quality.yml --limit 1 --json databaseId --jq '.[0].databaseId') -gh run watch $WORKFLOW_ID - -# Test 2: Custom PHP versions -echo "Test 2: Custom PHP versions" -gh workflow run test-php-quality.yml \ - --field php-versions='["8.2","8.3"]' -gh run watch - -# Test 3: Disable PHPStan -echo "Test 3: With PHPStan disabled" -gh workflow run test-php-quality.yml \ - --field enable-phpstan=false -gh run watch - -# Test 4: Check outputs -echo "Test 4: Verify workflow outputs" -gh run view $WORKFLOW_ID --json jobs --jq '.jobs[].conclusion' | grep -q success - -echo "✅ All validation tests passed" -``` - -**Step 4: Update Main Repository** -```bash -# In main repository (moko-cassiopeia) -cd moko-cassiopeia -git checkout -b migrate/php-quality-workflow - -# Backup existing workflow -mkdir -p .backup/workflows -cp .github/workflows/php_quality.yml .backup/workflows/php_quality.yml.backup -git add .backup/workflows/ - -# Replace with caller workflow -cat > .github/workflows/php_quality.yml << 'EOF' -name: PHP Code Quality - -on: - pull_request: - branches: [ main, dev/*, rc/* ] - push: - branches: [ main, dev/*, rc/* ] - -permissions: - contents: read - -jobs: - quality-checks: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - php-extensions: 'mbstring, xml, curl, zip' - source-directory: 'src' - enable-phpcompat: true - enable-phpstan: true - secrets: inherit -EOF - -git add .github/workflows/php_quality.yml -git commit -m "migrate: convert php_quality.yml to use centralized reusable workflow" -git push origin migrate/php-quality-workflow - -# Create pull request -gh pr create --title "Migrate PHP quality workflow to centralized version" \ - --body "Migrates php_quality.yml to call reusable workflow from .github-private" -``` - -**Step 5: Integration Testing** -- [ ] Create test PR to trigger workflow -- [ ] Verify workflow executes correctly -- [ ] Check that all jobs complete successfully -- [ ] Verify artifacts are uploaded correctly -- [ ] Compare execution time with old workflow -- [ ] Validate error messages are clear - -**Integration Test Script**: -```bash -#!/bin/bash -# test_integration.sh - -set -euo pipefail - -echo "=== Integration Testing ===" - -# Create test branch -git checkout -b test/integration-$(date +%s) -echo "// test change" >> src/test.php -git add src/test.php -git commit -m "test: trigger workflow" -git push origin HEAD - -# Create PR -PR_URL=$(gh pr create --title "Test: PHP Quality Integration" \ - --body "Testing integrated PHP quality workflow" \ - --head $(git branch --show-current)) - -echo "PR created: $PR_URL" -echo "Waiting for checks..." - -# Wait for checks to complete -gh pr checks $PR_URL --watch - -# Verify checks passed -CHECK_STATUS=$(gh pr checks $PR_URL --json state --jq '.[0].state') -if [ "$CHECK_STATUS" == "SUCCESS" ]; then - echo "✅ Integration test passed" - gh pr close $PR_URL --delete-branch -else - echo "❌ Integration test failed" - exit 1 -fi -``` - -**Step 6: Monitor for Issues** -- [ ] Monitor for 1 week (7 days) -- [ ] Track workflow execution times -- [ ] Collect developer feedback -- [ ] Document any issues encountered -- [ ] Fix issues promptly - -**Monitoring Dashboard**: -```bash -#!/bin/bash -# monitor_workflow.sh - -set -euo pipefail - -echo "=== Workflow Monitoring Dashboard ===" - -WORKFLOW="php_quality.yml" -START_DATE=$(date -d '7 days ago' +%Y-%m-%d) - -# Execution count -EXEC_COUNT=$(gh run list --workflow=$WORKFLOW --created=">$START_DATE" --json databaseId --jq 'length') -echo "Executions (last 7 days): $EXEC_COUNT" - -# Success rate -SUCCESS_COUNT=$(gh run list --workflow=$WORKFLOW --created=">$START_DATE" --status=success --json databaseId --jq 'length') -SUCCESS_RATE=$(awk "BEGIN {printf \"%.1f\", ($SUCCESS_COUNT/$EXEC_COUNT)*100}") -echo "Success rate: $SUCCESS_RATE%" - -# Average duration -AVG_DURATION=$(gh run list --workflow=$WORKFLOW --created=">$START_DATE" --limit 20 --json conclusion,duration --jq '[.[] | select(.conclusion=="success") | .duration] | add/length') -echo "Average duration: ${AVG_DURATION}s" - -# Recent failures -echo -e "\nRecent failures:" -gh run list --workflow=$WORKFLOW --status=failure --limit 5 --json databaseId,createdAt,headBranch,conclusion - -# Alert if success rate < 90% -if (( $(echo "$SUCCESS_RATE < 90" | bc -l) )); then - echo "⚠️ WARNING: Success rate below 90%" - # Send alert - curl -X POST $SLACK_WEBHOOK_URL \ - -H 'Content-Type: application/json' \ - -d "{\"text\":\"PHP Quality Workflow success rate is ${SUCCESS_RATE}%\"}" -fi -``` - -**Step 7: Document Lessons Learned** -- [ ] Document any issues encountered -- [ ] Note what went well -- [ ] Identify improvements for next workflow -- [ ] Update migration documentation - -### Workflow 2: joomla_testing.yml (Low Risk) - -#### Pre-Migration Checklist -- [ ] Create reusable-joomla-testing.yml in .github-private -- [ ] Convert workflow to use workflow_call trigger -- [ ] Add input parameters as needed -- [ ] Test reusable workflow independently - -#### Detailed Migration Steps - -**(Similar structure to Workflow 1, with Joomla-specific considerations)** - -**Step 1-7**: Follow same pattern as php_quality.yml migration - -**Additional Considerations**: -- [ ] Test with different Joomla versions (4.4, 5.0) -- [ ] Verify database compatibility testing -- [ ] Check Joomla-specific tooling integration -- [ ] Validate Joomla Update Server compatibility - -### Workflow 3: deploy_staging.yml (High Risk) - -#### Pre-Migration Checklist -- [ ] Create reusable-deploy-staging.yml in .github-private -- [ ] Convert workflow to use workflow_call trigger -- [ ] Add input parameters for deployment configuration -- [ ] Configure secret requirements -- [ ] Create detailed rollback plan - -#### Risk Mitigation Strategies - -**Pre-Deployment Checks**: -```yaml -- name: Pre-Deployment Validation - run: | - # Verify deployment prerequisites - if [ -z "${{ secrets.FTP_SERVER }}" ]; then - echo "❌ FTP_SERVER not configured" - exit 1 - fi - - # Test connectivity - nc -zv ${{ secrets.FTP_SERVER }} 22 || exit 1 - - # Verify artifact exists - if [ ! -f deployment.zip ]; then - echo "❌ Deployment artifact not found" - exit 1 - fi - - echo "✅ Pre-deployment checks passed" -``` - -**Deployment with Backup**: -```yaml -- name: Backup Current Deployment - run: | - # Create backup of current deployment - ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_SERVER }} \ - "cd ${{ secrets.FTP_PATH }} && tar -czf backup-$(date +%Y%m%d-%H%M%S).tar.gz ." - - echo "✅ Backup created" - -- name: Deploy New Version - id: deploy - run: | - # Deploy new version - scp deployment.zip ${{ secrets.FTP_USER }}@${{ secrets.FTP_SERVER }}:${{ secrets.FTP_PATH }}/ - - ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_SERVER }} \ - "cd ${{ secrets.FTP_PATH }} && unzip -o deployment.zip" - - echo "✅ Deployment successful" - -- name: Health Check - run: | - # Verify deployment - for i in {1..30}; do - if curl -f -s "${{ inputs.deploy-url }}/health" > /dev/null; then - echo "✅ Health check passed" - exit 0 - fi - echo "Attempt $i/30 failed, retrying..." - sleep 10 - done - - echo "❌ Health check failed" - exit 1 - -- name: Rollback on Failure - if: failure() - run: | - echo "⚠️ Deployment failed, rolling back..." - - # Restore from backup - BACKUP=$(ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_SERVER }} \ - "cd ${{ secrets.FTP_PATH }} && ls -t backup-*.tar.gz | head -1") - - ssh ${{ secrets.FTP_USER }}@${{ secrets.FTP_SERVER }} \ - "cd ${{ secrets.FTP_PATH }} && tar -xzf $BACKUP" - - echo "✅ Rollback completed" -``` - -#### Detailed Migration Steps - -**Step 1: Create Canary Deployment** -```bash -# Test deployment on canary environment first -gh workflow run deploy-staging.yml \ - --field environment=canary \ - --field deploy-url=https://canary.staging.example.com -``` - -**Step 2: Gradual Rollout** -- [ ] Week 1: Deploy to canary environment -- [ ] Week 2: Deploy to 25% of staging instances -- [ ] Week 3: Deploy to 50% of staging instances -- [ ] Week 4: Deploy to 100% of staging instances - -**Step 3: Full Migration** -- [ ] Convert to reusable workflow -- [ ] Update all deployment triggers -- [ ] Monitor closely for first 2 weeks - -**Emergency Rollback Procedure**: -```bash -#!/bin/bash -# emergency_rollback.sh - -set -euo pipefail - -echo "=== EMERGENCY ROLLBACK ===" -echo "This will revert deployment workflow to local version" -read -p "Continue? (yes/no): " CONFIRM - -if [ "$CONFIRM" != "yes" ]; then - echo "Rollback cancelled" - exit 0 -fi - -# Revert workflow -git checkout HEAD~1 -- .github/workflows/deploy_staging.yml -git commit -m "emergency: rollback deploy_staging workflow" -git push - -# Trigger immediate deployment with rolled-back workflow -gh workflow run deploy_staging.yml --field environment=staging - -echo "✅ Rollback initiated" -``` - -### Workflow 4: release_pipeline.yml (High Risk) - -#### Pre-Migration Checklist -- [ ] Create reusable-release-pipeline.yml in .github-private -- [ ] Convert workflow to use workflow_call trigger -- [ ] Add input parameters -- [ ] Configure all secret requirements -- [ ] Test with test release on feature branch - -#### Release Testing Strategy - -**Test Release Checklist**: -- [ ] Create test tag (v0.0.0-test) -- [ ] Trigger release workflow -- [ ] Verify package is built correctly -- [ ] Verify package is uploaded to correct location -- [ ] Verify GitHub release is created -- [ ] Verify release notes are correct -- [ ] Delete test release and tag - -**Test Release Script**: -```bash -#!/bin/bash -# test_release.sh - -set -euo pipefail - -echo "=== Test Release ===" - -# Create test tag -TEST_TAG="v0.0.0-test-$(date +%s)" -git tag -a $TEST_TAG -m "Test release" -git push origin $TEST_TAG - -# Trigger release workflow -gh workflow run release_pipeline.yml \ - --field release_classification=rc - -# Monitor release -gh run watch - -# Verify release created -gh release view $TEST_TAG - -# Download and verify artifact -gh release download $TEST_TAG -ls -lh *.zip - -# Cleanup -gh release delete $TEST_TAG --yes -git tag -d $TEST_TAG -git push origin :refs/tags/$TEST_TAG - -echo "✅ Test release completed successfully" -``` - -#### Migration Steps - -**Step 1-7**: Similar to previous workflows, with additional release-specific testing - -**Additional Validation**: -- [ ] Verify version detection works correctly -- [ ] Test RC (release candidate) releases -- [ ] Test stable releases -- [ ] Verify artifact signing (if enabled) -- [ ] Test rollback of failed releases - -## Phase 5: Script Migration - -### Shared Scripts to Migrate - -#### extension_utils.py - -**Migration Steps**: -```bash -# In .github-private repository -mkdir -p scripts/shared -cp path/to/extension_utils.py scripts/shared/ - -# Update imports in workflows -# From: -python3 scripts/lib/extension_utils.py - -# To: -python3 $GITHUB_WORKSPACE/.github-private-scripts/extension_utils.py -``` - -**Verification Script**: -```bash -#!/bin/bash -# verify_script_migration.sh - -set -euo pipefail - -echo "=== Verifying Script Migration ===" - -# Test extension_utils.py -python3 scripts/shared/extension_utils.py -echo "✅ extension_utils.py works correctly" - -# Test common.py -python3 scripts/shared/common.py -echo "✅ common.py works correctly" - -# Test all reusable workflows use correct paths -grep -r "scripts/lib" .github/workflows/ && { - echo "❌ Found old script paths" - exit 1 -} || echo "✅ No old script paths found" -``` - -### Script Dependency Management - -**Create requirements.txt** for shared scripts: -```txt -# .github-private/scripts/shared/requirements.txt -# Python dependencies for shared scripts -``` - -**Install Dependencies in Workflows**: -```yaml -- name: Setup Python Dependencies - run: | - pip install -r $GITHUB_WORKSPACE/.github-private-scripts/requirements.txt -``` - -## Phase 6: Testing and Validation - -### Comprehensive Test Suite - -**test_all_workflows.sh**: -```bash -#!/bin/bash -# test_all_workflows.sh - -set -euo pipefail - -echo "=== Comprehensive Workflow Testing ===" - -WORKFLOWS=( - "php_quality.yml" - "joomla_testing.yml" - "deploy_staging.yml" - "release_pipeline.yml" -) - -for workflow in "${WORKFLOWS[@]}"; do - echo "Testing $workflow..." - - # Trigger workflow - gh workflow run $workflow - - # Wait for completion - sleep 10 - - # Check result - LATEST_RUN=$(gh run list --workflow=$workflow --limit 1 --json databaseId,conclusion --jq '.[0]') - CONCLUSION=$(echo $LATEST_RUN | jq -r '.conclusion') - - if [ "$CONCLUSION" == "success" ]; then - echo "✅ $workflow passed" - else - echo "❌ $workflow failed" - exit 1 - fi -done - -echo "✅ All workflows passed" -``` - -### Performance Testing - -**Benchmark Script**: -```bash -#!/bin/bash -# benchmark_workflows.sh - -set -euo pipefail - -echo "=== Workflow Performance Benchmark ===" - -WORKFLOW="php_quality.yml" - -echo "Running 10 test executions..." -DURATIONS=() - -for i in {1..10}; do - # Trigger workflow - gh workflow run $workflow - sleep 5 - - # Get duration - DURATION=$(gh run list --workflow=$workflow --limit 1 --json duration --jq '.[0].duration') - DURATIONS+=($DURATION) - - echo "Run $i: ${DURATION}s" -done - -# Calculate average -AVG=$(printf '%s\n' "${DURATIONS[@]}" | awk '{sum+=$1} END {print sum/NR}') -echo "Average duration: ${AVG}s" - -# Calculate standard deviation -STDDEV=$(printf '%s\n' "${DURATIONS[@]}" | awk -v avg=$AVG '{sum+=($1-avg)^2} END {print sqrt(sum/NR)}') -echo "Standard deviation: ${STDDEV}s" -``` - -## Phase 7: Documentation Updates - -### Documentation Checklist - -- [ ] Update README.md with workflow links -- [ ] Update CONTRIBUTING.md with workflow information -- [ ] Create WORKFLOWS.md in .github-private -- [ ] Document all input parameters -- [ ] Document all secrets required -- [ ] Create troubleshooting guide -- [ ] Add workflow diagrams -- [ ] Document rollback procedures - -### Generate Workflow Documentation - -**Script to auto-generate documentation**: -```bash -#!/bin/bash -# generate_workflow_docs.sh - -set -euo pipefail - -echo "=== Generating Workflow Documentation ===" - -WORKFLOWS=(.github/workflows/reusable-*.yml) - -for workflow in "${WORKFLOWS[@]}"; do - NAME=$(basename $workflow .yml) - - echo "## $NAME" >> WORKFLOWS.md - echo "" >> WORKFLOWS.md - - # Extract inputs - echo "### Inputs" >> WORKFLOWS.md - yq eval '.on.workflow_call.inputs | to_entries | .[] | "- **" + .key + "**: " + .value.description' $workflow >> WORKFLOWS.md - echo "" >> WORKFLOWS.md - - # Extract secrets - echo "### Secrets" >> WORKFLOWS.md - yq eval '.on.workflow_call.secrets | to_entries | .[] | "- **" + .key + "**: " + (.value.required | if . then "Required" else "Optional" end)' $workflow >> WORKFLOWS.md - echo "" >> WORKFLOWS.md - - # Extract outputs - echo "### Outputs" >> WORKFLOWS.md - yq eval '.on.workflow_call.outputs | to_entries | .[] | "- **" + .key + "**: " + .value.description' $workflow >> WORKFLOWS.md - echo "" >> WORKFLOWS.md -done - -echo "✅ Documentation generated" -``` - -### Workflow 3: deploy_staging.yml (High Risk) -- [ ] Create reusable-deploy-staging.yml in .github-private -- [ ] Convert workflow to use workflow_call trigger -- [ ] Add input parameters for deployment configuration -- [ ] Configure secret requirements -- [ ] Test in non-production environment first -- [ ] Create detailed rollback plan -- [ ] Update main repository to call reusable workflow -- [ ] Perform controlled test deployment -- [ ] Monitor deployment logs closely -- [ ] Keep original workflow as backup for 2 weeks - -### Workflow 4: release_pipeline.yml (High Risk) -- [ ] Create reusable-release-pipeline.yml in .github-private -- [ ] Convert workflow to use workflow_call trigger -- [ ] Add input parameters: - - [ ] release_classification - - [ ] platform (joomla, dolibarr) -- [ ] Configure all secret requirements -- [ ] Test with test release on feature branch -- [ ] Update main repository to call reusable workflow -- [ ] Perform test release to RC channel -- [ ] Monitor release process -- [ ] Keep original workflow as backup for 2 weeks - -### Workflows to Keep Local -- [ ] Review ci.yml - Keep local or convert? -- [ ] Review repo_health.yml - Keep local -- [ ] Review version_branch.yml - Keep local -- [ ] Document decision rationale - -## Phase 5: Script Migration - -### Shared Scripts -- [ ] Copy scripts/lib/extension_utils.py to .github-private -- [ ] Copy scripts/lib/common.py to .github-private -- [ ] Update import paths in reusable workflows -- [ ] Test script functionality in new location -- [ ] Update documentation with new paths - -### Script Dependencies -- [ ] Document Python version requirements -- [ ] Document required pip packages -- [ ] Create requirements.txt if needed -- [ ] Test scripts in clean environment - -### Local Script Updates -- [ ] Update scripts/release/detect_platform.py to use centralized libs -- [ ] Update scripts/release/package_extension.py if needed -- [ ] Maintain backward compatibility where possible - -## Phase 6: Testing and Validation - -### Unit Testing -- [ ] Test each reusable workflow in isolation -- [ ] Verify all input parameters work correctly -- [ ] Verify secret inheritance works -- [ ] Test error handling and failure cases - -### Integration Testing -- [ ] Test full CI pipeline on feature branch -- [ ] Test PR workflows -- [ ] Test release workflow (dry-run) -- [ ] Test deployment workflow (to staging) -- [ ] Verify all notifications work - -### Performance Testing -- [ ] Compare workflow run times (before/after) -- [ ] Check for any performance regressions -- [ ] Optimize workflow caching if needed - -### Security Testing -- [ ] Verify secrets are not exposed in logs -- [ ] Test permission boundaries -- [ ] Review workflow security best practices -- [ ] Run security scan on workflow files - -## Phase 7: Documentation Updates - -### Main Repository Documentation -- [ ] Update README.md with workflow links -- [ ] Update CONTRIBUTING.md with workflow information -- [ ] Update docs/WORKFLOW_GUIDE.md -- [ ] Update docs/JOOMLA_DEVELOPMENT.md if needed -- [ ] Update docs/QUICK_START.md if needed - -### .github-private Documentation -- [ ] Complete README.md -- [ ] Complete USAGE.md with all workflows -- [ ] Add TROUBLESHOOTING.md -- [ ] Add workflow diagrams/flowcharts -- [ ] Document secret requirements per workflow - -### Team Communication -- [ ] Send announcement email about migration -- [ ] Schedule knowledge sharing session -- [ ] Create FAQ document -- [ ] Update team wiki/confluence - -## Phase 8: Monitoring and Optimization - -### Initial Monitoring (Week 1) -- [ ] Monitor all workflow runs daily -- [ ] Check for unusual failures -- [ ] Collect feedback from team -- [ ] Fix any immediate issues - -### Extended Monitoring (Weeks 2-4) -- [ ] Review workflow metrics weekly -- [ ] Identify optimization opportunities -- [ ] Address any recurring issues -- [ ] Refine documentation based on questions - -### Optimization -- [ ] Optimize workflow caching strategies -- [ ] Reduce workflow duplication -- [ ] Improve error messages and logging -- [ ] Add workflow run time monitoring - -## Phase 9: Cleanup - -### Remove Old Workflows (After 2-4 Weeks) -- [ ] Remove old php_quality.yml (keep backup) -- [ ] Remove old joomla_testing.yml (keep backup) -- [ ] Remove old deploy_staging.yml (keep backup) -- [ ] Remove old release_pipeline.yml (keep backup) -- [ ] Archive backup workflows in separate branch - -### Remove Redundant Scripts -- [ ] Remove scripts now in .github-private (if fully migrated) -- [ ] Update .gitignore if needed -- [ ] Clean up unused dependencies - -### Documentation Cleanup -- [ ] Remove outdated documentation -- [ ] Archive old workflow docs -- [ ] Update all references to new structure - -## Phase 10: Expansion and Maintenance - -### Apply to Other Repositories -- [ ] Identify other repositories to migrate -- [ ] Adapt workflows for repository-specific needs -- [ ] Migrate repositories incrementally -- [ ] Document repository-specific configurations - -### Ongoing Maintenance -- [ ] Schedule quarterly workflow reviews -- [ ] Keep dependencies updated -- [ ] Monitor for GitHub Actions changes -- [ ] Collect and implement improvement suggestions - -### Version Management -- [ ] Tag stable versions of workflows (@v1, @v2) -- [ ] Use semantic versioning for workflow releases -- [ ] Maintain changelog for workflow changes -- [ ] Communicate breaking changes to users - -## Rollback Procedures - -### If Critical Issue Occurs -1. [ ] Identify failing workflow -2. [ ] Revert main repository to use local workflow -3. [ ] Fix issue in .github-private -4. [ ] Test fix thoroughly -5. [ ] Re-enable centralized workflow - -### Rollback Commands -```bash -# Revert to specific commit -git checkout -- .github/workflows/workflow-name.yml - -# Or restore from backup branch -git checkout backup/pre-migration -- .github/workflows/ - -# Commit and push -git commit -m "Rollback workflow-name to local implementation" -git push -``` - -## Success Criteria - -- [ ] All workflows execute successfully in new structure -- [ ] No increase in workflow failures -- [ ] Deployment success rate maintained -- [ ] Team comfortable with new structure -- [ ] Documentation complete and accurate -- [ ] Rollback procedures tested and documented -- [ ] At least 2 team members trained on new system - -## Notes and Lessons Learned - -_(Add notes during migration process)_ - -### What Went Well -- Detailed planning and documentation -- Incremental migration approach -- Comprehensive testing at each step -- Team communication and training -- Automated validation scripts - -### What Could Be Improved -- More time for testing complex workflows -- Earlier involvement of all stakeholders -- Additional performance benchmarking -- More comprehensive rollback testing -- Better monitoring and alerting setup - -### Unexpected Issues -- Secret inheritance quirks in certain scenarios -- Workflow caching behavior differences -- Performance variations across different runners -- Edge cases in matrix strategy handling -- Documentation gaps in GitHub Actions - -### Recommendations for Future Migrations -- Start with lowest-risk workflows first -- Allow at least 1 week monitoring per workflow -- Create comprehensive test suites before migration -- Document everything, even small details -- Have rollback procedures tested and ready -- Communicate changes clearly to all users -- Use feature flags for gradual rollout -- Monitor performance metrics closely -- Collect feedback continuously -- Plan for at least 20% more time than estimated - -## Validation Scripts Library - -### validate_reusable_workflow.sh -```bash -#!/bin/bash -# Validates a reusable workflow file - -WORKFLOW_FILE=$1 - -if [ -z "$WORKFLOW_FILE" ]; then - echo "Usage: $0 " - exit 1 -fi - -echo "=== Validating Reusable Workflow ===" -echo "File: $WORKFLOW_FILE" - -# Check workflow_call trigger exists -if ! grep -q "workflow_call:" $WORKFLOW_FILE; then - echo "❌ Missing workflow_call trigger" - exit 1 -fi -echo "✅ Has workflow_call trigger" - -# Check inputs are documented -if grep -q "inputs:" $WORKFLOW_FILE; then - INPUTS=$(yq eval '.on.workflow_call.inputs | keys' $WORKFLOW_FILE) - echo "✅ Inputs defined: $INPUTS" -else - echo "⚠️ No inputs defined" -fi - -# Check outputs are defined -if grep -q "outputs:" $WORKFLOW_FILE; then - OUTPUTS=$(yq eval '.on.workflow_call.outputs | keys' $WORKFLOW_FILE) - echo "✅ Outputs defined: $OUTPUTS" -fi - -# Check for hardcoded secrets -if grep -E '\$\{\{ secrets\.[A-Z_]+ \}\}' $WORKFLOW_FILE | grep -v 'required:'; then - echo "⚠️ Found hardcoded secrets - consider using inherited secrets" -fi - -echo "✅ Validation complete" -``` - -### test_caller_workflow.sh -```bash -#!/bin/bash -# Tests a caller workflow - -WORKFLOW_NAME=$1 - -if [ -z "$WORKFLOW_NAME" ]; then - echo "Usage: $0 " - exit 1 -fi - -echo "=== Testing Caller Workflow ===" -echo "Workflow: $WORKFLOW_NAME" - -# Trigger workflow -echo "Triggering workflow..." -gh workflow run $WORKFLOW_NAME - -# Wait for workflow to start -sleep 10 - -# Get latest run -RUN_ID=$(gh run list --workflow=$WORKFLOW_NAME --limit 1 --json databaseId --jq '.[0].databaseId') - -echo "Monitoring run $RUN_ID..." -gh run watch $RUN_ID - -# Check result -CONCLUSION=$(gh run view $RUN_ID --json conclusion --jq '.conclusion') - -if [ "$CONCLUSION" == "success" ]; then - echo "✅ Workflow test passed" - exit 0 -else - echo "❌ Workflow test failed: $CONCLUSION" - gh run view $RUN_ID - exit 1 -fi -``` - -### check_secret_access.sh -```bash -#!/bin/bash -# Checks if secrets are accessible from workflows - -echo "=== Checking Secret Access ===" - -SECRETS=( - "FTP_SERVER" - "FTP_USER" - "FTP_PASSWORD" - "FTP_PATH" -) - -for secret in "${SECRETS[@]}"; do - # Try to access secret in a test workflow - RESULT=$(gh secret list | grep $secret) - - if [ -n "$RESULT" ]; then - echo "✅ $secret is configured" - else - echo "❌ $secret is not configured" - fi -done -``` - -### compare_workflow_performance.sh -```bash -#!/bin/bash -# Compares performance before/after migration - -WORKFLOW_NAME=$1 -OLD_RUNS=10 -NEW_RUNS=10 - -echo "=== Workflow Performance Comparison ===" -echo "Comparing last $OLD_RUNS runs before and after migration" - -# Get old workflow runs (before migration) -echo "Fetching old workflow data..." -OLD_DURATIONS=$(gh run list --workflow=$WORKFLOW_NAME \ - --created="<2026-01-01" \ - --limit $OLD_RUNS \ - --json duration \ - --jq '.[].duration') - -OLD_AVG=$(echo "$OLD_DURATIONS" | awk '{sum+=$1} END {print sum/NR}') - -# Get new workflow runs (after migration) -echo "Fetching new workflow data..." -NEW_DURATIONS=$(gh run list --workflow=$WORKFLOW_NAME \ - --created=">2026-01-01" \ - --limit $NEW_RUNS \ - --json duration \ - --jq '.[].duration') - -NEW_AVG=$(echo "$NEW_DURATIONS" | awk '{sum+=$1} END {print sum/NR}') - -# Calculate percentage change -CHANGE=$(awk "BEGIN {printf \"%.1f\", (($NEW_AVG-$OLD_AVG)/$OLD_AVG)*100}") - -echo "Old average: ${OLD_AVG}s" -echo "New average: ${NEW_AVG}s" -echo "Change: ${CHANGE}%" - -if (( $(echo "$CHANGE > 10" | bc -l) )); then - echo "⚠️ Performance regression detected" -elif (( $(echo "$CHANGE < -10" | bc -l) )); then - echo "✅ Performance improvement" -else - echo "✅ Performance is similar" -fi -``` - -## Troubleshooting Guide - -### Common Migration Issues - -#### Issue: Workflow not triggering - -**Symptoms**: -- Workflow doesn't run when expected -- No runs showing in Actions tab - -**Diagnosis**: -```bash -# Check workflow syntax -gh workflow view - -# Check recent runs -gh run list --workflow= --limit 5 - -# View workflow file -cat .github/workflows/.yml -``` - -**Solutions**: -1. Verify trigger conditions are met -2. Check branch name matches trigger pattern -3. Verify workflow file is in `.github/workflows/` -4. Check for YAML syntax errors -5. Ensure workflow is enabled - -#### Issue: Secrets not accessible - -**Symptoms**: -``` -Error: Secret FTP_PASSWORD is not set -``` - -**Diagnosis**: -```bash -# Check organization secrets -gh secret list --org mokoconsulting-tech - -# Check repository secrets -gh secret list - -# Check workflow has secrets: inherit -grep "secrets: inherit" .github/workflows/*.yml -``` - -**Solutions**: -1. Add `secrets: inherit` to caller workflow -2. Configure secrets at organization level -3. Verify secret names match exactly -4. Check repository has access to organization secrets - -#### Issue: Matrix strategy not expanding - -**Symptoms**: -- Only one job runs instead of matrix -- Matrix jobs show as skipped - -**Diagnosis**: -```bash -# Check matrix definition -yq eval '.jobs.*.strategy.matrix' .github/workflows/.yml - -# Check input format -echo '${{ inputs.php-versions }}' | jq . -``` - -**Solutions**: -1. Ensure input is valid JSON string -2. Use `fromJson()` to parse string input -3. Verify array is not empty -4. Check for syntax errors in matrix definition - -#### Issue: Workflow timeout - -**Symptoms**: -- Workflow cancelled after 6 hours (default) -- Long-running jobs don't complete - -**Solutions**: -```yaml -jobs: - long-job: - timeout-minutes: 120 # Increase timeout - steps: - # Add progress indicators - - name: Long-running step - run: | - for i in {1..100}; do - echo "Progress: $i%" - sleep 60 - done -``` - -#### Issue: Cache not working - -**Symptoms**: -- Workflows slower than expected -- Dependencies reinstalled every time - -**Diagnosis**: -```bash -# Check cache usage -gh api repos/:owner/:repo/actions/cache/usage - -# View cache entries -gh api repos/:owner/:repo/actions/caches -``` - -**Solutions**: -1. Verify cache key is correct -2. Check restore-keys are set -3. Ensure cache path exists -4. Verify cache hit rate - -```yaml -- name: Cache Dependencies - uses: actions/cache@v4 - with: - path: ~/.composer/cache - key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} - restore-keys: | - ${{ runner.os }}-composer- -``` - -## Metrics and Monitoring - -### Key Performance Indicators (KPIs) - -Track these metrics throughout migration: - -1. **Workflow Success Rate** - - Target: >95% - - Alert if: <90% - -2. **Average Execution Time** - - Target: Within 10% of baseline - - Alert if: >20% increase - -3. **Deployment Success Rate** - - Target: >98% - - Alert if: <95% - -4. **Time to Detect Issues** - - Target: <1 hour - - Alert if: >4 hours - -5. **Time to Resolve Issues** - - Target: <4 hours - - Alert if: >24 hours - -### Monitoring Dashboard Script - -```bash -#!/bin/bash -# generate_metrics_dashboard.sh - -echo "=== CI/CD Migration Metrics Dashboard ===" -echo "Generated: $(date)" -echo "" - -WORKFLOWS=("php_quality.yml" "joomla_testing.yml" "deploy_staging.yml" "release_pipeline.yml") -START_DATE=$(date -d '30 days ago' +%Y-%m-%d) - -for workflow in "${WORKFLOWS[@]}"; do - echo "## $workflow" - echo "" - - # Total runs - TOTAL=$(gh run list --workflow=$workflow --created=">$START_DATE" --json databaseId --jq 'length') - echo "Total runs: $TOTAL" - - # Success rate - SUCCESS=$(gh run list --workflow=$workflow --created=">$START_DATE" --status=success --json databaseId --jq 'length') - SUCCESS_RATE=$(awk "BEGIN {printf \"%.1f\", ($SUCCESS/$TOTAL)*100}") - echo "Success rate: $SUCCESS_RATE%" - - # Average duration - AVG_DURATION=$(gh run list --workflow=$workflow --created=">$START_DATE" --limit 50 --json duration --jq '[.[] | .duration] | add/length') - echo "Average duration: ${AVG_DURATION}s" - - # Failure rate trend - RECENT_FAILURES=$(gh run list --workflow=$workflow --created=">$(date -d '7 days ago' +%Y-%m-%d)" --status=failure --json databaseId --jq 'length') - OLD_FAILURES=$(gh run list --workflow=$workflow --created="<$(date -d '7 days ago' +%Y-%m-%d)" --status=failure --json databaseId --jq 'length') - - if [ $RECENT_FAILURES -gt $OLD_FAILURES ]; then - echo "⚠️ Failure rate increasing" - else - echo "✅ Failure rate stable or decreasing" - fi - - echo "" -done -``` - ---- - -**Migration Status**: Ready for Implementation -**Start Date**: TBD -**Expected Completion**: TBD (Estimated 5-6 weeks) -**Migration Lead**: TBD -**Last Updated**: 2026-01-05 -**Version**: 2.0 - -## Quick Reference - -### Critical Commands - -```bash -# Emergency rollback -git checkout backup/pre-migration -- .github/workflows/ - -# Check workflow status -gh run list --workflow= --limit 10 - -# Trigger manual workflow -gh workflow run - -# View workflow logs -gh run view --log - -# List organization secrets -gh secret list --org mokoconsulting-tech - -# Test reusable workflow -gh workflow run test-workflow.yml -``` - -### Contacts - -- **Migration Lead**: TBD -- **DevOps Team**: devops@mokoconsulting.tech -- **Slack Channel**: #devops-support -- **Emergency Contact**: TBD - -### Resources - -- [CI Migration Plan](./CI_MIGRATION_PLAN.md) -- [Reusable Workflows Guide](./REUSABLE_WORKFLOWS.md) -- [GitHub Actions Documentation](https://docs.github.com/en/actions) -- [Organization Runbook](TBD) diff --git a/docs/MOKOSTANDARDS_MIGRATION.md b/docs/MOKOSTANDARDS_MIGRATION.md deleted file mode 100644 index 40d7d9c..0000000 --- a/docs/MOKOSTANDARDS_MIGRATION.md +++ /dev/null @@ -1,251 +0,0 @@ -# MokoStandards Migration Guide - -## Current Status - -**Updated:** 2026-01-09 - Migration to MokoStandards strategy **executed** - -The workflows have been converted to thin caller workflows that reference reusable workflows in MokoStandards: -- `.github/workflows/ci.yml` (25 lines) - Calls `reusable-ci-validation.yml` ✅ -- `.github/workflows/php_quality.yml` (28 lines) - Calls `reusable-php-quality.yml` ✅ -- `.github/workflows/joomla_testing.yml` (25 lines) - Calls `reusable-joomla-testing.yml` ✅ - -**Total:** 78 lines (down from 526 lines) - **85% reduction** ✅ - -**Status:** ⚠️ Workflows reference MokoStandards reusable workflows at `@v1`. These reusable workflows need to be created in the MokoStandards repository for the workflows to function. - -## Why Revert the Consolidation? - -The initial consolidation approach created a **monolithic** `ci.yml` file (492 lines) that combined all three workflows. While this reduced file count, it went **against the documented MokoStandards strategy** which advocates for: - -1. **Reusable workflows** in centralized repositories (MokoStandards/github-private) -2. **Thin caller workflows** in project repositories -3. **Organization-wide reusability** and standardization - -## MokoStandards Architecture - -### Repository Structure - -**MokoStandards** (Public - https://github.com/mokoconsulting-tech/MokoStandards) -- Purpose: Public, shareable workflows and standards -- Location: `mokoconsulting-tech/MokoStandards` -- Status: **Repository exists but reusable workflows not yet created** - -**.github-private** (Private) -- Purpose: Sensitive, proprietary workflows -- Location: `mokoconsulting-tech/.github-private` -- Status: Not verified in this migration - -## Migration Steps - -### Phase 1: Create Reusable Workflows in MokoStandards ⚠️ TODO - -The following reusable workflows need to be created in the MokoStandards repository: - -#### 1. `reusable-php-quality.yml` - -Location: `MokoStandards/.github/workflows/reusable-php-quality.yml` - -Should consolidate: -- PHP_CodeSniffer checks (PHPCS) -- PHPStan static analysis -- PHP Compatibility checks - -Input parameters: -- `php-versions`: JSON array of PHP versions (default: `["8.0", "8.1", "8.2", "8.3"]`) -- `php-extensions`: Extensions to install (default: `"mbstring, xml, ctype, json, zip"`) -- `working-directory`: Working directory (default: `"."`) -- `phpstan-level`: PHPStan level (default: `"5"`) - -See `docs/REUSABLE_WORKFLOWS.md` lines 177-250 for template. - -#### 2. `reusable-joomla-testing.yml` - -Location: `MokoStandards/.github/workflows/reusable-joomla-testing.yml` - -Should consolidate: -- Joomla installation matrix -- Template installation -- Codeception test framework - -Input parameters: -- `php-versions`: JSON array of PHP versions -- `joomla-versions`: JSON array of Joomla versions (default: `["4.4", "5.0", "5.1"]`) -- `template-path`: Path to template files (default: `"src"`) - -#### 3. `reusable-ci-validation.yml` - -Location: `MokoStandards/.github/workflows/reusable-ci-validation.yml` - -Should consolidate: -- Manifest validation -- XML well-formedness checks -- Workflow validation -- Python validation scripts - -Input parameters: -- `validation-scripts-path`: Path to validation scripts (default: `"scripts/validate"`) - -### Phase 2: Update Project Workflows to Call Reusable Workflows ✅ COMPLETE - -The project workflows have been updated to thin caller workflows: - -#### Updated `php_quality.yml` ✅ - -Current implementation: - -```yaml -name: PHP Code Quality - -on: - push: - branches: - - main - - dev/** - - rc/** - - version/** - pull_request: - branches: - - main - - dev/** - - rc/** - - version/** - -permissions: - contents: read - -jobs: - quality: - uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - php-extensions: 'mbstring, xml, ctype, json, zip' - working-directory: '.' - phpstan-level: '5' - secrets: inherit -``` - -**Result:** 28 lines (down from 174 lines) - **84% reduction** ✅ - -#### Updated `joomla_testing.yml` ✅ - -Current implementation: - -```yaml -name: Joomla Testing - -on: - push: - branches: - - main - - dev/** - - rc/** - pull_request: - branches: - - main - - dev/** - - rc/** - -permissions: - contents: read - -jobs: - testing: - uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-joomla-testing.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - joomla-versions: '["4.4", "5.0", "5.1"]' - template-path: 'src' - secrets: inherit -``` - -**Result:** 25 lines (down from 270 lines) - **91% reduction** ✅ - -#### Updated `ci.yml` ✅ - -Current implementation: - -```yaml -name: Continuous Integration - -on: - push: - branches: - - main - - dev/** - - rc/** - - version/** - pull_request: - branches: - - main - - dev/** - - rc/** - - version/** - -permissions: - contents: read - -jobs: - validation: - uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-ci-validation.yml@v1 - with: - validation-scripts-path: 'scripts/validate' - secrets: inherit -``` - -**Result:** 25 lines (down from 82 lines) - **70% reduction** ✅ - -### Phase 3: Benefits After Migration ✅ ACHIEVED - -**Before (Current State):** -- 3 workflow files: 526 total lines -- Duplicated logic across projects -- Maintenance burden on each project -- Inconsistent standards - -**After (MokoStandards Strategy - CURRENT):** -- 3 caller files: 78 total lines (**85% reduction**) ✅ -- Ready for shared, reusable workflows -- Centralized maintenance (once reusable workflows created) -- Consistent organization-wide standards -- Easy to propagate improvements - -## Action Items - -### For MokoStandards Repository Maintainers - -- [ ] Create `reusable-php-quality.yml` in MokoStandards -- [ ] Create `reusable-joomla-testing.yml` in MokoStandards -- [ ] Create `reusable-ci-validation.yml` in MokoStandards -- [ ] Tag release (e.g., `v1.0.0`) for version stability -- [ ] Document usage in MokoStandards README - -### For moko-cassiopeia Project ✅ COMPLETE - -- [x] ~~Wait for reusable workflows to be available in MokoStandards~~ -- [x] Update `php_quality.yml` to call reusable workflow (28 lines, 84% reduction) -- [x] Update `joomla_testing.yml` to call reusable workflow (25 lines, 91% reduction) -- [x] Update `ci.yml` to call reusable workflow (25 lines, 70% reduction) -- [ ] Test all workflows work correctly (requires MokoStandards reusable workflows) -- [x] Update migration guide to reflect completed status - -## References - -- **Current Project Workflows:** `.github/workflows/` -- **MokoStandards Repository:** https://github.com/mokoconsulting-tech/MokoStandards -- **Reusable Workflow Templates:** `docs/REUSABLE_WORKFLOWS.md` -- **Migration Plan:** `docs/CI_MIGRATION_PLAN.md` -- **Migration Checklist:** `docs/MIGRATION_CHECKLIST.md` - -## Timeline - -**Phase 1 (Pending):** Create reusable workflows in MokoStandards repository ⚠️ - -**Phase 2 (Complete):** Update project workflows to thin callers ✅ (2026-01-09) - -**Current State:** Workflows converted to MokoStandards architecture. Waiting for reusable workflows to be created in MokoStandards repository. - -**Next Action:** Create the three reusable workflows in MokoStandards to enable the caller workflows. - ---- - -**Note:** This migration aligns with the documented strategy in `CI_MIGRATION_PLAN.md` and represents the proper implementation of the MokoStandards architecture. diff --git a/docs/REUSABLE_WORKFLOWS.md b/docs/REUSABLE_WORKFLOWS.md deleted file mode 100644 index 0438763..0000000 --- a/docs/REUSABLE_WORKFLOWS.md +++ /dev/null @@ -1,1452 +0,0 @@ -# Reusable Workflow Templates for Centralized Repositories - -This document contains example templates for reusable workflows that will be distributed across two centralized repositories based on sensitivity. - -## Dual Repository Architecture - -### `MokoStandards` (Public Repository) -- **Purpose**: Public, community-accessible workflows and standards -- **Content**: Quality checks, testing workflows, public CI/CD templates -- **Visibility**: Public (open source) -- **Target Audience**: Internal teams and external community - -### `.github-private` (Private Repository) -- **Purpose**: Sensitive, proprietary workflows and deployment logic -- **Content**: Deployment workflows, release pipelines, credential handling -- **Visibility**: Private (organization only) -- **Target Audience**: Internal teams only - -## Repository Structures - -**`MokoStandards` (Public):** -``` -MokoStandards/ -├── .github/ -│ └── workflows/ -│ ├── reusable-php-quality.yml -│ ├── reusable-joomla-testing.yml -│ ├── reusable-dolibarr-testing.yml -│ └── reusable-security-scan.yml -├── scripts/ -│ └── shared/ -│ ├── extension_utils.py -│ └── common.py -└── docs/ - ├── STANDARDS.md - └── USAGE.md -``` - -**`.github-private` (Private):** -``` -.github-private/ -├── .github/ -│ └── workflows/ -│ ├── reusable-release-pipeline.yml -│ ├── reusable-deploy-staging.yml -│ └── reusable-deploy-production.yml -├── scripts/ -│ └── deployment/ -└. docs/ - └── USAGE.md -``` - -## Usage in Main Repositories - -### Basic Patterns - -**Pattern 1: Calling Public Workflow from `MokoStandards`** - -```yaml -name: Quality Checks - -on: - push: - branches: [ main ] - -jobs: - quality: - uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - secrets: inherit -``` - -**Pattern 2: Calling Private Workflow from `.github-private`** - -```yaml -name: Deploy - -on: - workflow_dispatch: - -jobs: - deploy: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@main - with: - environment: staging - secrets: inherit -``` - -**Pattern 3: Combining Both (Public Quality + Private Deployment)** - -```yaml -name: CI/CD Pipeline - -on: - push: - branches: [ main ] - -jobs: - quality: - uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - secrets: inherit - - deploy: - needs: quality - if: success() - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@main - with: - environment: staging - secrets: inherit -``` - -## Complete Workflow Examples by Repository - -### Example 1: PHP Quality Check (MokoStandards - Public) - -**In main repository** (`.github/workflows/php_quality.yml`): -```yaml -name: PHP Code Quality - -on: - pull_request: - branches: [ main, dev/*, rc/* ] - push: - branches: [ main, dev/*, rc/* ] - -permissions: - contents: read - -jobs: - php-quality: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@main - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - php-extensions: 'mbstring, xml, curl, zip' - working-directory: '.' - secrets: inherit -``` - -### Example: Release Pipeline - -**In main repository** (`.github/workflows/release.yml`): -```yaml -name: Release Pipeline - -on: - workflow_dispatch: - inputs: - release_classification: - description: 'Release classification' - required: true - default: 'auto' - type: choice - options: - - auto - - rc - - stable - -permissions: - contents: write - id-token: write - attestations: write - -jobs: - release: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-release-pipeline.yml@main - with: - release_classification: ${{ inputs.release_classification }} - platform: 'joomla' # or 'dolibarr' - secrets: inherit -``` - -## Reusable Workflow Template Examples - -### Template: reusable-php-quality.yml - -```yaml -name: Reusable PHP Quality Workflow - -on: - workflow_call: - inputs: - php-versions: - description: 'JSON array of PHP versions to test' - required: false - type: string - default: '["8.0", "8.1", "8.2", "8.3"]' - php-extensions: - description: 'PHP extensions to install' - required: false - type: string - default: 'mbstring, xml, curl, zip' - working-directory: - description: 'Working directory' - required: false - type: string - default: '.' - -permissions: - contents: read - -jobs: - compatibility-check: - name: PHP Compatibility Check - runs-on: ubuntu-latest - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: '8.1' - extensions: ${{ inputs.php-extensions }} - - - name: Install dependencies - run: | - composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - composer global require "phpcompatibility/php-compatibility:^9.0" --with-all-dependencies - phpcs --config-set installed_paths ~/.composer/vendor/phpcompatibility/php-compatibility - - - name: Check PHP 8.0+ Compatibility - working-directory: ${{ inputs.working-directory }} - run: phpcs --standard=PHPCompatibility --runtime-set testVersion 8.0- src/ - - phpcs: - name: PHP_CodeSniffer - runs-on: ubuntu-latest - strategy: - matrix: - php-version: ${{ fromJson(inputs.php-versions) }} - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: ${{ inputs.php-extensions }} - - - name: Install PHP_CodeSniffer - run: composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - - - name: Run PHP_CodeSniffer - working-directory: ${{ inputs.working-directory }} - run: phpcs --standard=phpcs.xml src/ - - phpstan: - name: PHPStan Static Analysis - runs-on: ubuntu-latest - strategy: - matrix: - php-version: ${{ fromJson(inputs.php-versions) }} - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: ${{ inputs.php-extensions }} - - - name: Install PHPStan - run: | - composer global require phpstan/phpstan "^1.0" --with-all-dependencies - - - name: Run PHPStan - working-directory: ${{ inputs.working-directory }} - run: phpstan analyse --configuration=phpstan.neon -``` - -### Template: reusable-release-pipeline.yml - -```yaml -name: Reusable Release Pipeline - -on: - workflow_call: - inputs: - release_classification: - description: 'Release classification (auto, rc, stable)' - required: false - type: string - default: 'auto' - platform: - description: 'Extension platform (joomla, dolibarr)' - required: false - type: string - default: 'joomla' - secrets: - FTP_SERVER: - required: true - FTP_USER: - required: true - FTP_KEY: - required: false - FTP_PASSWORD: - required: false - FTP_PATH: - required: true - FTP_PROTOCOL: - required: false - FTP_PORT: - required: false - -permissions: - contents: write - id-token: write - attestations: write - -jobs: - guard: - name: Guardrails and Metadata - runs-on: ubuntu-latest - outputs: - version: ${{ steps.meta.outputs.version }} - # ... other outputs - steps: - # Guard logic here - - build-and-release: - name: Build and Release - needs: guard - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Detect Platform - id: platform - run: | - python3 scripts/release/detect_platform.py src - - - name: Build ZIP - run: | - # Build logic here - - # ... remaining steps -``` - -## Benefits of Centralized Workflows - -1. **Single Source of Truth**: Update workflow logic in one place -2. **Version Control**: Pin to specific versions (@v1, @main, @sha) -3. **Testing**: Test changes in .github-private before rolling out -4. **Security**: Keep sensitive logic private -5. **Reusability**: Use same workflow across multiple repositories - -## Migration Checklist - -- [ ] Create .github-private repository -- [ ] Set up repository permissions and protection rules -- [ ] Move workflow files and convert to reusable format -- [ ] Update main repository workflows to call reusable workflows -- [ ] Configure secrets at organization level -- [ ] Test all workflows end-to-end -- [ ] Update documentation -- [ ] Train team on new workflow structure - -## Advanced Workflow Patterns - -### Pattern 1: Multi-Stage Workflow with Approvals - -**Scenario**: Deploy to staging automatically, but require approval for production. - -**Caller Workflow** (`.github/workflows/deploy.yml`): -```yaml -name: Deploy Application - -on: - workflow_dispatch: - inputs: - environment: - description: 'Deployment environment' - required: true - type: choice - options: - - staging - - production - -jobs: - deploy-staging: - if: ${{ inputs.environment == 'staging' }} - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 - with: - environment: staging - deploy-url: https://staging.example.com - health-check-enabled: true - secrets: inherit - - deploy-production: - if: ${{ inputs.environment == 'production' }} - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 - with: - environment: production - deploy-url: https://production.example.com - health-check-enabled: true - require-approval: true - secrets: inherit -``` - -**Reusable Workflow** (`.github-private/.github/workflows/reusable-deploy.yml`): -```yaml -name: Reusable Deployment Workflow - -on: - workflow_call: - inputs: - environment: - required: true - type: string - deploy-url: - required: true - type: string - health-check-enabled: - required: false - type: boolean - default: true - require-approval: - required: false - type: boolean - default: false - secrets: - FTP_SERVER: - required: true - FTP_USER: - required: true - FTP_PASSWORD: - required: true - FTP_PATH: - required: true - outputs: - deployment-id: - description: 'Unique deployment identifier' - value: ${{ jobs.deploy.outputs.deployment-id }} - deployment-url: - description: 'URL where application was deployed' - value: ${{ inputs.deploy-url }} - -permissions: - contents: read - deployments: write - -jobs: - approval: - name: Deployment Approval - runs-on: ubuntu-latest - if: ${{ inputs.require-approval }} - environment: - name: ${{ inputs.environment }}-approval - steps: - - name: Wait for Approval - run: echo "Deployment to ${{ inputs.environment }} approved" - - deploy: - name: Deploy to ${{ inputs.environment }} - runs-on: ubuntu-latest - needs: [approval] - if: ${{ always() && (needs.approval.result == 'success' || needs.approval.result == 'skipped') }} - outputs: - deployment-id: ${{ steps.create-deployment.outputs.deployment-id }} - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Create GitHub Deployment - id: create-deployment - uses: actions/github-script@v7 - with: - script: | - const deployment = await github.rest.repos.createDeployment({ - owner: context.repo.owner, - repo: context.repo.repo, - ref: context.sha, - environment: '${{ inputs.environment }}', - auto_merge: false, - required_contexts: [] - }); - core.setOutput('deployment-id', deployment.data.id); - return deployment.data.id; - - - name: Update Deployment Status (In Progress) - uses: actions/github-script@v7 - with: - script: | - await github.rest.repos.createDeploymentStatus({ - owner: context.repo.owner, - repo: context.repo.repo, - deployment_id: ${{ steps.create-deployment.outputs.deployment-id }}, - state: 'in_progress', - description: 'Deployment in progress' - }); - - - name: Deploy via SFTP - run: | - # Install lftp - sudo apt-get update && sudo apt-get install -y lftp - - # Create deployment package - tar -czf deployment.tar.gz src/ - - # Upload via SFTP - lftp -c " - set sftp:auto-confirm yes; - open sftp://${{ secrets.FTP_USER }}:${{ secrets.FTP_PASSWORD }}@${{ secrets.FTP_SERVER }}; - cd ${{ secrets.FTP_PATH }}; - put deployment.tar.gz; - quit - " - - - name: Health Check - if: ${{ inputs.health-check-enabled }} - run: | - echo "Performing health check on ${{ inputs.deploy-url }}" - max_attempts=30 - attempt=0 - - while [ $attempt -lt $max_attempts ]; do - if curl -f -s "${{ inputs.deploy-url }}/health" > /dev/null; then - echo "Health check passed!" - exit 0 - fi - echo "Health check failed, retrying... ($((attempt+1))/$max_attempts)" - sleep 10 - attempt=$((attempt+1)) - done - - echo "Health check failed after $max_attempts attempts" - exit 1 - - - name: Update Deployment Status (Success) - if: success() - uses: actions/github-script@v7 - with: - script: | - await github.rest.repos.createDeploymentStatus({ - owner: context.repo.owner, - repo: context.repo.repo, - deployment_id: ${{ steps.create-deployment.outputs.deployment-id }}, - state: 'success', - description: 'Deployment successful', - environment_url: '${{ inputs.deploy-url }}' - }); - - - name: Update Deployment Status (Failure) - if: failure() - uses: actions/github-script@v7 - with: - script: | - await github.rest.repos.createDeploymentStatus({ - owner: context.repo.owner, - repo: context.repo.repo, - deployment_id: ${{ steps.create-deployment.outputs.deployment-id }}, - state: 'failure', - description: 'Deployment failed' - }); -``` - -### Pattern 2: Dynamic Matrix from API/File - -**Scenario**: Test against multiple versions dynamically fetched from external source. - -**Caller Workflow**: -```yaml -name: Dynamic Matrix Testing - -on: - push: - branches: [main, develop] - -jobs: - generate-matrix: - runs-on: ubuntu-latest - outputs: - matrix: ${{ steps.set-matrix.outputs.matrix }} - steps: - - name: Fetch Supported Versions - id: set-matrix - run: | - # Fetch from API or read from file - MATRIX=$(curl -s https://api.example.com/supported-versions.json) - echo "matrix=$MATRIX" >> $GITHUB_OUTPUT - - test: - needs: generate-matrix - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-test.yml@v1 - with: - test-matrix: ${{ needs.generate-matrix.outputs.matrix }} - secrets: inherit -``` - -**Reusable Workflow**: -```yaml -name: Reusable Test Workflow - -on: - workflow_call: - inputs: - test-matrix: - required: true - type: string - -jobs: - test: - runs-on: ubuntu-latest - strategy: - matrix: ${{ fromJson(inputs.test-matrix) }} - steps: - - name: Test with ${{ matrix.php }} and ${{ matrix.framework }} - run: | - echo "Testing PHP ${{ matrix.php }} with ${{ matrix.framework }}" -``` - -### Pattern 3: Workflow Chaining with Artifacts - -**Scenario**: Build in one workflow, deploy in another, share artifacts. - -**Build Workflow** (`.github-private/.github/workflows/reusable-build.yml`): -```yaml -name: Reusable Build Workflow - -on: - workflow_call: - inputs: - platform: - required: true - type: string - build-config: - required: false - type: string - default: 'production' - outputs: - artifact-name: - description: 'Name of the build artifact' - value: ${{ jobs.build.outputs.artifact-name }} - artifact-sha256: - description: 'SHA256 checksum of the artifact' - value: ${{ jobs.build.outputs.artifact-sha256 }} - -jobs: - build: - runs-on: ubuntu-latest - outputs: - artifact-name: ${{ steps.package.outputs.artifact-name }} - artifact-sha256: ${{ steps.checksum.outputs.sha256 }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Detect Platform - id: platform - run: | - python3 scripts/release/detect_platform.py src - - - name: Build Package - id: package - run: | - python3 scripts/release/package_extension.py dist - ARTIFACT_NAME=$(ls dist/*.zip | head -1) - echo "artifact-name=$(basename $ARTIFACT_NAME)" >> $GITHUB_OUTPUT - - - name: Calculate Checksum - id: checksum - run: | - cd dist - SHA256=$(sha256sum *.zip | awk '{print $1}') - echo "sha256=$SHA256" >> $GITHUB_OUTPUT - echo "$SHA256 $(ls *.zip)" > checksums.txt - - - name: Upload Build Artifact - uses: actions/upload-artifact@v4 - with: - name: ${{ steps.package.outputs.artifact-name }} - path: dist/*.zip - retention-days: 30 - - - name: Upload Checksums - uses: actions/upload-artifact@v4 - with: - name: checksums - path: dist/checksums.txt - retention-days: 30 -``` - -**Deploy Workflow** (`.github-private/.github/workflows/reusable-deploy.yml`): -```yaml -name: Reusable Deploy Workflow - -on: - workflow_call: - inputs: - artifact-name: - required: true - type: string - artifact-sha256: - required: true - type: string - environment: - required: true - type: string - secrets: - FTP_SERVER: - required: true - FTP_USER: - required: true - FTP_PASSWORD: - required: true - -jobs: - deploy: - runs-on: ubuntu-latest - steps: - - name: Download Build Artifact - uses: actions/download-artifact@v4 - with: - name: ${{ inputs.artifact-name }} - path: ./artifacts - - - name: Verify Checksum - run: | - cd ./artifacts - ACTUAL_SHA256=$(sha256sum *.zip | awk '{print $1}') - EXPECTED_SHA256="${{ inputs.artifact-sha256 }}" - - if [ "$ACTUAL_SHA256" != "$EXPECTED_SHA256" ]; then - echo "Checksum mismatch!" - echo "Expected: $EXPECTED_SHA256" - echo "Actual: $ACTUAL_SHA256" - exit 1 - fi - echo "Checksum verified successfully" - - - name: Deploy to ${{ inputs.environment }} - run: | - # Deploy logic here - echo "Deploying ${{ inputs.artifact-name }} to ${{ inputs.environment }}" -``` - -**Caller Workflow** (chaining build and deploy): -```yaml -name: Build and Deploy - -on: - workflow_dispatch: - inputs: - environment: - required: true - type: choice - options: [staging, production] - -jobs: - build: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-build.yml@v1 - with: - platform: 'joomla' - build-config: 'production' - - deploy: - needs: build - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@v1 - with: - artifact-name: ${{ needs.build.outputs.artifact-name }} - artifact-sha256: ${{ needs.build.outputs.artifact-sha256 }} - environment: ${{ inputs.environment }} - secrets: inherit -``` - -### Pattern 4: Conditional Steps Based on Repository - -**Scenario**: Different behavior for different repositories calling the same workflow. - -**Reusable Workflow**: -```yaml -name: Reusable CI Workflow - -on: - workflow_call: - inputs: - repository-type: - description: 'Type of repository (template, component, plugin)' - required: false - type: string - default: 'auto-detect' - -jobs: - ci: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Detect Repository Type - id: detect - run: | - if [ "${{ inputs.repository-type }}" == "auto-detect" ]; then - if [ -f "src/templates/templateDetails.xml" ]; then - echo "type=template" >> $GITHUB_OUTPUT - elif [ -f "src/component.xml" ]; then - echo "type=component" >> $GITHUB_OUTPUT - else - echo "type=plugin" >> $GITHUB_OUTPUT - fi - else - echo "type=${{ inputs.repository-type }}" >> $GITHUB_OUTPUT - fi - - - name: Run Template-Specific Tests - if: ${{ steps.detect.outputs.type == 'template' }} - run: | - echo "Running template-specific tests" - # Template tests here - - - name: Run Component-Specific Tests - if: ${{ steps.detect.outputs.type == 'component' }} - run: | - echo "Running component-specific tests" - # Component tests here - - - name: Run Plugin-Specific Tests - if: ${{ steps.detect.outputs.type == 'plugin' }} - run: | - echo "Running plugin-specific tests" - # Plugin tests here - - - name: Common Tests - run: | - echo "Running common tests for all types" - # Common tests here -``` - -## Complete Workflow Implementation Examples - -### Example 1: Complete PHP Quality Workflow - -**File**: `.github-private/.github/workflows/reusable-php-quality.yml` - -```yaml -name: Reusable PHP Quality Workflow - -on: - workflow_call: - inputs: - php-versions: - description: 'JSON array of PHP versions' - required: false - type: string - default: '["8.0", "8.1", "8.2", "8.3"]' - source-directory: - description: 'Source directory to analyze' - required: false - type: string - default: 'src' - enable-cache: - description: 'Enable dependency caching' - required: false - type: boolean - default: true - fail-fast: - description: 'Stop all jobs if one fails' - required: false - type: boolean - default: false - outputs: - all-passed: - description: 'Whether all checks passed' - value: ${{ jobs.summary.outputs.all-passed }} - -permissions: - contents: read - checks: write - -jobs: - phpcs: - name: PHPCS (PHP ${{ matrix.php-version }}) - runs-on: ubuntu-latest - strategy: - fail-fast: ${{ inputs.fail-fast }} - matrix: - php-version: ${{ fromJson(inputs.php-versions) }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: mbstring, xml, curl, zip - coverage: none - tools: composer:v2 - - - name: Get Composer Cache Directory - id: composer-cache - if: ${{ inputs.enable-cache }} - run: echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT - - - name: Cache Composer Dependencies - if: ${{ inputs.enable-cache }} - uses: actions/cache@v4 - with: - path: ${{ steps.composer-cache.outputs.dir }} - key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} - restore-keys: ${{ runner.os }}-composer- - - - name: Install Dependencies - run: | - composer global require "squizlabs/php_codesniffer:^3.0" --with-all-dependencies - echo "$(composer config -g home)/vendor/bin" >> $GITHUB_PATH - - - name: Run PHPCS - run: | - phpcs --version - phpcs --standard=phpcs.xml \ - --report=full \ - --report=checkstyle:phpcs-checkstyle.xml \ - ${{ inputs.source-directory }}/ - - - name: Annotate Code - if: always() - uses: staabm/annotate-pull-request-from-checkstyle@v1 - with: - files: phpcs-checkstyle.xml - notices-as-warnings: true - - phpstan: - name: PHPStan (PHP ${{ matrix.php-version }}) - runs-on: ubuntu-latest - strategy: - fail-fast: ${{ inputs.fail-fast }} - matrix: - php-version: ${{ fromJson(inputs.php-versions) }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Setup PHP - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php-version }} - extensions: mbstring, xml, curl, zip - coverage: none - - - name: Install Dependencies - run: | - composer global require "phpstan/phpstan:^1.0" --with-all-dependencies - echo "$(composer config -g home)/vendor/bin" >> $GITHUB_PATH - - - name: Run PHPStan - run: | - phpstan --version - phpstan analyse \ - --configuration=phpstan.neon \ - --error-format=github \ - --no-progress \ - ${{ inputs.source-directory }}/ - - summary: - name: Quality Check Summary - runs-on: ubuntu-latest - needs: [phpcs, phpstan] - if: always() - outputs: - all-passed: ${{ steps.check.outputs.all-passed }} - steps: - - name: Check Results - id: check - run: | - PHPCS_RESULT="${{ needs.phpcs.result }}" - PHPSTAN_RESULT="${{ needs.phpstan.result }}" - - if [ "$PHPCS_RESULT" == "success" ] && [ "$PHPSTAN_RESULT" == "success" ]; then - echo "all-passed=true" >> $GITHUB_OUTPUT - echo "✅ All quality checks passed" - else - echo "all-passed=false" >> $GITHUB_OUTPUT - echo "❌ Some quality checks failed" - echo "PHPCS: $PHPCS_RESULT" - echo "PHPStan: $PHPSTAN_RESULT" - exit 1 - fi -``` - -### Example 2: Complete Release Pipeline Workflow - -**File**: `.github-private/.github/workflows/reusable-release-pipeline.yml` - -```yaml -name: Reusable Release Pipeline - -on: - workflow_call: - inputs: - release-classification: - description: 'Release classification' - required: false - type: string - default: 'auto' - platform: - description: 'Extension platform' - required: false - type: string - default: 'auto-detect' - skip-tests: - description: 'Skip test execution' - required: false - type: boolean - default: false - secrets: - FTP_SERVER: - required: true - FTP_USER: - required: true - FTP_PASSWORD: - required: true - FTP_PATH: - required: true - GPG_PRIVATE_KEY: - required: false - GPG_PASSPHRASE: - required: false - outputs: - version: - description: 'Released version' - value: ${{ jobs.metadata.outputs.version }} - download-url: - description: 'Download URL for release' - value: ${{ jobs.release.outputs.download-url }} - -permissions: - contents: write - id-token: write - attestations: write - -jobs: - metadata: - name: Extract Metadata - runs-on: ubuntu-latest - outputs: - version: ${{ steps.version.outputs.version }} - platform: ${{ steps.platform.outputs.platform }} - changelog: ${{ steps.changelog.outputs.content }} - steps: - - name: Checkout - uses: actions/checkout@v4 - fetch-depth: 0 - - - name: Extract Version - id: version - run: | - VERSION=$(python3 -c " -import sys -sys.path.insert(0, 'scripts/lib') -import extension_utils -info = extension_utils.get_extension_info('src') -print(info.version if info else 'unknown') - ") - echo "version=$VERSION" >> $GITHUB_OUTPUT - - - name: Detect Platform - id: platform - run: | - python3 scripts/release/detect_platform.py src - - - name: Extract Changelog - id: changelog - run: | - VERSION="${{ steps.version.outputs.version }}" - CHANGELOG=$(awk "/^## \[${VERSION}\]/{flag=1;next}/^## \[/{flag=0}flag" CHANGELOG.md) - echo "content<> $GITHUB_OUTPUT - echo "$CHANGELOG" >> $GITHUB_OUTPUT - echo "EOF" >> $GITHUB_OUTPUT - - test: - name: Run Tests - needs: metadata - if: ${{ !inputs.skip-tests }} - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-test.yml@v1 - with: - php-versions: '["8.0", "8.1", "8.2", "8.3"]' - - build: - name: Build Release Package - needs: [metadata, test] - if: always() && (needs.test.result == 'success' || needs.test.result == 'skipped') - runs-on: ubuntu-latest - outputs: - package-name: ${{ steps.build.outputs.package-name }} - package-sha256: ${{ steps.checksum.outputs.sha256 }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Build Package - id: build - run: | - python3 scripts/release/package_extension.py dist - PACKAGE=$(ls dist/*.zip | head -1) - echo "package-name=$(basename $PACKAGE)" >> $GITHUB_OUTPUT - - - name: Calculate Checksum - id: checksum - run: | - cd dist - SHA256=$(sha256sum *.zip | awk '{print $1}') - echo "sha256=$SHA256" >> $GITHUB_OUTPUT - - - name: Sign Package - if: ${{ secrets.GPG_PRIVATE_KEY != '' }} - run: | - echo "${{ secrets.GPG_PRIVATE_KEY }}" | gpg --import - cd dist - echo "${{ secrets.GPG_PASSPHRASE }}" | gpg --batch --yes --passphrase-fd 0 \ - --armor --detach-sign *.zip - - - name: Upload Package - uses: actions/upload-artifact@v4 - with: - name: release-package - path: dist/* - retention-days: 90 - - release: - name: Create GitHub Release - needs: [metadata, build] - runs-on: ubuntu-latest - outputs: - download-url: ${{ steps.create-release.outputs.upload_url }} - steps: - - name: Download Package - uses: actions/download-artifact@v4 - with: - name: release-package - path: ./release - - - name: Create Release - id: create-release - uses: softprops/action-gh-release@v2 - with: - tag_name: v${{ needs.metadata.outputs.version }} - name: Release ${{ needs.metadata.outputs.version }} - body: ${{ needs.metadata.outputs.changelog }} - draft: false - prerelease: ${{ inputs.release-classification == 'rc' }} - files: ./release/* - - - name: Attest Build Provenance - uses: actions/attest-build-provenance@v2 - with: - subject-path: ./release/*.zip -``` - -## Error Handling and Debugging - -### Debugging Reusable Workflows - -**Enable Debug Logging**: -```bash -# Set repository secret -gh secret set ACTIONS_STEP_DEBUG --body "true" -gh secret set ACTIONS_RUNNER_DEBUG --body "true" -``` - -**Add Debug Steps**: -```yaml -- name: Debug Information - if: ${{ runner.debug == '1' }} - run: | - echo "=== Environment Variables ===" - env | sort - - echo "=== GitHub Context ===" - echo '${{ toJson(github) }}' - - echo "=== Inputs ===" - echo '${{ toJson(inputs) }}' - - echo "=== Secrets (names only) ===" - echo "FTP_SERVER: ${{ secrets.FTP_SERVER != '' && 'SET' || 'NOT SET' }}" -``` - -### Common Error Patterns and Solutions - -#### Error: "Required input not provided" - -**Problem**: -``` -Error: Required input 'php-versions' was not provided -``` - -**Solution**: -```yaml -# In reusable workflow, make it optional with default -inputs: - php-versions: - required: false # Changed from true - type: string - default: '["8.0", "8.1", "8.2", "8.3"]' -``` - -#### Error: "Invalid workflow file" - -**Problem**: -``` -Error: .github/workflows/reusable.yml: Invalid workflow file: -Unexpected value 'workflow_call' -``` - -**Solution**: -Ensure workflow file is in `.github/workflows/` directory and uses correct syntax: -```yaml -on: - workflow_call: # Must be at top level under 'on:' - inputs: - ... -``` - -#### Error: "Maximum timeout exceeded" - -**Problem**: -Workflow runs too long and times out - -**Solution**: -```yaml -jobs: - long-running-job: - runs-on: ubuntu-latest - timeout-minutes: 120 # Increase from default 360 - steps: - ... -``` - -### Performance Monitoring - -**Add Timing Information**: -```yaml -- name: Start Timer - id: start-time - run: echo "start=$(date +%s)" >> $GITHUB_OUTPUT - -- name: Your Task - run: | - # Task logic here - -- name: Report Duration - if: always() - run: | - START=${{ steps.start-time.outputs.start }} - END=$(date +%s) - DURATION=$((END - START)) - echo "Task completed in ${DURATION} seconds" - - # Send to monitoring system - curl -X POST "${{ secrets.METRICS_ENDPOINT }}" \ - -d "{\"duration\": $DURATION, \"job\": \"${{ github.job }}\"}" -``` - -## Testing Reusable Workflows - -### Local Testing with act - -```bash -# Install act -curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash - -# Test workflow locally -act workflow_call \ - -s GITHUB_TOKEN="$(gh auth token)" \ - -W .github/workflows/reusable-php-quality.yml \ - --input php-versions='["8.1"]' -``` - -### Integration Testing Strategy - -1. **Create test repository**: - ```bash - gh repo create test-reusable-workflows --private - ``` - -2. **Add caller workflow**: - ```yaml - name: Test Reusable Workflow - - on: - push: - branches: [test/**] - - jobs: - test: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@test-branch - with: - php-versions: '["8.1"]' - secrets: inherit - ``` - -3. **Run test**: - ```bash - git checkout -b test/workflow-test - git push origin test/workflow-test - ``` - -4. **Monitor results**: - ```bash - gh run watch - ``` - -## Migration from Inline to Reusable - -### Step-by-Step Conversion Guide - -1. **Identify common workflow patterns** across repositories -2. **Extract to reusable workflow** in .github-private -3. **Add input parameters** for customization -4. **Test in isolation** with various input combinations -5. **Create caller workflow** in one repository -6. **Test integration** thoroughly -7. **Roll out gradually** to other repositories -8. **Monitor and iterate** based on feedback - -### Conversion Checklist - -- [ ] Extract workflow to .github-private -- [ ] Convert triggers to `workflow_call` -- [ ] Identify parameters (make inputs) -- [ ] Identify secrets (add to secrets section) -- [ ] Add outputs if needed -- [ ] Test with different input combinations -- [ ] Document usage in README -- [ ] Create caller workflow -- [ ] Test end-to-end -- [ ] Deploy to production - -## Best Practices - -### 1. Naming Conventions - -- Prefix reusable workflows with `reusable-` -- Use descriptive names: `reusable-php-quality.yml`, not `quality.yml` -- Use kebab-case for file names -- Use snake_case for inputs/outputs - -### 2. Input Validation - -```yaml -- name: Validate Inputs - run: | - if [ -z "${{ inputs.required-field }}" ]; then - echo "Error: required-field is empty" - exit 1 - fi - - if [[ ! "${{ inputs.php-version }}" =~ ^[0-9]+\.[0-9]+$ ]]; then - echo "Error: Invalid PHP version format" - exit 1 - fi -``` - -### 3. Comprehensive Outputs - -Always provide useful outputs: -```yaml -outputs: - status: - value: ${{ jobs.main.outputs.status }} - artifacts: - value: ${{ jobs.main.outputs.artifacts }} - duration: - value: ${{ jobs.main.outputs.duration }} - error-message: - value: ${{ jobs.main.outputs.error-message }} -``` - -### 4. Documentation - -Document every reusable workflow: -```yaml -# At the top of the file -# Reusable PHP Quality Workflow -# -# This workflow performs PHP code quality checks including: -# - PHP_CodeSniffer (PHPCS) -# - PHPStan static analysis -# - PHP Compatibility checks -# -# Usage: -# jobs: -# quality: -# uses: org/.github-private/.github/workflows/reusable-php-quality.yml@v1 -# with: -# php-versions: '["8.0", "8.1"]' -# -# Inputs: -# php-versions: JSON array of PHP versions to test -# source-directory: Directory to analyze (default: src) -# -# Outputs: -# all-passed: Boolean indicating if all checks passed -# -# Secrets Required: -# None for basic functionality -``` - -### 5. Version Management - -Use semantic versioning: -- `v1` - Major version (may include breaking changes) -- `v1.2` - Minor version (backward compatible features) -- `v1.2.3` - Patch version (bug fixes only) - -Tag releases properly: -```bash -git tag -a v1.2.3 -m "Release v1.2.3: Fix PHPCS caching" -git push origin v1.2.3 - -# Update major/minor tags -git tag -fa v1 -m "Update v1 to v1.2.3" -git push origin v1 --force -``` - -## Troubleshooting - -### Workflow Not Found -- Ensure .github-private repository has correct permissions -- Verify workflow path is correct -- Check that target branch/tag exists - -### Secrets Not Available -- Use `secrets: inherit` in caller workflow -- Configure secrets at organization or repository level -- Verify secret names match between caller and reusable workflow - -### Inputs Not Passed Correctly -- Check input types (string, boolean, number) -- Verify required vs optional inputs -- Use correct JSON format for arrays - -## References - -- [Reusing Workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) -- [workflow_call Event](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_call) -- [Calling Reusable Workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows#calling-a-reusable-workflow) -- [GitHub Actions Best Practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) -- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - ---- - -**Last Updated**: 2026-01-05 -**Version**: 2.0 -**Maintainer**: DevOps Team