diff --git a/docs/CI_MIGRATION_PLAN.md b/docs/CI_MIGRATION_PLAN.md index ded1fe6..89155ea 100644 --- a/docs/CI_MIGRATION_PLAN.md +++ b/docs/CI_MIGRATION_PLAN.md @@ -1,13 +1,31 @@ -# CI/CD Migration to .github-private Repository +# CI/CD Migration to Centralized Repositories ## Purpose -This document outlines the plan and preparation steps for migrating CI/CD workflows to a centralized `.github-private` repository. This approach provides: +This document outlines the plan and preparation steps for migrating CI/CD workflows to centralized repositories. The organization uses a **dual-repository architecture**: -- **Security**: Keep sensitive CI/CD logic and configurations private +### Repository Architecture + +**`.github-private`** (Private & Secure Centralization) +- **Purpose**: Sensitive/proprietary workflows and deployment logic +- **Visibility**: Private (organization members only) +- **Content**: Deployment workflows, release pipelines, sensitive scripts +- **Use Case**: Production deployments, security-sensitive operations + +**`MokoStandards`** (Public Centralization) +- **Purpose**: Public/shareable workflows and best practices +- **Visibility**: Public (community accessible) +- **Content**: Quality checks, testing workflows, public templates +- **Use Case**: Open-source projects, community contributions, standardization + +This dual approach provides: + +- **Security**: Keep sensitive CI/CD logic and configurations private in `.github-private` +- **Community**: Share public workflows and standards via `MokoStandards` - **Reusability**: Share common workflows across multiple repositories -- **Maintainability**: Centralize CI/CD updates in one location +- **Maintainability**: Centralize CI/CD updates in one location per type - **Organization**: Separate CI/CD infrastructure from application code +- **Flexibility**: Choose appropriate repository based on workflow sensitivity ## Current State @@ -35,37 +53,73 @@ This document outlines the plan and preparation steps for migrating CI/CD workfl - Repository-specific configuration files - Application scripts that are part of the product -**Files to Move to .github-private:** -- Complex workflow definitions -- Shared workflow templates -- Sensitive deployment scripts -- Organization-wide CI/CD tooling +**Files to Move to Centralized Repositories:** + +**To `.github-private` (Private):** +- Deployment workflows (deploy_staging.yml, production deployments) +- Release pipelines with sensitive logic (release_pipeline.yml) +- Workflows containing proprietary business logic +- Scripts with deployment credentials handling + +**To `MokoStandards` (Public):** +- Quality check workflows (php_quality.yml) +- Testing workflows (joomla_testing.yml) +- Public CI/CD templates and examples +- Shared utility scripts (extension_utils.py, common.py) ### Phase 2: Structure Setup -Create `.github-private` repository with structure: +Create both centralized repositories with appropriate structure: + +**`.github-private` Repository Structure:** ``` .github-private/ ├── .github/ │ └── workflows/ -│ ├── reusable-php-quality.yml -│ ├── reusable-release-pipeline.yml -│ ├── reusable-joomla-testing.yml -│ └── reusable-deploy-staging.yml +│ ├── reusable-release-pipeline.yml (sensitive) +│ ├── reusable-deploy-staging.yml (sensitive) +│ └── reusable-deploy-production.yml (sensitive) +├── scripts/ +│ ├── deployment/ +│ │ ├── deploy.sh +│ │ └── rollback.sh +│ └── release/ +│ └── publish.py +└── docs/ + ├── WORKFLOWS.md + └── DEPLOYMENT.md +``` + +**`MokoStandards` Repository Structure:** +``` +MokoStandards/ +├── .github/ +│ └── workflows/ +│ ├── reusable-php-quality.yml (public) +│ ├── reusable-joomla-testing.yml (public) +│ ├── reusable-dolibarr-testing.yml (public) +│ └── reusable-security-scan.yml (public) ├── scripts/ │ ├── shared/ │ │ ├── extension_utils.py -│ │ └── deployment/ +│ │ ├── common.py +│ │ └── validators/ │ └── templates/ +│ ├── workflow-templates/ +│ └── action-templates/ └── docs/ - └── WORKFLOWS.md + ├── STANDARDS.md + ├── WORKFLOWS.md + └── CONTRIBUTING.md ``` ### Phase 3: Conversion **Main Repository Workflows (Simplified Callers):** -Example `php_quality.yml`: +**Example 1: Public Quality Workflow** (calls `MokoStandards`) + +`php_quality.yml`: ```yaml name: PHP Code Quality @@ -77,15 +131,40 @@ on: jobs: quality: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-php-quality.yml@main + uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 with: php-versions: '["8.0", "8.1", "8.2", "8.3"]' secrets: inherit ``` -**Centralized Reusable Workflow:** +**Example 2: Private Deployment Workflow** (calls `.github-private`) -Located in `.github-private/.github/workflows/reusable-php-quality.yml`: +`deploy.yml`: +```yaml +name: Deploy to Staging + +on: + workflow_dispatch: + inputs: + environment: + required: true + type: choice + options: [staging, production] + +jobs: + deploy: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@main + with: + environment: ${{ inputs.environment }} + platform: 'joomla' + secrets: inherit +``` + +**Centralized Reusable Workflow Examples:** + +**In `MokoStandards` (Public):** + +Located in `MokoStandards/.github/workflows/reusable-php-quality.yml`: ```yaml name: Reusable PHP Quality Workflow @@ -101,25 +180,59 @@ jobs: # Full implementation here ``` +**In `.github-private` (Private):** + +Located in `.github-private/.github/workflows/reusable-deploy.yml`: +```yaml +name: Reusable Deployment Workflow + +on: + workflow_call: + inputs: + environment: + required: true + type: string + secrets: + FTP_HOST: + required: true + FTP_USER: + required: true + +jobs: + # Deployment logic here +``` + ### Phase 4: Migration Steps -1. **Create .github-private repository** - - Initialize with README and LICENSE - - Set up branch protection rules - - Configure team access +1. **Create both centralized repositories** + - **`.github-private`**: Private repository (mokoconsulting-tech/.github-private) + - **`MokoStandards`**: Public repository (mokoconsulting-tech/MokoStandards) + - Initialize each with README and LICENSE + - Set up appropriate branch protection rules + - Configure access: private (team only) and public (community) -2. **Move workflows to .github-private** - - Convert to reusable workflows with `workflow_call` triggers +2. **Categorize and move workflows** + - **Sensitive workflows → `.github-private`**: + - deployment workflows, release pipelines + - Convert to reusable workflows with `workflow_call` triggers + - Add security controls and audit logging + + - **Public workflows → `MokoStandards`**: + - quality checks, testing workflows + - Add comprehensive documentation and examples + - Enable community contributions + - Test each workflow in isolation - Add proper input parameters and secrets handling 3. **Update main repository workflows** - - Replace with simplified caller workflows - - Update documentation - - Test integration + - Replace with simplified caller workflows pointing to appropriate repository + - Update documentation with new workflow references + - Test integration end-to-end 4. **Migrate shared scripts** - - Move organization-wide scripts to .github-private + - **Deployment/sensitive scripts → `.github-private/scripts/`** + - **Public utilities → `MokoStandards/scripts/shared/`** - Keep product-specific scripts in main repo - Update import paths and references @@ -154,37 +267,57 @@ jobs: ## Workflow Categorization -### Workflows to Centralize (Move to .github-private) +### Workflows to Centralize + +#### To `MokoStandards` (Public Repository) 1. **php_quality.yml** ✓ - Reason: Standardized quality checks across all PHP projects - Type: Reusable workflow - - Sensitivity: Low + - Sensitivity: Low (no secrets, publicly shareable) + - Target: `MokoStandards/.github/workflows/reusable-php-quality.yml` -2. **release_pipeline.yml** ✓ +2. **joomla_testing.yml** ✓ + - Reason: Shared across Joomla projects, community benefit + - Type: Reusable workflow + - Sensitivity: Low (testing patterns, no sensitive data) + - Target: `MokoStandards/.github/workflows/reusable-joomla-testing.yml` + +3. **ci.yml** (partially) + - Reason: Generic CI patterns can be shared + - Type: Reusable workflow template + - Sensitivity: Low (standard CI practices) + - Target: `MokoStandards/.github/workflows/reusable-ci-base.yml` + +#### To `.github-private` (Private Repository) + +1. **release_pipeline.yml** ✓ - Reason: Complex release logic, contains sensitive deployment steps - Type: Reusable workflow - - Sensitivity: High + - Sensitivity: High (deployment credentials, business logic) + - Target: `.github-private/.github/workflows/reusable-release-pipeline.yml` -3. **deploy_staging.yml** ✓ - - Reason: Contains deployment credentials and logic +2. **deploy_staging.yml** ✓ + - Reason: Contains deployment credentials and proprietary logic - Type: Reusable workflow - - Sensitivity: High + - Sensitivity: High (FTP credentials, server details) + - Target: `.github-private/.github/workflows/reusable-deploy-staging.yml` -4. **joomla_testing.yml** ✓ - - Reason: Shared across Joomla projects +3. **deploy_production.yml** ✓ + - Reason: Production deployment with strict security requirements - Type: Reusable workflow - - Sensitivity: Low + - Sensitivity: Critical (production access) + - Target: `.github-private/.github/workflows/reusable-deploy-production.yml` ### Workflows to Keep Local (Main Repository) -1. **ci.yml** - - Reason: Project-specific CI steps - - Can call centralized reusable workflows if needed +1. **ci.yml** (project-specific parts) + - Reason: Repository-specific CI steps + - Can call centralized workflows from both `MokoStandards` and `.github-private` 2. **repo_health.yml** - - Reason: Repository-specific health checks - - Keep local with option to extend from centralized base + - Reason: Repository-specific health checks and metrics + - Keep local with option to extend from `MokoStandards` base 3. **version_branch.yml** - Reason: Project-specific versioning strategy @@ -194,13 +327,32 @@ jobs: ### Scripts to Centralize +#### To `MokoStandards` (Public) + 1. **scripts/lib/extension_utils.py** ✓ - Shared across all extension projects - - Platform detection logic + - Platform detection logic (Joomla/Dolibarr) + - Target: `MokoStandards/scripts/shared/extension_utils.py` 2. **scripts/lib/common.py** ✓ - Universal utility functions - - No project-specific logic + - No project-specific or sensitive logic + - Target: `MokoStandards/scripts/shared/common.py` + +3. **scripts/release/detect_platform.py** ✓ + - Platform detection helper + - Publicly useful for other projects + - Target: `MokoStandards/scripts/shared/detect_platform.py` + +#### To `.github-private` (Private) + +1. **scripts/release/deployment/** ✓ + - Deployment scripts with credential handling + - Target: `.github-private/scripts/deployment/` + +2. **scripts/release/publish.py** (if sensitive) + - Release publishing with proprietary logic + - Target: `.github-private/scripts/release/publish.py` ### Scripts to Keep Local diff --git a/docs/MIGRATION_CHECKLIST.md b/docs/MIGRATION_CHECKLIST.md index b5f7170..8127529 100644 --- a/docs/MIGRATION_CHECKLIST.md +++ b/docs/MIGRATION_CHECKLIST.md @@ -1,6 +1,12 @@ -# .github-private Migration Checklist +# Centralized CI/CD Migration Checklist -This checklist guides the migration of CI/CD workflows from individual repositories to a centralized `.github-private` repository. +This checklist guides the migration of CI/CD workflows from individual repositories to centralized repositories using a dual-repository architecture. + +## Architecture Overview + +**Two Centralized Repositories:** +1. **`MokoStandards`** (Public) - Community workflows, quality checks, testing +2. **`.github-private`** (Private) - Sensitive workflows, deployments, proprietary logic ## Phase 1: Planning and Preparation @@ -8,13 +14,16 @@ This checklist guides the migration of CI/CD workflows from individual repositor - [x] Create CI_MIGRATION_PLAN.md - [x] Create REUSABLE_WORKFLOWS.md - [x] Create migration checklist +- [x] Define dual-repository architecture (MokoStandards + .github-private) - [ ] Review and approve migration plan with team - [ ] Identify workflow owners and stakeholders - [ ] Schedule migration windows ### Repository Inventory - [x] List all workflows in current repository -- [x] Identify workflows to centralize +- [x] Identify workflows to centralize (categorized by sensitivity) +- [x] Identify public workflows for MokoStandards +- [x] Identify sensitive workflows for .github-private - [x] Identify workflows to keep local - [x] Document workflow dependencies - [x] List all secrets used by workflows @@ -22,16 +31,26 @@ This checklist guides the migration of CI/CD workflows from individual repositor ### Risk Assessment - [ ] Identify critical workflows that cannot have downtime -- [ ] Create rollback procedures +- [ ] Create rollback procedures for both repositories - [ ] Set up monitoring for workflow failures - [ ] Communicate migration plan to team +- [ ] Plan staged rollout strategy -## Phase 2: .github-private Repository Setup +## Phase 2: Centralized Repository Setup -### Repository Creation +### MokoStandards Repository Creation (Public) +- [ ] Create `MokoStandards` repository in organization +- [ ] Set repository to Public +- [ ] Initialize with README explaining public standards +- [ ] Add LICENSE file (appropriate for public repository) +- [ ] Create initial branch structure (main, develop) +- [ ] Enable GitHub Pages for documentation +- [ ] Set up community contribution guidelines + +### .github-private Repository Creation (Private) - [ ] Create `.github-private` repository in organization - [ ] Set repository to Private -- [ ] Initialize with README +- [ ] Initialize with README explaining private workflows - [ ] Add LICENSE file - [ ] Create initial branch structure (main, develop) diff --git a/docs/REUSABLE_WORKFLOWS.md b/docs/REUSABLE_WORKFLOWS.md index 6e4bd68..1aaf0f9 100644 --- a/docs/REUSABLE_WORKFLOWS.md +++ b/docs/REUSABLE_WORKFLOWS.md @@ -1,42 +1,120 @@ -# Reusable Workflow Templates for .github-private +# Reusable Workflow Templates for Centralized Repositories -This directory contains example templates for reusable workflows that will be moved to the `.github-private` repository. +This document contains example templates for reusable workflows that will be distributed across two centralized repositories based on sensitivity. -## Structure +## Dual Repository Architecture +### `MokoStandards` (Public Repository) +- **Purpose**: Public, community-accessible workflows and standards +- **Content**: Quality checks, testing workflows, public CI/CD templates +- **Visibility**: Public (open source) +- **Target Audience**: Internal teams and external community + +### `.github-private` (Private Repository) +- **Purpose**: Sensitive, proprietary workflows and deployment logic +- **Content**: Deployment workflows, release pipelines, credential handling +- **Visibility**: Private (organization only) +- **Target Audience**: Internal teams only + +## Repository Structures + +**`MokoStandards` (Public):** +``` +MokoStandards/ +├── .github/ +│ └── workflows/ +│ ├── reusable-php-quality.yml +│ ├── reusable-joomla-testing.yml +│ ├── reusable-dolibarr-testing.yml +│ └── reusable-security-scan.yml +├── scripts/ +│ └── shared/ +│ ├── extension_utils.py +│ └── common.py +└── docs/ + ├── STANDARDS.md + └── USAGE.md +``` + +**`.github-private` (Private):** ``` .github-private/ ├── .github/ │ └── workflows/ -│ ├── reusable-php-quality.yml │ ├── reusable-release-pipeline.yml -│ ├── reusable-joomla-testing.yml -│ └── reusable-deploy-staging.yml -└── docs/ +│ ├── reusable-deploy-staging.yml +│ └── reusable-deploy-production.yml +├── scripts/ +│ └── deployment/ +└. docs/ └── USAGE.md ``` ## Usage in Main Repositories -### Basic Pattern +### Basic Patterns + +**Pattern 1: Calling Public Workflow from `MokoStandards`** ```yaml -name: Workflow Name +name: Quality Checks on: push: branches: [ main ] jobs: - job-name: - uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-workflow-name.yml@main + quality: + uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 with: - # Input parameters - parameter-name: value + php-versions: '["8.0", "8.1", "8.2", "8.3"]' secrets: inherit ``` -### Example: PHP Quality Check +**Pattern 2: Calling Private Workflow from `.github-private`** + +```yaml +name: Deploy + +on: + workflow_dispatch: + +jobs: + deploy: + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@main + with: + environment: staging + secrets: inherit +``` + +**Pattern 3: Combining Both (Public Quality + Private Deployment)** + +```yaml +name: CI/CD Pipeline + +on: + push: + branches: [ main ] + +jobs: + quality: + uses: mokoconsulting-tech/MokoStandards/.github/workflows/reusable-php-quality.yml@v1 + with: + php-versions: '["8.0", "8.1", "8.2", "8.3"]' + secrets: inherit + + deploy: + needs: quality + if: success() + uses: mokoconsulting-tech/.github-private/.github/workflows/reusable-deploy.yml@main + with: + environment: staging + secrets: inherit +``` + +## Complete Workflow Examples by Repository + +### Example 1: PHP Quality Check (MokoStandards - Public) **In main repository** (`.github/workflows/php_quality.yml`): ```yaml