MokoConsulting/MokoGIT
1
0
Fork 0
Code Issues 148 Pull Requests 1 Releases 96 Wiki Activity

Actions: a concurrency: block prevents run CREATION — push/dispatch return 204 but no run is created #799

New Issue
Open
opened 2026-07-17 04:17:59 +00:00 by jmiller · 3 comments
jmiller commented 2026-07-17 04:17:59 +00:00
Owner
Copy Link
Copy Source

Summary

MokoGIT's Actions engine fails to create a run for any workflow that declares a top-level concurrency: block. On push (and on workflow_dispatch), the request returns 204/success but no run is ever created — the workflow is silently skipped. Removing the concurrency: block makes the exact same workflow run normally.

This is distinct from #798 (which was about the .mokogit/workflows/ path not being indexed at all). Here the path is indexed and other workflows in the same repo run fine on the same push; only the one(s) declaring concurrency: are dropped.

Impact

  • The tiered deploy workflows (deploy-dev.yml / deploy-rc.yml / deploy-prod.yml) all ship with a concurrency: block in Template-Go (the canonical sync source), so every Go repo's deploys silently stopped firing.
  • Symptom is easy to misdiagnose as "workflow not registered" / "encoding" / "stale registration" because there is no error — just a missing run.
  • The block re-syncs from Template-Go on every workflow sync, so a downstream repo that removes it locally gets it re-added and breaks again.

Evidence (repo: MokoConsulting/mokoai)

Holding everything else constant and toggling only the concurrency: block on deploy-dev.yml:

Commit concurrency: block deploy-dev run created?
85527db, e0d22a1 present No — 0 runs
e0d22a1 present, fresh group name (deploy-dev-mokoai-v2) No — 0 runs (fresh group did not help)
d63d854, 0cc0735 removed Yes — run created, executed

Same pattern reproduced on deploy-prod.yml: the 01.01.00 merge to main with the block present created 0 deploy-prod runs; after removing the block (PR mokoai#87) the next main push created and completed a deploy-prod run.

Reproduction

  1. Take any repo whose Actions are working from .mokogit/workflows/.
  2. Add a top-level block to one workflow: 
    concurrency:
  group: my-group
  cancel-in-progress: true
  3. Push a commit that triggers that workflow.
  4. Observe: sibling workflows on the same push create runs; the one with concurrency: creates none. POST .../actions/workflows/<file>/dispatches returns 204 but also creates nothing.
  5. Remove the block, push again → the workflow runs normally.

Diagnostic note

The REST tasks endpoint (GET /api/v1/repos/{o}/{r}/actions/tasks) only lists started/completed runs — queued (0s) runs do not appear there, which makes this bug look like "nothing was created" even in cases where a run is merely queued. Confirm via the web Actions page (which shows 0s/queued entries) before concluding no run exists.

Likely area

Gitea Actions run-creation path — evaluation of the concurrency expression / group when scheduling a new run (the fork may be erroring or short-circuiting run creation when a concurrency group is declared, rather than queuing/cancelling as upstream does). Worth diffing the fork's services/actions notifier / jobparser concurrency handling against upstream Gitea.

Workaround applied (not a fix)

Removed the concurrency: block from deploy-dev/rc/prod.yml in Template-Go (PR MokoConsulting/Template-Go#18, merged) and in the affected downstream repo (mokoai). Each keeps a NOTE comment. Once this MokoGIT bug is fixed, the concurrency: blocks should be restored template-wide (serialized deploys are desirable).

Related: #798 (path indexing — separate root cause, same "no runs created" symptom).

## Summary MokoGIT's Actions engine **fails to create a run for any workflow that declares a top-level `concurrency:` block**. On push (and on `workflow_dispatch`), the request returns 204/success but **no run is ever created** — the workflow is silently skipped. Removing the `concurrency:` block makes the exact same workflow run normally. This is distinct from #798 (which was about the `.mokogit/workflows/` path not being *indexed* at all). Here the path is indexed and other workflows in the same repo run fine on the same push; only the one(s) declaring `concurrency:` are dropped. ## Impact - The tiered deploy workflows (`deploy-dev.yml` / `deploy-rc.yml` / `deploy-prod.yml`) all ship with a `concurrency:` block in `Template-Go` (the canonical sync source), so **every Go repo's deploys silently stopped firing**. - Symptom is easy to misdiagnose as "workflow not registered" / "encoding" / "stale registration" because there is no error — just a missing run. - The block re-syncs from `Template-Go` on every workflow sync, so a downstream repo that removes it locally gets it re-added and breaks again. ## Evidence (repo: MokoConsulting/mokoai) Holding everything else constant and toggling only the `concurrency:` block on `deploy-dev.yml`: | Commit | `concurrency:` block | deploy-dev run created? | |---|---|---| | 85527db, e0d22a1 | present | **No — 0 runs** | | e0d22a1 | present, **fresh group name** (`deploy-dev-mokoai-v2`) | **No — 0 runs** (fresh group did not help) | | d63d854, 0cc0735 | removed | **Yes — run created, executed** | Same pattern reproduced on `deploy-prod.yml`: the `01.01.00` merge to `main` with the block present created **0** deploy-prod runs; after removing the block (PR mokoai#87) the next `main` push created and completed a deploy-prod run. ## Reproduction 1. Take any repo whose Actions are working from `.mokogit/workflows/`. 2. Add a top-level block to one workflow: ```yaml concurrency: group: my-group cancel-in-progress: true ``` 3. Push a commit that triggers that workflow. 4. Observe: sibling workflows on the same push create runs; the one with `concurrency:` creates none. `POST .../actions/workflows/<file>/dispatches` returns 204 but also creates nothing. 5. Remove the block, push again → the workflow runs normally. ## Diagnostic note The REST tasks endpoint (`GET /api/v1/repos/{o}/{r}/actions/tasks`) only lists **started/completed** runs — **queued (0s) runs do not appear there**, which makes this bug look like "nothing was created" even in cases where a run is merely queued. Confirm via the web Actions page (which shows 0s/queued entries) before concluding no run exists. ## Likely area Gitea Actions run-creation path — evaluation of the `concurrency` expression / group when scheduling a new run (the fork may be erroring or short-circuiting run creation when a concurrency group is declared, rather than queuing/cancelling as upstream does). Worth diffing the fork's `services/actions` notifier / `jobparser` concurrency handling against upstream Gitea. ## Workaround applied (not a fix) Removed the `concurrency:` block from `deploy-dev/rc/prod.yml` in `Template-Go` (PR MokoConsulting/Template-Go#18, merged) and in the affected downstream repo (mokoai). Each keeps a NOTE comment. **Once this MokoGIT bug is fixed, the `concurrency:` blocks should be restored template-wide** (serialized deploys are desirable). Related: #798 (path indexing — separate root cause, same "no runs created" symptom).
jmiller commented 2026-07-17 04:32:35 +00:00
Author
Owner
Copy Link
Copy Source

Branch created: feature/799-actions-a-concurrency-block-prevents-run

git fetch origin
git checkout feature/799-actions-a-concurrency-block-prevents-run
Branch created: [`feature/799-actions-a-concurrency-block-prevents-run`](https://git.mokoconsulting.tech/MokoConsulting/MokoGIT/src/branch/feature/799-actions-a-concurrency-block-prevents-run) ```bash git fetch origin git checkout feature/799-actions-a-concurrency-block-prevents-run ```
jmiller commented 2026-07-17 04:41:22 +00:00
Author
Owner
Copy Link
Copy Source

Root-cause analysis (source-level)

Traced this in the MokoGIT source. The bug is an incomplete migration of Actions concurrency from run-level to attempt-level — concurrency workflows enter a half-wired code path, so the run is created but never dispatched (it shows in the Actions UI at 0s and never executes; the REST .../actions/tasks endpoint only lists started/completed runs, which is why it looks like "no run created"). Workflows without a concurrency: block skip this path entirely and run normally.

The inconsistency (file : line)

  1. models/migrations/v1_27/v331.go — AddActionRunAttemptModel: adds the action_run_attempt table + action_run.latest_attempt_id, and intentionally DROPS action_run.concurrency_group / action_run.concurrency_cancel (see its own comment: "intentionally removes the legacy run-level concurrency columns"). Concurrency is meant to live on action_run_attempt now.
  2. models/actions/run.go (ActionRun struct, ~L52-54) still declares RawConcurrency / ConcurrencyGroup / ConcurrencyCancel. Because the model keeps those fields, xorm's startup Sync re-adds the columns the migration just dropped — a schema flip-flop, and a sign the model side wasn't updated with the migration.
  3. services/actions/run.go — InsertRun: builds an attempt := &ActionRunAttempt{...} (~L95), fills it via EvaluateRunConcurrencyFillModel, calls PrepareToStartRunWithConcurrency(attempt) — but never db.Inserts the attempt. (The only db.Insert(..., attempt) in the tree is in rerun.go.) So action_run_attempt gets no row and run.LatestAttemptID stays 0.
  4. services/actions/clear_tasks.go — shouldBlockRunByConcurrency / CancelPreviousJobsByRunConcurrency operate on attempt-level data (GetConcurrentRunAttemptsAndJobs → FindConcurrentRunAttempts over action_run_attempt), which is empty because of #3.
  5. services/actions/job_emitter.go — checkRunConcurrency (~L172) still keys off run-level run.ConcurrencyGroup.

So the implementation is split across both models (run-level and attempt-level) with the attempt row never persisted — mutually inconsistent, and only exercised when a concurrency: block is present.

Fix direction (pick one, applied consistently)

  • Complete the attempt-level migration: InsertRun must db.Insert the ActionRunAttempt and set run.LatestAttemptID; move the remaining run-level reads (job_emitter.checkRunConcurrency) onto the attempt; and drop the now-dead ConcurrencyGroup/ConcurrencyCancel fields from the ActionRun model so the schema stops flip-flopping. OR
  • Revert to run-level concurrency: don't drop the action_run columns in v331, and remove the attempt-based shouldBlock/Cancel paths.

Confirming the exact failing op

Code analysis points to the created-but-not-dispatched path above. Pinning whether it fails at insert-time (SQL) vs emit-time (no dispatch) needs one of: the mokogit container error log around a concurrency-triggered push, or a build+test repro — both require host/build access I don't have in this context.

Interim mitigation (already shipped)

concurrency: blocks removed from deploy-dev/rc/prod in Template-Go (MokoConsulting/Template-Go#18, merged) and downstream (mokoai), so deploys fire again. These should be restored once this is fixed.

## Root-cause analysis (source-level) Traced this in the MokoGIT source. The bug is an **incomplete migration of Actions concurrency from run-level to attempt-level** — concurrency workflows enter a half-wired code path, so the run is *created but never dispatched* (it shows in the Actions UI at `0s` and never executes; the REST `.../actions/tasks` endpoint only lists started/completed runs, which is why it looks like "no run created"). Workflows without a `concurrency:` block skip this path entirely and run normally. ### The inconsistency (file : line) 1. **`models/migrations/v1_27/v331.go` — `AddActionRunAttemptModel`**: adds the `action_run_attempt` table + `action_run.latest_attempt_id`, and **intentionally DROPS** `action_run.concurrency_group` / `action_run.concurrency_cancel` (see its own comment: "intentionally removes the legacy run-level concurrency columns"). Concurrency is meant to live on `action_run_attempt` now. 2. **`models/actions/run.go` (ActionRun struct, ~L52-54)** still declares `RawConcurrency` / `ConcurrencyGroup` / `ConcurrencyCancel`. Because the model keeps those fields, xorm's startup `Sync` **re-adds** the columns the migration just dropped — a schema flip-flop, and a sign the model side wasn't updated with the migration. 3. **`services/actions/run.go` — `InsertRun`**: builds an `attempt := &ActionRunAttempt{...}` (~L95), fills it via `EvaluateRunConcurrencyFillModel`, calls `PrepareToStartRunWithConcurrency(attempt)` — but **never `db.Insert`s the attempt**. (The only `db.Insert(..., attempt)` in the tree is in `rerun.go`.) So `action_run_attempt` gets no row and `run.LatestAttemptID` stays `0`. 4. **`services/actions/clear_tasks.go` — `shouldBlockRunByConcurrency` / `CancelPreviousJobsByRunConcurrency`** operate on **attempt-level** data (`GetConcurrentRunAttemptsAndJobs` → `FindConcurrentRunAttempts` over `action_run_attempt`), which is empty because of #3. 5. **`services/actions/job_emitter.go` — `checkRunConcurrency` (~L172)** still keys off **run-level** `run.ConcurrencyGroup`. So the implementation is split across both models (run-level *and* attempt-level) with the attempt row never persisted — mutually inconsistent, and only exercised when a `concurrency:` block is present. ### Fix direction (pick one, applied consistently) - **Complete the attempt-level migration**: `InsertRun` must `db.Insert` the `ActionRunAttempt` and set `run.LatestAttemptID`; move the remaining run-level reads (`job_emitter.checkRunConcurrency`) onto the attempt; and drop the now-dead `ConcurrencyGroup`/`ConcurrencyCancel` fields from the `ActionRun` model so the schema stops flip-flopping. **OR** - **Revert to run-level concurrency**: don't drop the `action_run` columns in v331, and remove the attempt-based `shouldBlock/Cancel` paths. ### Confirming the exact failing op Code analysis points to the created-but-not-dispatched path above. Pinning whether it fails at insert-time (SQL) vs emit-time (no dispatch) needs one of: the `mokogit` container error log around a concurrency-triggered push, or a build+test repro — both require host/build access I don't have in this context. ### Interim mitigation (already shipped) `concurrency:` blocks removed from `deploy-dev/rc/prod` in Template-Go (MokoConsulting/Template-Go#18, merged) and downstream (mokoai), so deploys fire again. These should be restored once this is fixed.
jmiller commented 2026-07-17 06:48:02 +00:00
Author
Owner
Copy Link
Copy Source

Update — appears RESOLVED in stable-439, and my earlier root-cause was wrong

Re-tested against the currently deployed prod mokogit (stable-439-g81d065b9bb, restarted ~06:26 UTC 2026-07-17) using a throwaway repo, and the bug no longer reproduces:

  • Pushed a workflow with a concurrency: block on both .gitea/workflows/ and .mokogit/workflows/ paths.
  • A run is created in both cases (the original symptom was zero runs created), and the .gitea-path concurrency runs executed to status=success (verified in action_run for the test repo). The .mokogit-path concurrency run was likewise created and queued.

So run-creation with a concurrency: block works on stable-439. Whatever shipped in that build fixed it.

Correction to my earlier comment: the "incomplete attempt-level refactor / attempt never inserted" hypothesis was not the cause. action_run_attempt stays empty (ATTEMPTS=0) even for runs that complete successfully, so an empty attempt table is normal, not the bug. Please disregard that root-cause analysis.

Suggested follow-ups

  • Confirm what stable-439 changed and close this if satisfied.
  • The interim mitigation (concurrency blocks removed from deploy-dev/rc/prod in Template-Go#18 and downstream) can now be reverted to restore serialized deploys, since concurrency works again.
## Update — appears RESOLVED in `stable-439`, and my earlier root-cause was wrong Re-tested against the **currently deployed prod `mokogit` (`stable-439-g81d065b9bb`, restarted ~06:26 UTC 2026-07-17)** using a throwaway repo, and the bug no longer reproduces: - Pushed a workflow **with** a `concurrency:` block on both `.gitea/workflows/` and `.mokogit/workflows/` paths. - **A run is created in both cases** (the original symptom was zero runs created), and the `.gitea`-path concurrency runs **executed to `status=success`** (verified in `action_run` for the test repo). The `.mokogit`-path concurrency run was likewise created and queued. So run-creation with a `concurrency:` block works on `stable-439`. Whatever shipped in that build fixed it. **Correction to my earlier comment:** the "incomplete attempt-level refactor / attempt never inserted" hypothesis was **not** the cause. `action_run_attempt` stays empty (`ATTEMPTS=0`) even for runs that complete successfully, so an empty attempt table is normal, not the bug. Please disregard that root-cause analysis. ### Suggested follow-ups - Confirm what `stable-439` changed and **close this** if satisfied. - The interim mitigation (concurrency blocks removed from `deploy-dev/rc/prod` in Template-Go#18 and downstream) can now be **reverted to restore serialized deploys**, since concurrency works again.
Sign in to join this conversation.
Labels
Clear labels
breaking-change
Breaking API or behavior change
 ci-cd
CI/CD pipeline changes
 config
Configuration changes
 dependencies
Dependency updates
 deploy-failure
Deployment failed
 docker
Docker/container changes
 documentation
Documentation changes
 feature: wiki
Wiki system enhancements
 good first issue
Good for newcomers
 health-check
Repo health check result
 help wanted
Extra attention needed
 mokostandards
Related to MokoStandards framework
 push-failure
Git push operation failed
 security
Security vulnerability or hardening
 size/l
200-500 lines changed
 size/m
50-200 lines changed
 size/s
10-50 lines changed
 size/xl
500+ lines changed
 size/xs
< 10 lines changed
 standards-drift
Deviates from MokoStandards
 standards-update
MokoStandards compliance update
 sync-failure
Sync or mirror failed
 tech-debt
Technical debt and TODO/FIXME items
 upstream
upstream
Inherited from upstream Gitea
 work-in-progress
Draft or incomplete work
bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 wontfix
This won't be fixed
No labels
Priority Medium
Type Feature
Milestone
No items
No Milestone
Assignees
Clear assignees
jmiller (Jonathan Miller) moko-deploy (Moko Deploy Bot)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: MokoConsulting/MokoGIT#799
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?