Files
Jonathan Miller dd6fc4b69c
Branch Policy Check / Verify merge target (pull_request) Successful in 1s
PR RC Release / Build RC Release (pull_request) Successful in 2s
Universal: Pre-Release / Build Pre-Release (${{ inputs.stability || 'development' }}) (pull_request) Failing after 21s
fix(actions): retry workflow insertion on database deadlock
When multiple workflows are triggered by a single event (e.g. a
pull_request with several matching workflow files), each InsertRun
transaction acquires an X-lock on the repository row via
UpdateRepoRunsNumbers and an index lock on action_run. Two concurrent
transactions can deadlock when each holds one lock and waits for the
other. InnoDB kills the lighter transaction, but handleWorkflows only
logged the error and silently dropped the workflow run — making it
appear as though pull_request events were never fired.

This was the root cause of API-created PRs appearing to not trigger
Actions workflows: the notification pipeline was correct, but the DB
insert was lost to an unretried deadlock.

The fix wraps PrepareRunAndInsert in a retry loop (up to 3 attempts
with exponential backoff) that detects deadlock errors across MySQL,
PostgreSQL, and SQLite. On deadlock, the rolled-back run fields are
reset before the next attempt.

Also adds db.IsErrDeadlock() for cross-engine deadlock detection and
unit tests for the same.

Closes #220

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-26 15:16:30 -05:00

100 lines
2.3 KiB
Go

// Copyright 2021 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package db
import (
"fmt"
"strings"
"git.mokoconsulting.tech/MokoConsulting/MokoGitea/modules/util"
)
// ErrCancelled represents an error due to context cancellation
type ErrCancelled struct {
Message string
}
// IsErrCancelled checks if an error is a ErrCancelled.
func IsErrCancelled(err error) bool {
_, ok := err.(ErrCancelled)
return ok
}
func (err ErrCancelled) Error() string {
return "Cancelled: " + err.Message
}
// ErrCancelledf returns an ErrCancelled for the provided format and args
func ErrCancelledf(format string, args ...any) error {
return ErrCancelled{
fmt.Sprintf(format, args...),
}
}
// ErrSSHDisabled represents an "SSH disabled" error.
type ErrSSHDisabled struct{}
// IsErrSSHDisabled checks if an error is a ErrSSHDisabled.
func IsErrSSHDisabled(err error) bool {
_, ok := err.(ErrSSHDisabled)
return ok
}
func (err ErrSSHDisabled) Error() string {
return "SSH is disabled"
}
// ErrNotExist represents a non-exist error.
type ErrNotExist struct {
Resource string
ID int64
}
// IsErrNotExist checks if an error is an ErrNotExist
func IsErrNotExist(err error) bool {
_, ok := err.(ErrNotExist)
return ok
}
func (err ErrNotExist) Error() string {
name := "record"
if err.Resource != "" {
name = err.Resource
}
if err.ID != 0 {
return fmt.Sprintf("%s does not exist [id: %d]", name, err.ID)
}
return name + " does not exist"
}
// Unwrap unwraps this as a ErrNotExist err
func (err ErrNotExist) Unwrap() error {
return util.ErrNotExist
}
// IsErrDeadlock checks whether err is a database deadlock.
// MySQL returns error 1213 (ER_LOCK_DEADLOCK / SQLSTATE 40001).
// PostgreSQL returns SQLSTATE 40P01 with "deadlock detected".
// SQLite returns SQLITE_BUSY (error 5) with "database is locked".
func IsErrDeadlock(err error) bool {
if err == nil {
return false
}
msg := err.Error()
// MySQL / MariaDB: "Error 1213 (40001): Deadlock found when trying to get lock"
if strings.Contains(msg, "Error 1213") || strings.Contains(msg, "40001") {
return true
}
// PostgreSQL: "deadlock detected"
if strings.Contains(msg, "deadlock detected") {
return true
}
// SQLite: "database is locked"
if strings.Contains(msg, "database is locked") {
return true
}
return false
}