Compare commits

...

5 Commits

Author SHA1 Message Date
gitea-actions[bot] 6ad2f6b6af chore(version): pre-release bump to 01.29.00-rc [skip ci]
Publish to Composer / Publish Package (release) Failing after 6s
2026-06-22 00:10:26 +00:00
gitea-actions[bot] b49f0e9064 chore(release): build 01.28.00-rc [skip ci]
Publish to Composer / Publish Package (release) Failing after 34s
2026-06-22 00:09:59 +00:00
Jonathan Miller 8b6e260b28 fix: graceful degradation when remote upload fails (#66)
Joomla: Extension CI / Tests (PHP 8.2) (pull_request) Blocked by required conditions
Joomla: Extension CI / Tests (PHP 8.3) (pull_request) Blocked by required conditions
Joomla: Extension CI / PHPStan Analysis (pull_request) Blocked by required conditions
Joomla: Extension CI / Build RC Pre-Release (pull_request) Blocked by required conditions
Universal: PR Check / Build RC Package (pull_request) Blocked by required conditions
Universal: PR Check / Report Issues (pull_request) Blocked by required conditions
Generic: Repo Health / Scripts governance (pull_request) Blocked by required conditions
Generic: Repo Health / Repository health (pull_request) Blocked by required conditions
Generic: Repo Health / Report Issues (pull_request) Blocked by required conditions
Universal: PR Check / Branch Policy (pull_request) Failing after 2s
Joomla: Extension CI / Release Readiness Check (pull_request) Failing after 5s
Generic: Repo Health / Access control (pull_request) Successful in 2s
Generic: Repo Health / Site Health (pull_request) Has been skipped
Universal: PR Check / Validate PR (pull_request) Failing after 9s
Universal: PR Check / Secret Scan (pull_request) Successful in 11s
Branch Cleanup / Delete merged branch (pull_request) Has been skipped
RC Revert / Rename rc/ back to dev/ (pull_request) Has been skipped
Universal: Workflow Sync Trigger / Sync workflows to live repos (pull_request) Has been skipped
Universal: Auto Version Bump / Version Bump (push) Successful in 11s
Joomla: Extension CI / Lint & Validate (pull_request) Failing after 44s
Joomla: Metadata Validation / Validate Joomla Metadata (pull_request) Successful in 47s
Universal: Pre-Release / Build Pre-Release (${{ inputs.stability || github.ref_name }}) (push) Successful in 17s
Universal: Build & Release / Promote to RC (pull_request) Has been skipped
Universal: Build & Release / Build & Release Pipeline (pull_request) Has been skipped
Remote upload failure (S3/FTP/GDrive) no longer marks the entire
backup as failed. The local archive is preserved with status
'complete' and the upload failure is logged as a warning. Applies
to both BackupEngine and SteppedBackupEngine.

Closes #66
2026-06-21 19:09:20 -05:00
Jonathan Miller eb7f48d3a2 feat: extend snapshots to include custom fields and tags (#57)
When articles are included in a snapshot, now also captures:
- #__tags (tag definitions)
- #__fields (custom field definitions for com_content.article)
- #__fields_values (custom field values)
- #__fields_categories (field-to-category mappings)

Restore correctly scopes deletes to avoid touching non-content fields.

Closes #57
2026-06-21 19:08:53 -05:00
Jonathan Miller 974b971340 feat: snapshot retention and automatic cleanup (#63)
Add retention settings for content snapshots (max count, max age days)
in component options. System plugin runs cleanupOldSnapshots() alongside
existing backup cleanup, deleting JSON files and DB records.

Closes #63
2026-06-21 19:08:27 -05:00
19 changed files with 307 additions and 58 deletions
+1 -1
View File
@@ -5,7 +5,7 @@
# FILE INFORMATION
# DEFGROUP: Gitea.Workflow
# INGROUP: mokocli.Automation
# VERSION: 01.00.00
# VERSION: 01.29.00
# BRIEF: Auto-create feature branch when an issue is opened
name: "Universal: Issue Branch"
+9 -4
View File
@@ -1,6 +1,15 @@
# Changelog
## [Unreleased]
## [01.28.00] --- 2026-06-22
### Changed
- Remote upload failure no longer marks the entire backup as failed — local archive is preserved with status 'complete' (#66)
### Added
- Snapshots now capture tags, custom fields, field values, and field-category mappings when articles are included (#57)
- Snapshot retention settings: max count and max age with automatic cleanup (#63)
## [01.27.03] --- 2026-06-21
## [01.27.03] --- 2026-06-21
@@ -8,7 +17,3 @@
## [01.27.00] --- 2026-06-21
## [01.27.00] --- 2026-06-21
## [01.27.00] --- 2026-06-21
## [01.27.00] --- 2026-06-21
+1 -1
View File
@@ -1,6 +1,6 @@
# MokoSuiteBackup
<!-- VERSION: 01.27.03 -->
<!-- VERSION: 01.29.00 -->
Full-site backup and restore for Joomla — database, files, and configuration.
@@ -118,6 +118,27 @@
/>
</fieldset>
<fieldset name="snapshot_cleanup" label="COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_RETENTION">
<field
name="snapshot_retention_count"
type="number"
label="COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_COUNT"
description="COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_COUNT_DESC"
default="20"
min="0"
max="100"
/>
<field
name="snapshot_retention_days"
type="number"
label="COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_AGE"
description="COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_AGE_DESC"
default="30"
min="0"
max="365"
/>
</fieldset>
<fieldset name="notifications" label="COM_MOKOJOOMBACKUP_CONFIG_NOTIFICATIONS">
<field
name="notify_email"
@@ -269,6 +269,13 @@ COM_MOKOJOOMBACKUP_CONFIG_NOTIFY_SUCCESS_DESC="Send email when any backup comple
COM_MOKOJOOMBACKUP_CONFIG_NOTIFY_FAILURE="Notify on Failure"
COM_MOKOJOOMBACKUP_CONFIG_NOTIFY_FAILURE_DESC="Send email when any backup fails (unless overridden by profile)."
; Snapshot Retention
COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_RETENTION="Snapshot Retention"
COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_COUNT="Max Snapshot Count"
COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_COUNT_DESC="Maximum number of content snapshots to keep. Oldest are removed first. Set to 0 for unlimited."
COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_AGE="Max Snapshot Age (days)"
COM_MOKOJOOMBACKUP_CONFIG_SNAPSHOT_MAX_AGE_DESC="Delete snapshots older than this many days. Set to 0 for unlimited."
; Web Cron
COM_MOKOJOOMBACKUP_CONFIG_WEBCRON="Web Cron"
COM_MOKOJOOMBACKUP_CONFIG_WEBCRON_ENABLED="Enable Web Cron"
@@ -7,7 +7,7 @@
-->
<extension type="component" method="upgrade">
<name>MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-02</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -255,26 +255,36 @@ class BackupEngine
}
$remoteFilename = '';
$uploadFailed = false;
// Step 3: Remote upload (if configured)
// Wrapped in its own try-catch so a remote failure does not mark
// the entire backup as failed — the local archive is preserved.
$remoteStorage = $profile->remote_storage ?? 'none';
if ($remoteStorage !== 'none') {
$this->log('Starting remote upload (' . $remoteStorage . ')...');
$uploader = $this->createUploader($remoteStorage, $profile);
$uploadResult = $uploader->upload($archivePath, $archiveName);
try {
$this->log('Starting remote upload (' . $remoteStorage . ')...');
$uploader = $this->createUploader($remoteStorage, $profile);
$uploadResult = $uploader->upload($archivePath, $archiveName);
if ($uploadResult['success']) {
$remoteFilename = $uploadResult['remote_path'] ?? $archiveName;
$this->log('Remote upload complete: ' . $uploadResult['message']);
if ($uploadResult['success']) {
$remoteFilename = $uploadResult['remote_path'] ?? $archiveName;
$this->log('Remote upload complete: ' . $uploadResult['message']);
// Delete local copy if configured
if (empty($profile->remote_keep_local) && is_file($archivePath)) {
@unlink($archivePath);
$this->log('Local copy removed (remote_keep_local = off)');
// Delete local copy if configured
if (empty($profile->remote_keep_local) && is_file($archivePath)) {
@unlink($archivePath);
$this->log('Local copy removed (remote_keep_local = off)');
}
} else {
$uploadFailed = true;
$this->log('WARNING: Remote upload failed: ' . $uploadResult['message']);
$this->log('Local backup is preserved.');
}
} else {
$this->log('WARNING: Remote upload failed: ' . $uploadResult['message']);
} catch (\Throwable $e) {
$uploadFailed = true;
$this->log('WARNING: Remote upload threw an exception: ' . $e->getMessage());
$this->log('Local backup is preserved.');
}
}
@@ -309,9 +319,14 @@ class BackupEngine
$db->updateObject('#__mokosuitebackup_records', $update, 'id');
// Send success notification
// Send success notification (backup completed, even if upload failed)
NotificationSender::send($profile, $update, true, implode("\n", $this->log));
// If remote upload failed, also send a failure notification for the upload
if ($uploadFailed) {
NotificationSender::send($profile, $update, false, "Remote upload failed — see backup log for details.\n\n" . implode("\n", $this->log));
}
// Dispatch event for actionlog and other listeners
$this->dispatchAfterRun(true, $recordId, $description, $profileId, $origin);
@@ -41,6 +41,10 @@ class SnapshotEngine
private const ARTICLE_RELATED = [
'#__workflow_associations',
'#__contentitem_tag_map',
'#__tags',
'#__fields',
'#__fields_values',
'#__fields_categories',
];
/**
@@ -107,6 +111,32 @@ class SnapshotEngine
$rows = $this->dumpTagMap($db, $prefix);
$data['tables']['#__contentitem_tag_map'] = $rows;
$this->log(' #__contentitem_tag_map: ' . count($rows) . ' rows');
// Tags — dump all (shared, small table)
$rows = $this->dumpTable($db, str_replace('#__', $prefix, '#__tags'), '#__tags', 'articles');
$data['tables']['#__tags'] = $rows;
$this->log(' #__tags: ' . count($rows) . ' rows');
// Custom fields — only com_content.article context
$rows = $this->dumpFilteredTable(
$db,
str_replace('#__', $prefix, '#__fields'),
'#__fields',
'context',
'com_content.article'
);
$data['tables']['#__fields'] = $rows;
$this->log(' #__fields: ' . count($rows) . ' rows');
// Field values — dump all (small, article-scoped)
$rows = $this->dumpTable($db, str_replace('#__', $prefix, '#__fields_values'), '#__fields_values', 'articles');
$data['tables']['#__fields_values'] = $rows;
$this->log(' #__fields_values: ' . count($rows) . ' rows');
// Field-category mappings — only for com_content.article fields
$rows = $this->dumpFieldCategories($db, $prefix);
$data['tables']['#__fields_categories'] = $rows;
$this->log(' #__fields_categories: ' . count($rows) . ' rows');
}
// Count items
@@ -231,6 +261,30 @@ class SnapshotEngine
return $db->loadAssocList() ?: [];
}
/**
* Dump field-category mappings for com_content.article fields.
*
* Uses a subquery: field_id IN (SELECT id FROM #__fields WHERE context = 'com_content.article')
*/
private function dumpFieldCategories(object $db, string $prefix): array
{
$fcTable = $prefix . 'fields_categories';
$fTable = $prefix . 'fields';
$subQuery = $db->getQuery(true)
->select($db->quoteName('id'))
->from($db->quoteName($fTable))
->where($db->quoteName('context') . ' = ' . $db->quote('com_content.article'));
$query = $db->getQuery(true)
->select('*')
->from($db->quoteName($fcTable))
->where($db->quoteName('field_id') . ' IN (' . $subQuery . ')');
$db->setQuery($query);
return $db->loadAssocList() ?: [];
}
private function log(string $message): void
{
$this->log[] = '[' . date('H:i:s') . '] ' . $message;
@@ -33,6 +33,10 @@ class SnapshotRestoreEngine
'#__contentitem_tag_map' => null, // composite key, handled specially
'#__modules' => 'id',
'#__modules_menu' => null, // composite key, handled specially
'#__tags' => 'id',
'#__fields' => 'id',
'#__fields_values' => null, // composite key, handled specially
'#__fields_categories' => null, // composite key, handled specially
];
/**
@@ -282,6 +286,40 @@ class SnapshotRestoreEngine
$query->where($db->quoteName('moduleid') . ' IN (' . implode(',', $moduleIds) . ')');
break;
case '#__tags':
// Only delete tags that exist in the snapshot — never wipe all tags
$ids = array_filter(array_column($rows, 'id'));
if (empty($ids)) {
return;
}
$ids = array_map('intval', $ids);
$query->where($db->quoteName('id') . ' IN (' . implode(',', $ids) . ')');
break;
case '#__fields':
// Only delete custom fields scoped to com_content.article
$query->where($db->quoteName('context') . ' = ' . $db->quote('com_content.article'));
break;
case '#__fields_values':
// Delete all field values — they are article-scoped
break;
case '#__fields_categories':
// Delete field-category mappings for com_content.article fields only
$prefix = $db->getPrefix();
$fTable = $prefix . 'fields';
$subQuery = $db->getQuery(true)
->select($db->quoteName('id'))
->from($db->quoteName($fTable))
->where($db->quoteName('context') . ' = ' . $db->quote('com_content.article'));
$query->where($db->quoteName('field_id') . ' IN (' . $subQuery . ')');
break;
// #__content and #__content_frontpage are fully owned by com_content
default:
break;
@@ -303,6 +341,10 @@ class SnapshotRestoreEngine
$tables[] = '#__content_frontpage';
$tables[] = '#__workflow_associations';
$tables[] = '#__contentitem_tag_map';
$tables[] = '#__tags';
$tables[] = '#__fields';
$tables[] = '#__fields_values';
$tables[] = '#__fields_categories';
}
if (in_array('categories', $types)) {
@@ -389,37 +389,47 @@ class SteppedBackupEngine
private function stepUpload(SteppedSession $session): void
{
$db = Factory::getDbo();
// Reload profile for remote settings
$query = $db->getQuery(true)
->select('*')
->from($db->quoteName('#__mokosuitebackup_profiles'))
->where($db->quoteName('id') . ' = ' . $session->profileId);
$db->setQuery($query);
$profile = $db->loadObject();
$uploader = match ($session->remoteStorage) {
'ftp' => new FtpUploader($profile),
'google_drive' => new GoogleDriveUploader($profile),
's3' => new S3Uploader($profile),
default => throw new \InvalidArgumentException('Unknown storage: ' . $session->remoteStorage),
};
$session->log('Starting remote upload (' . $session->remoteStorage . ')...');
$result = $uploader->upload($session->archivePath, $session->archiveName);
$remoteFilename = '';
$uploadFailed = false;
if ($result['success']) {
$remoteFilename = $result['remote_path'] ?? $session->archiveName;
$session->log('Remote upload complete: ' . $result['message']);
// Wrapped in its own try-catch so a remote failure does not mark
// the entire backup as failed — the local archive is preserved.
try {
// Reload profile for remote settings
$query = $db->getQuery(true)
->select('*')
->from($db->quoteName('#__mokosuitebackup_profiles'))
->where($db->quoteName('id') . ' = ' . $session->profileId);
$db->setQuery($query);
$profile = $db->loadObject();
if (!$session->remoteKeepLocal && is_file($session->archivePath)) {
@unlink($session->archivePath);
$session->log('Local copy removed');
$uploader = match ($session->remoteStorage) {
'ftp' => new FtpUploader($profile),
'google_drive' => new GoogleDriveUploader($profile),
's3' => new S3Uploader($profile),
default => throw new \InvalidArgumentException('Unknown storage: ' . $session->remoteStorage),
};
$session->log('Starting remote upload (' . $session->remoteStorage . ')...');
$result = $uploader->upload($session->archivePath, $session->archiveName);
if ($result['success']) {
$remoteFilename = $result['remote_path'] ?? $session->archiveName;
$session->log('Remote upload complete: ' . $result['message']);
if (!$session->remoteKeepLocal && is_file($session->archivePath)) {
@unlink($session->archivePath);
$session->log('Local copy removed');
}
} else {
$uploadFailed = true;
$session->log('WARNING: Remote upload failed: ' . $result['message']);
$session->log('Local backup is preserved.');
}
} else {
$session->log('WARNING: Remote upload failed: ' . $result['message']);
} catch (\Throwable $e) {
$uploadFailed = true;
$session->log('WARNING: Remote upload threw an exception: ' . $e->getMessage());
$session->log('Local backup is preserved.');
}
// Update record with remote filename
@@ -433,14 +443,16 @@ class SteppedBackupEngine
$session->currentStep++;
$session->phase = 'complete';
$session->statusMessage = 'Backup complete';
$this->completeRecord($session);
$session->statusMessage = $uploadFailed
? 'Backup complete (remote upload failed — local archive preserved)'
: 'Backup complete';
$this->completeRecord($session, $uploadFailed);
}
/**
* Mark the backup record as complete.
*/
private function completeRecord(SteppedSession $session): void
private function completeRecord(SteppedSession $session, bool $uploadFailed = false): void
{
$db = Factory::getDbo();
$logContent = implode("\n", $session->log);
@@ -490,6 +502,11 @@ class SteppedBackupEngine
];
NotificationSender::send($profile, $record, true, $logContent);
// If remote upload failed, also send a failure notification for the upload
if ($uploadFailed) {
NotificationSender::send($profile, $record, false, "Remote upload failed — see backup log for details.\n\n" . $logContent);
}
}
} catch (\Throwable $e) {
error_log('MokoSuiteBackup: SteppedBackupEngine notification failed: ' . $e->getMessage());
@@ -7,7 +7,7 @@
-->
<extension type="plugin" group="actionlog" method="upgrade">
<name>Action Log - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-04</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -7,7 +7,7 @@
-->
<extension type="plugin" group="console" method="upgrade">
<name>Console - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-04</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -7,7 +7,7 @@
-->
<extension type="plugin" group="content" method="upgrade">
<name>Content - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-04</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<extension type="plugin" group="quickicon" method="upgrade">
<name>Quick Icon - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-02</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -7,7 +7,7 @@
-->
<extension type="plugin" group="system" method="upgrade">
<name>System - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-02</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -136,6 +136,7 @@ final class MokoSuiteBackup extends CMSPlugin implements SubscriberInterface
$session->set('mokosuitebackup.last_cleanup', time());
$this->cleanupOldBackups();
$this->cleanupOldSnapshots();
}
/**
@@ -152,6 +153,93 @@ final class MokoSuiteBackup extends CMSPlugin implements SubscriberInterface
}
}
/**
* Remove old content snapshots per component retention settings.
*
* Respects snapshot_retention_days (max age) and snapshot_retention_count
* (max number to keep). A value of 0 means unlimited for that setting.
*/
private function cleanupOldSnapshots(): void
{
try {
$this->doSnapshotCleanup();
} catch (\Throwable $e) {
error_log('MokoSuiteBackup: cleanupOldSnapshots() failed: ' . $e->getMessage());
}
}
private function doSnapshotCleanup(): void
{
$db = Factory::getDbo();
$params = ComponentHelper::getParams('com_mokosuitebackup');
$retentionDays = (int) $params->get('snapshot_retention_days', 30);
$retentionCount = (int) $params->get('snapshot_retention_count', 20);
// Delete snapshots older than retention_days
if ($retentionDays > 0) {
$cutoff = date('Y-m-d H:i:s', strtotime("-{$retentionDays} days"));
$query = $db->getQuery(true)
->select([$db->quoteName('id'), $db->quoteName('data_file')])
->from($db->quoteName('#__mokosuitebackup_snapshots'))
->where($db->quoteName('created') . ' < ' . $db->quote($cutoff))
->order($db->quoteName('created') . ' DESC');
$db->setQuery($query);
$expired = $db->loadObjectList();
foreach ($expired as $snapshot) {
$this->deleteSnapshotRecord($db, $snapshot);
}
}
// Enforce max count (keep newest)
if ($retentionCount > 0) {
$query = $db->getQuery(true)
->select('COUNT(*)')
->from($db->quoteName('#__mokosuitebackup_snapshots'));
$db->setQuery($query);
$totalCount = (int) $db->loadResult();
if ($totalCount > $retentionCount) {
$excess = $totalCount - $retentionCount;
$query = $db->getQuery(true)
->select([$db->quoteName('id'), $db->quoteName('data_file')])
->from($db->quoteName('#__mokosuitebackup_snapshots'))
->order($db->quoteName('created') . ' ASC');
$db->setQuery($query, 0, $excess);
$oldest = $db->loadObjectList();
foreach ($oldest as $snapshot) {
$this->deleteSnapshotRecord($db, $snapshot);
}
}
}
}
/**
* Delete a snapshot record and its JSON data file.
*/
private function deleteSnapshotRecord(object $db, object $snapshot): void
{
if (!empty($snapshot->data_file) && is_file($snapshot->data_file)) {
if (!@unlink($snapshot->data_file)) {
error_log('MokoSuiteBackup: Could not delete snapshot file (id=' . $snapshot->id . '): ' . $snapshot->data_file);
return;
}
}
try {
$db->setQuery(
$db->getQuery(true)
->delete($db->quoteName('#__mokosuitebackup_snapshots'))
->where($db->quoteName('id') . ' = ' . (int) $snapshot->id)
);
$db->execute();
} catch (\Exception $e) {
error_log('MokoSuiteBackup: Could not delete snapshot record ' . $snapshot->id . ': ' . $e->getMessage());
}
}
private function doCleanup(): void
{
$db = Factory::getDbo();
@@ -7,7 +7,7 @@
-->
<extension type="plugin" group="task" method="upgrade">
<name>Task - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-02</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
@@ -7,7 +7,7 @@
-->
<extension type="plugin" group="webservices" method="upgrade">
<name>Web Services - MokoSuiteBackup</name>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-02</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
+1 -1
View File
@@ -8,7 +8,7 @@
<extension type="package" method="upgrade">
<name>Package - MokoSuiteBackup</name>
<packagename>mokosuitebackup</packagename>
<version>01.27.03</version>
<version>01.29.00</version>
<creationDate>2026-06-02</creationDate>
<author>Moko Consulting</author>
<authorEmail>hello@mokoconsulting.tech</authorEmail>