docs(e1): backup-format contract + live-(A) agreement guard (opens Track E)#61
Merged
Merged
Conversation
…ack E)
## What
modules/backup/ carries the live daily backup generator (cron.sh) plus a ten-file
PITR/encrypted draft cluster, and across them a backup artifact is named, located,
time-stamped, encrypted, and checksummed three incompatible ways. Wiring the encrypted
backup (E2) and binlog/PITR (E3) would cement those three dialects unless one contract
is fixed first. E1 fixes it — behavior-free: a doc + a guard + the ledger.
New docs/backup-format-contract.md — the canonical source of truth for backup-artifact
shape. It transcribes the live (A) format exactly from cron.sh and the restore arm
(cli/actools:241): ${INSTALL_DIR}/backups/<env>_db_<YYYY-MM-DD>.sql.gz (+ .sha256),
content gzip(mariadb-dump --single-transaction --quick actools_<env>), flat directory,
daily timestamp, 7-day retention, integrity-or-delete, and the umask-077
--defaults-extra-file password shape. It then defines the canonical scheme X (naming
grammar <env>_<kind>_<timestamp>.<ext>[.age][.sha256]; Age encryption with the .sha256
taken over the ciphertext; the PITR nested layout rooted under ${INSTALL_DIR}/backups/;
the secure password shape mandatory for every producer) and a divergence ledger
recording, per dialect, where the encrypted (B) and PITR (C) drafts diverge from X and
what E2/E3 must do to conform. B and C are marked TARGET / NOT YET LIVE throughout.
New tests/guards/backup_format_contract_guard_test.bats — pins ONLY the live (A)
producer<->consumer agreement at this baseline, mirroring the discipline of
cron_security_shape_guard_test.bats. It renders the live cron through the existing
tests/helpers/capture_backup_cron.sh and asserts the producer DB pattern
(<env>_db_<…>.sql.gz under a backups/ root), the .sha256 integrity sidecar + sha256sum
-c, the restore consumer's default glob agreeing on the same root/stem/extension, and
the consumer checksum convention. Two permanent non-vacuity arms doctor the producer
stem (_db_ -> _database_) and the restore glob on OFF-TREE scratch copies and assert the
agreement oracle FAILS ("CONTRACT DRIFT: producer writes '…' but the restore consumer
globs '…'"); the repo is never modified. Discovered by the recursive bats job (lint.yml:
bats -r tests/) — no workflow edit.
## Gate
Behavior-free — no producer, consumer, draft, module, or golden is touched; nothing
executable changes, so no branch e2e is required (like C1/D1b/D2/V1).
Runtime-authority-changes: none (cli/actools, cron.sh, all modules byte-identical).
## Scope / verification
- Scope = 3 files: docs/backup-format-contract.md (new),
tests/guards/backup_format_contract_guard_test.bats (new), and
docs/runbooks/PHASE0_LEDGER.md (Entry 031 + ratify Entry 030).
- bats -r tests/: 241 -> 247 (+6 new arms, all green); the 12 pre-existing
jq-dependent secrets/state failures are environmental and unchanged (no regression).
- cron.sh, cli/actools, the 10 draft files, tests/fixtures/golden/backup-cron/*,
capture_backup_cron.sh, and modules/host/age.sh are byte-identical to baseline c17197f.
- Author actools-pl <feezixmp@gmail.com>; no Co-authored-by.
Ledger: add Entry 031 (E1, Pending); ratify Entry 030 (V2, c17197f/#59, closes Track V),
original Pending text preserved verbatim. Entries 029..001 byte-identical.
REVIEW (guard non-vacuity + doc-vs-code spot checks) then DOC-CHECK (the contract doc is
the primary deliverable) follow. The coding window does not self-approve.
actools-pl
added a commit
that referenced
this pull request
Jun 30, 2026
…store decrypts (Track E) (#62) ## What Wire Age-encryption-at-rest into the canonical backup path per the E1 contract (scheme X, variant B). The standalone modules/backup/encrypted_backup.sh draft — which dumped with an argv password (-p"${DB_ROOT_PASS}", the form cron_security_shape_guard_test.bats forbids) and used a per-second timestamp — is absorbed and removed. Its encryption logic moves into the live generator modules/backup/cron.sh behind a new ENABLE_ENCRYPTED_BACKUP flag (default off, threaded into the render exactly as ENABLE_S3_STORAGE). Producer (cron.sh): when the flag is on, each DB dump and (S3-off) files archive is encrypted after its dump+checksum+integrity step with age -r "$(cat ${INSTALL_DIR}/.age-public-key)" -o "<artifact>.age" "<artifact>"; the .sha256 is taken over the CIPHERTEXT; on a successful encrypt+verify the plaintext artifact + its sidecar are removed (integrity-or-delete: on failure the partial .age is removed instead). The secure umask-077 --defaults-extra-file dump shape is unchanged (encryption is a post-dump host-side step touching no credential), so the draft's argv-password flaw never ships. Retention and the rclone push are extended to cover *.age / *.age.sha256. The daily <YYYY-MM-DD> timestamp is kept (per-second stays PITR/C). Consumer (cli/actools restore arm): selects the newest of <env>_db_*.sql.gz or *.sql.gz.age; requires ${INSTALL_DIR}/.age-key.txt when the selection is encrypted; decrypts with age --decrypt -i before gunzip | mariadb; and — hardened — ABORTS before any DROP DATABASE when a present .sha256 fails (a missing sidecar still only warns). No dispatch command added or removed (REGISTERED stays 30). Contract (docs/backup-format-contract.md): variant B moves from TARGET / NOT YET LIVE to LIVE (gated); the scheme-X timestamp grammar drops "encrypted" from the per-second producer list (encrypted-daily uses the daily timestamp). A is unchanged in substance; C stays not-yet-live. Guards: backup_format_contract_guard_test.bats gains arms 7-10 (encrypted producer, encrypted consumer, anchored-root non-vacuity, ciphertext-checksum non-vacuity), makes arm 6's consumer-drift sed global (the restore arm now has two "${env}"_db_ globs), and anchors the consumer-root check to the closing quote. cron_security_shape_guard_test.bats gains arm 5 (secure shape holds with encryption on). The backup-cron golden is regenerated (not hand-edited); golden_drift 6/6 and backup_cron_drift 3/3 stay green. e2e (.github/workflows/e2e.yml): the install env sets ENABLE_ENCRYPTED_BACKUP=true and a new "Encrypted backup round-trip" step produces a backup, asserts the encrypted artifact + verifying ciphertext checksum + no plaintext remains, then restores non-interactively and asserts the DB reloads. ## Gate Behavior-CHANGING — a branch e2e MUST be green before merge (the round-trip step is the live proof; "MariaDB ready." is the real signal, an SSH-timeout is infra -> re-run). ## Scope / verification - bats -r tests/: 247 -> 252 total (+5 new arms, all green); the 12 pre-existing jq-dependent secrets/state failures are environmental and unchanged (no regression). - modules/host/age.sh, tests/helpers/capture_backup_cron.sh, and all PITR/binlog drafts are byte-identical to baseline 6d06a73. - Consequential edits (declared): live_module_file_inventory_test.bats manifest and runtime-authority-map.md drop the removed draft. - Author actools-pl <feezixmp@gmail.com>; single-author commit, no co-author trailer. Ledger: add Entry 032 (E2, Pending; behavior-changing; branch-e2e-required); ratify Entry 031 (E1, 6d06a73/#61, opens Track E), original Pending text preserved. Entries 030..001 byte-identical. REVIEW (re-derive scope; byte-identity; guard non-vacuity; golden drift; patch reproduces the tree) then DOC-CHECK then the operator's green branch e2e follow. The coding window does not self-approve.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Opens Track E (the 4.5 build) with the canonical backup-format contract — the single
source of truth the encrypted-backup (E2) and PITR (E3) drafts must conform to.
Behavior-free: a contract doc + a guard pinning the live producer↔consumer agreement
Why
modules/backup/holds the livecron.shplus three divergent backup dialects:backups/<env>_db_<YYYY-MM-DD>.sql.gz+.sha256, flat, plaintext.<env>_db_<ts>.sql.gz.age, per-second ts.backups/db/<date>/,binlogs,
--master-data=2.Wiring B/C without a contract would cement three incompatible schemes.
Deliverables
docs/backup-format-contract.md— the live (A) format pinned exactly fromcron.sh/restore; the canonical scheme X (naming grammar, Age encryption with thechecksum over the ciphertext, the PITR layout, integrity-or-delete, the umask-077
--defaults-extra-filepassword shape mandatory for all producers); the A/B/Cdivergence ledger with E2/E3 conformance actions. B and C are marked
TARGET / NOT YET LIVE.
tests/guards/backup_format_contract_guard_test.bats— renders the live cron viacapture_backup_cron.shand asserts the producer DB pattern, the.sha256sidecar,the consumer↔producer glob agreement, and the checksum convention, with two biting
non-vacuity arms (producer-stem drift, restore-glob drift). Pins only live (A).
Recorded finding (for E2/E3)
All three drafts pass the DB password on argv (
encrypted_backup.sh:30,db-full-backup.sh:54/98,pitr-restore.sh:155/203) — the insecure shapecron_security_shape_guardpurged from the live cron. The contract documents theexact lines E2/E3 must convert to the secure
--defaults-extra-fileshape.Verification
non-vacuity reproduced with an independent oracle.
doc-claim guard 5/5; version header byte-identical to a sibling doc.
author
actools-pl <feezixmp@gmail.com>.bats -r tests/: +6 guard arms, 0 new failures.