artifact-audit 2026-07-25: monthly drift pass (6 fixes, 12 flags) - #12
Merged
Conversation
…guardrail docs, runbooks Monthly artifact-audit pass against the knowledge base (verified 2026-07-20). All fixes backed by High-confidence KB entries or verified internal cross-references; hard-stop and protected-area findings are flagged in the PR body only, not rewritten. - new-loop skill: /loop guidance now states session scope, 7-day expiry, and points laptop-closed scheduling at Routines (/schedule); /goal validator described as 'separate validator model (defaults to Haiku)' - guardrails/README.md: document native Claude Code runaway-loop caps (v2.1.212) as complements to (not replacements for) the three hard stops; add Tesla $200/week spend-cap receipt - guardrails/checklist.md: note auto mode is on by default on Bedrock/Vertex/Foundry since v2.1.207 (disableAutoMode) - templates/loop/loop.md: /review-pr -> /review (skill does not exist) - runbooks/first-loop.md: fix primer cross-reference (§5 -> §4 Skills) - runbooks/ui-audit-overnight.md: fix garbled sentence Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KzGwSojW7vAPQiM6pDggD3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Artifact drift
First scheduled run of the monthly
artifact-auditskill. Four parallel audit agents (skills / templates / guardrails / runbooks+docs) checked every practice artifact against the knowledge base (last verified 2026-07-20). No prior artifact-audit PR was open; no update-knowledge PR was in flight. Per the skill: fixes below are backed by High-confidence KB entries or verified internal cross-references; everything touching a hard stop, protected file, or Medium/unverifiable evidence is flagged, not rewritten.Fixed
.claude/skills/new-loop/SKILL.md/looponly for recurring/scheduled work" steered scheduled/unattended jobs to/loop, which is session-scoped (no laptop-closed runs; recurring tasks expire after 7 days). Now states the limits and points laptop-closed scheduling at a Routine (/schedule, alias/routines)./loop+ Routines entries — High (code.claude.com/docs/en/routines read directly).claude/skills/new-loop/SKILL.md/goalwith a Haiku validator" → "separate validator model (defaults to Haiku)" — Haiku is the default, not fixed.guardrails/README.mdguardrails/README.mdguardrails/checklist.mddisableAutoModeto opt out) — exactly the unintended-autonomy case the item exists to catch.templates/loop/loop.md/loop 15m /review-pr 1234calls a skill that doesn't exist →/review(the bundled PR-review skill).runbooks/first-loop.mdrunbooks/ui-audit-overnight.mdFINDINGS.mdis already in place.").Flagged for you (not changed — human decision needed)
Hard-stop / protected areas:
templates/ralph/run.sh— the budget ceiling is a disarmed placeholder by default (highest-severity finding).cost_so_far()returns${LOOP_COST_USD:-0}, which nothing in the loop updates — so hard stop knowledge: 2026-06-22 update pass (7-day delta from Jun 15) #3 evaluates0 >= 10forever unless the user wires a meter. Two decisions: (a) the comment's exampleclaude costcommand is not documented anywhere in the KB (documented spend surfaces are/usage, the Rate Limits API, the Analytics Admin API); (b) consider making the script refuse to start when the meter is unwired, per the repo's own "never run an uncapped loop." Related non-hard-stop nit:eval "$SUCCESS_CHECK"evals an env-overridable string..claude/skills/artifact-audit/SKILL.md(this skill's own guardrails) — "the nativeCLAUDE_CODE_MAX_SUBAGENTS_PER_SESSIONcap bounds the ~4-agent fan-out" overstates the default: it's 200 natively (v2.1.212), no practical bound unless set ≈12 in the Routine env asrunbooks/staying-current.mdinstructs. Suggested wording: "…theCLAUDE_CODE_MAX_SUBAGENTS_PER_SESSIONcap (set ≈12 in the Routine env — native default is 200) bounds the ~4-agent fan-out."guardrails/README.mdhard-stops section never states thatmax_turns/max_budget_usd(Agent SDK) and--max-iterations(ralph-wiggum) all default to unlimited — KB-High and load-bearing, but inside the protected three-hard-stops text. (Alsomax_budgets_usdtypo →max_budget_usd.)templates/ui-audit/PROMPT.mdreferences aRUN.mdthat doesn't exist in the template dir ("stop at the caps set in RUN.md") — the caps actually live inrunbooks/ui-audit-overnight.md's invocation. Flag-only because it defines where the loop's caps live.runbooks/ui-audit-overnight.md— the recommended interactive lane's budget stop is a morning/usagecheck, i.e. an alert, not enforcement (the runbook is honest about it and offersrun.shas the hard path).Possibly-dead or KB-unverifiable claims (re-verify against live docs; candidates for the KB re-verify backlog):
runbooks/babysit-prs.md— the laptop-closed path rests on a "cloud auto-fix-PR feature", a "web CI bar", and/autofix-pr, none of which the KB tracks. The KB-verified mechanism is a Routine with a GitHub-event (pull_request) trigger via/schedule. If/autofix-prisn't real, this runbook's core path is a dead end — likely needs a rewrite around Routines. (High severity if confirmed.)Ctrl+Cfor/goaland/cancel-ralphappear inguardrails/checklist.md,runbooks/first-loop.md, andrunbooks/ui-audit-overnight.md, but neither is in the KB (onlyEscfor/loopis verified).runbooks/staying-current.md— four unbacked claims:/scheduledisabled in web sessions; a/schedule updatesubcommand; a "per-account daily routine-run cap" (load-bearing — both maintenance skills cite that "Budget & caps" section as their budget hard stop); Routine network tiers (Trusted/Full/Custom).templates/goal/recipes.md— operational details not in the KB (alias liststop/off/reset/cancel, ~4,000-char condition limit, trust-dialog/disableAllHooksprerequisites, headlessclaude -p "/goal …"). Not contradicted, just unverified.Behavior-shift notes (v2.1.198, subagents background-by-default + auto-commit/push/draft-PR on worktree finish):
templates/loop/loop.mdandrunbooks/babysit-prs.mdboth say "use a worktree agent to fix them" with no note that worktree agents now auto-commit/push/open draft PRs without pausing — decide whether to annotate or constrain.Descriptive-only observations:
guardrails/README.mdintro says Anthropic's billing layer "does not auto-disable" — still true per the KB, but now understates the Jul 2 Claude Enterprise spend controls (alerts/entitlements, Medium-High, not confirmed hard-stop). Human to decide whether/how to acknowledge without softening the rule.CLAUDE.md(protected) — its Routines note omits that Routines run with no permission prompts (KB-High, safety-relevant).Clean (no drift found)
loop-guardrailsandupdate-knowledgeskills,templates/ralph/PROMPT.md,templates/goal/recipes.md(load-bearing claims), the ui-audit PREFLIGHT/PAGES/FINDINGS files,runbooks/overnight-build.md,README.md,budget.env, and both self-edit-guard files (all sentinel regions, paths, and the 10-line cap verified on disk).No
self-edit:commits in this PR; nothing here touchesknowledge/. Human reviews and merges — never merged to main by the routine.🤖 Generated with Claude Code
https://claude.ai/code/session_01KzGwSojW7vAPQiM6pDggD3
Generated by Claude Code