Skip to content

fix: audit follow-ups — batch 1.5, OpenCode spec-workflow port (Option A), tollgate v2 Stop gate#67

Merged
hapo-nghialuu merged 14 commits into
devfrom
fix/audit-batch-1
Jul 17, 2026
Merged

fix: audit follow-ups — batch 1.5, OpenCode spec-workflow port (Option A), tollgate v2 Stop gate#67
hapo-nghialuu merged 14 commits into
devfrom
fix/audit-batch-1

Conversation

@hapo-nghialuu

@hapo-nghialuu hapo-nghialuu commented Jul 17, 2026

Copy link
Copy Markdown
Member

Summary

Follow-up train to the 2026-07 full-package audit (docs/audit-cafekit-vs-claude-code-2026-07.md, run against v0.13.2). Batch 1 (9 fixes) merged in #66; hook hardening merged in #65. This PR carries four workstreams, 14 commits, each independently revertable. Branch is rebased on dev after both merges; the full self-test suite passes at 150 tests on the final tree.


A. Batch-1 leftover cleanup (a705f98, 71f6b45)

  • Purged the last references to the removed generate-graph/impact-analysis skills and stale Gemini model IDs missed by fix: audit batch 1 — validator, installer, and payload fixes #66 (READMEs, ai-multimodal references, changelog).
  • Test-count parsers in run-skill-self-tests.mjs anchor on line starts; ANSI color prefixes from FORCE_COLOR environments broke them into false NO_TESTS. Child processes now run with NO_COLOR=1, FORCE_COLOR=0.

B. Audit batch 1.5 — items #10/#11/#13 (cb05ace, f963bff, 937fa5c, 6dfdb65)

C. OpenCode spec-workflow port, Option A (dc74f1e, 41de4d0, 256f44b, 0ea97bc)

Closes 2 of the 3 enforcement hooks the audit flagged as missing from the OpenCode runtime:

  • plugins/spec-state.ts — tollgate reminder via chat.message, pushing a schema-complete synthetic text part (id+sessionID+messageID required; a bare part crashes the turn — verified on opencode 1.17.15). Same state-change fingerprint gate as the Claude hook. Escape hatch spec.tollgate: false.
  • plugins/task-scaffold-guard.ts — blocks hand-written task files on tool.execute.before. Covers OpenCode-specific creation vectors the Claude hook never needed: apply_patch (*** Add File: scan, defensive) and edit on a non-existent task file (OpenCode's edit can create files, unlike Claude's Edit). Sandbox-virtual paths resolved against the plugin directory.
  • Escape-hatch hardening (both platforms): smoke tests proved that when the block message advertised the runtime.json override, the model simply wrote the override and disarmed the guard. Block messages no longer mention the hatch; it stays functional for humans and is documented in docs/opencode-hook-port-audit.md only.
  • agent.cjs port explicitly dropped (unreliable subagent surface, low ROI) — decision recorded in the port audit doc.

D. Tollgate v2 — Stop-gate completion enforcement (6df8473, 80b1ae0, 74ca18c)

Audit §3.4.1: the per-prompt tollgate was a reminder, not a barrier — the model could mark tasks done with no verification evidence. Enforcement now moves to the turn boundary:

  • New hooks/spec-gate.cjs (Stop hook, registered first on Stop). When a turn ends with newly-done tasks in the active spec's task_registry, each must carry a verification receipt: task-md Status: done (a), an Evidence section (all 3 heading aliases) (b) containing no {{...}} placeholders and at least a fenced block or PASS/FAIL/exit-code line (c), and completed_at set (d). Violations emit {"decision":"block","reason":…} listing each failing task and the exact fix — the assistant must repair before it can end the turn.
  • Safety valves (scaffold-guard playbook): stop_hook_active loop guard; first run seeds the status cache without blocking (legacy specs are never gated retroactively); failing tasks are held at their old cache status so the gate re-fires on the next Stop; escape hatch spec.completion_gate: false (missing key → ON); crash-wrapper fail-open.
  • spec-state.cjs slimmed: the bilingual ALL-CAPS tollgate wall is gone — state-change turns get a ≤7-line neutral summary; unchanged turns keep the one-liner. Hard enforcement now lives on Stop, so the loud prompt no longer pays its context cost.
  • runtime.json template documents spec.{scaffold_guard, completion_gate}; rules/state-sync.md describes the gate; 7 new behavioral tests in hooks/__tests__/spec-gate.test.js (subprocess harness from fix(hooks): harden env escaping and privacy symlink resolution #65).

Behavior changes at a glance

Surface Before After
Ending a turn with a receipt-less done task allowed blocked (Stop gate, actionable reason)
Per-prompt tollgate block red ALL-CAPS bilingual wall (~460 tok) ≤7-line neutral summary
Scaffold-guard block message advertised its own disable switch hatch hidden from the model (both platforms)
OpenCode task-file creation (write/apply_patch/edit-create) unguarded blocked → scaffold + Edit-fill
OpenCode spec tollgate none injected per turn (chat.message)
Document-skill authorship haposoft Anthropic, PBC — adapted by Haposoft

Verification

Known follow-ups (non-blocking)

  1. Config key naming drift: Claude uses spec.completion_gate, OpenCode uses spec.tollgate; tollgate is not listed in the shipped runtime.json template. Unify or document.
  2. OpenCode reminder still loud: spec-state.ts preserves the full URGENT block. Defensible — OpenCode has no Stop-gate equivalent, so the loud prompt is its only completion control — but the port-audit doc should state that rationale explicitly.
  3. .opencode/plugins/.logs/ cache dir should be confirmed covered by the installed .opencode/.gitignore.
  4. Gate scope matches spec-state.cjs: only the first in_progress spec is gated per turn (pre-existing limitation, now shared by two hooks).

Upgrade path

Existing installs pick everything up on npx @haposoft/cafekit --upgrade: the settings merge appends spec-gate.cjs to the existing Stop entry (per-command merge shipped in #66), the manifest ships the new hook, and the first-run cache seed guarantees no retroactive blocking on in-flight specs.

… changelog

Review findings on PR #66 (self-review):
- video-analysis.md / audio-processing.md still carried gemini-3-pro-preview
  and gemini-1.5-flash; model sections were also duplicated after the bulk
  rename — collapsed to a single gemma-4-31b-it recommendation pointing at
  runtime.json.
- Root README and package README still advertised hapo:generate-graph and
  listed impact-analysis as bundled (package README ships in the npm tarball).
- CHANGELOG [Unreleased] and docs/project-changelog.md now document the batch
  including the breaking skill removal.
parseNodeTestCount anchors on line-start 'ℹ tests N'; when the child
inherits a color-forcing env the ANSI prefix breaks the match and the
suite false-fails with NO_TESTS despite all tests passing. Spawn test
children with NO_COLOR=1 / FORCE_COLOR=0.
…ins (Option A)

Closes the spec-workflow enforcement gap on OpenCode. Both plugins were
independently verified with live smoke tests on opencode 1.17.15:

- spec-state.ts injects the tollgate via chat.message. Parts pushed onto
  output.parts MUST be schema-complete (id/sessionID/messageID) — the
  original bare {type,text} part failed OpenCode's durable-part schema and
  crashed every user turn ('invalid user part before save'); fixed and
  verified end-to-end (tollgate text reaches the model).
- task-scaffold-guard.ts blocks write/apply_patch on task files, and
  additionally gates edit on non-existent task files: OpenCode's edit can
  CREATE files (unlike Claude's Edit), a bypass proven live. Edit on an
  existing stub stays allowed. Existence checks resolve the specs/ suffix
  against the plugin directory (tool paths may be sandbox-virtual).
- Block message no longer advertises the runtime.json escape hatch: the
  smoke test caught the model writing scaffold_guard:false itself and
  disabling the guard. The hatch stays functional for humans.
- session.ts compaction banner + AGENTS.md now map Claude-only tools to
  OpenCode built-ins (AskUserQuestion->question, TodoWrite->todowrite,
  Task->agent/subtask).
- agent.cjs remains unported (documented: low ROI, unreliable surface).
- Port audit + plugin contract docs updated with the verified findings.
…odel

The OpenCode smoke test proved the self-disarm vector: shown the override
line in the block message, the model wrote runtime.json with
scaffold_guard:false and bypassed the guard instead of scaffolding. The
Claude hook printed the same line — removed. The escape hatch remains
functional and documented for humans.
Owner-local project preferences, split out of the feature commit for a
clean history.
@hapo-nghialuu hapo-nghialuu changed the title fix: audit batch 1 — 9 verified fixes from the 2026-07 full-package audit fix: audit follow-ups — batch 1.5, OpenCode spec-workflow port (Option A), tollgate v2 Stop gate Jul 17, 2026
@hapo-nghialuu
hapo-nghialuu merged commit ec9a13b into dev Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant