Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions docs/project-changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,35 @@ All notable changes to CafeKit are documented here, following

## [Unreleased]

## [0.13.2] - 2026-06-21

### Added — Enforce scaffold on task creation
- **`task-scaffold-guard.cjs` (PreToolUse hook)**: hard-blocks any `Write` to `specs/<feature>/tasks/task-*.md`, forcing task files to be generated via `spec-scaffold.cjs` then `Edit`-filled. Fixes the field-test dodge where the model hand-wrote every task file and skipped the opt-in scaffold step. Only the `Write` tool on a task-file path is blocked — `Edit`/`MultiEdit`, other `Write`s, and the scaffold script itself are untouched.
- **Safety valves**: fail-open when the scaffold script is missing (no deadlock), an actionable block message with the exact command, and a `.claude/runtime.json` escape hatch (`"spec": { "scaffold_guard": false }`).
- **Validator placeholder gate**: `validate-spec-output.cjs` now hard-fails a task that still has an unfilled `{{...}}` scaffold placeholder (warns on a leftover `.../` path). Fill-side complement to the guard: the hook forces creation via scaffold, this proves the stub was completed.

### Changed
- `settings.json`: guard registered under a dedicated `Write` matcher in `PreToolUse`.
- `migration-manifest.json`: hook declared in `runtime.files` so the installer ships it.
- Specs `SKILL.md` Step 7: scaffold is now mandatory (Write to a task file is blocked), not a suggestion.

### Notes
- Buys process discipline and consistent `task_registry`/`task_files`, not a large token reduction — task content is still emitted via `Edit`.

## [0.13.1] - 2026-06-21

### Added — Specs v2 (quality + grounding + output-cost)
- **Layer 2 grounding (`spec-ground.cjs`)**: new deterministic check that greps the real work tree to verify every `Related Files` path a task cites (Modify/Delete/Read) exists, or is Created earlier in the spec. Closes the gap where the structural validator checks spec *shape* but is blind to phantom file paths. Active-grep (not opt-in). Wired into Step 8.5 + the `--validate` gate. `--root` for monorepo/sibling-project specs.
- **Spec scaffolding (`spec-scaffold.cjs`)**: generates spec.json + doc templates + task stubs so the model Edit-fills placeholders instead of hand-Writing whole files (the dominant output-token cost). `--tasks-only` merges task stubs + task_files/task_registry into an existing spec without overwriting filled tasks. Wired into Step 7.
- **Execution Tier (Light/Standard/Deep)**: auto-scales research/discovery/review depth by complexity so small specs skip full-pipeline overhead. Recorded in `spec.json.design_context.execution_tier`. Quality floor (scope_lock, EARS, Layer 1 + Layer 2) never skips.
- **Evidence-gated red-team**: `review.md` Step 5.5 auto-rejects findings that don't cite a concrete task/section or verbatim quote (spec-level analogue of ck-plan's `file:line` gate).
- **Definition of a Complete Task (DoCT)**: explicit quality bar in SKILL.md mapping each completeness element (real paths, contract, measurable acceptance, real evidence commands, reachability, requirement mapping) to its enforcing mechanism.
- **Frontend Fidelity Rule**: when a frontend task has a provided visual reference (design image, Figma, screenshot, palette, design tokens, style guide), the task MUST reproduce it faithfully — extract concrete values (exact hex, font, spacing, verbatim UI text), state a `match <reference>` constraint, and prove fidelity in Evidence. New components with a reference must cite its tokens; conditional so brownfield reuse is unaffected. (`tasks-generation.md` + DoCT)
- **Complexity Smell Check (by numbers)**: quantitative YAGNI tripwires in scope inquiry (>8 files / >2 new services / >12 tasks → challenge; >15 tasks → split into sibling specs), targeting the mega-spec failure mode. Surfaced for the user to decide (Expand/Hold/Reduce/Split), never silently built. (`scope-inquiry.md` + SKILL.md Step 3)

### Changed — Specs v2
- **Literal `R{N}.{M}` is now the default requirement format** (template + SKILL.md Step 5). Wakes per-criterion coverage (P1d), which the model previously dodged by writing a bare numbered list. NFR section uses literal IDs too.

## [0.13.0] - 2026-06-18

### Changed
Expand Down
67 changes: 67 additions & 0 deletions notes/v0.13.0-field-test-post-list-screen.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Field Test v0.13.0 — spec `post_list_screen` (e8194-ai-mining-sns)

> Ghi chú đánh giá thực địa. Spec production thật, đa tầng (NestJS + Nuxt 3 + MySQL).
> Conversation: `~/.claude/projects/-Users-nghialuutrung-Desktop-e8194-ai-mining-sns/b6cfe865-b814-40d7-b82a-30874b2030ef/`
> Ngày: 2026-06-18. Chưa commit — memo nội bộ.

## Bối cảnh test
- Yêu cầu: tính năng màn danh sách bài đăng (一覧画面), 5 tab trạng thái, bulk actions, filter, 9 ảnh references.
- Bản chạy: **đúng 0.13.0** (đã verify: Step 8 Hydration đã bỏ, không còn `task-hydration.md`/`tasks-parallel-analysis.md`, `cafekit.json` version=0.13.0).
- Kết quả: 16 task files, validator PASS, red-team 4-persona, `ready_for_implementation=true`.

## 1. Đo lường vận hành (số cứng từ session JSONL)
| Chỉ số | Giá trị | So game spec (27m) |
|---|---|---|
| Thời gian | **1h38m04s** | 3.6× |
| Output tokens | **935.000** | 3× |
| cache_read đỉnh | **296.000** (vượt 200K, nhờ model 1M) | — |
| Timeout/API error | **8 lần** | rủi ro resume |
| Tool calls | 27 Read · 21 Write · 16 Edit · 12 TaskUpdate · 6 TaskCreate · 2 Agent | output-bound |

→ Thủ phạm 1h38m = **spec quá lớn (16 task) + output 935K + 8 timeout**, KHÔNG phải context nạp.

## 2. Cải tiến v0.13.0 phát huy thế nào — KẾT QUẢ HỖN HỢP
| Cải tiến | Thực địa | Bằng chứng |
|---|---|---|
| P1b timestamps | ✅ thành công (gián tiếp) | 6 mốc riêng biệt, không reuse init |
| **P1d sub-criteria** | ❌ **BỊ NÉ** | requirements dùng 90 numbered-list, 0 literal `Rx.y`. Log AI ghi thẳng: *"dạng list số — tránh kích hoạt check sub-criteria"* |
| **#3 contract drift** | ❌ **BỊ NÉ** | design 0 contract marker. Log: *"cố ý KHÔNG dùng marker... rủi ro lớn hơn lợi ích"* |
| P0 tollgate + #2 + A1 | ⚪ vô nghĩa ở case này | tiết kiệm vài K token, bị 935K output át |

### PHÁT HIỆN GỐC: incentive ngược
- P1d/#3 là opt-in **CÓ downside** (dùng sai → validator FAIL). AI tính ra **né hoàn toàn là tối ưu** (pass chắc, 0 rủi ro).
- "Format rõ ràng" vô tình thành "gánh thêm rủi ro validator" → ai cũng tránh.
- Mỉa mai: spec này CHÍNH XÁC là loại đa-tầng BE↔FE mà #3 sinh ra để bảo vệ. RT-2 (red-team thủ công) đã bắt đúng 1 contract drift — nhưng bằng LLM, không phải cơ chế deterministic đã xây.

## 3. Chất lượng SẢN PHẨM: xuất sắc (9/10)
- **Coverage 14/14 requirement** đều có task ref (R9 ít nhất 1, R6 nhiều nhất 4). Không mồ côi.
- Contract khóa từ **source thật**: PostStatus 1-8, CancelApproval=4, platform X=3/IG=2/FB=1, delete_flg=1.
- Evidence trung thực: ghi rõ "FE không có test runner → chỉ typecheck+lint+manual", không bịa test.
- Red-team bắt 4 lỗ thật, đáng giá nhất **RT-2**: endpoint `bulk-cancel-approval` xung đột cap với spec `cancel_bulk_approval` đang tồn tại → reconcile max 200 BE / cap 10 UI.

## 4. Đọc ảnh references: CHÍNH XÁC xuất sắc
- Chép đúng **từng chữ JP** modal/button/field từ 6 ảnh đã verify tận mắt (R4.3/R4.5/R8.3/R8.6 khớp nguyên văn).
- **Bắt mâu thuẫn 2 ảnh cùng màn**: image_1 nút 「編集」 vs image_2 nút 「設定」 → Open Q1. Chứng minh nhìn+so sánh thật, không bịa.
- **Phân biệt nội dung vs nhiễu**: banner đỏ「予期しないエラー」 ở ảnh 7/8 (artifact lỗi screenshot) → không chép mù, ghi Open Q6 hỏi nguồn text.
- Lưu ý: tiêu đề màn thật trong ảnh là 「投稿カレンダー」, spec đặt route `/posts/list` theo mô tả — hợp lý nhưng cần PM xác nhận route.

## 5. Chỉ dẫn UI trong task: đủ structure, thiếu pixel-style
- **CÓ (tầng structure/behavior)**: 14 component đặt tên (`PostListTable.vue`, `ModalRepost.vue`...), file path chính xác, Read/Create/Modify rõ, label JP nguyên văn, `#DAF1EE` ×5, tham chiếu "khớp ảnh 2/3", format `M月D日 HH:mm`.
- **THIẾU (tầng pixel)**: 0 spacing/padding/font-size/tailwind/token. Component **Create mới** (`PostListTable`, `ModalRepost`, `PostListPlatformFilter`) chỉ tả cấu trúc, không tả visual → dev tự suy từ ảnh + component anh em.
- Đánh giá: hợp lý cho **brownfield** (reuse design system sẵn thay tả pixel). Khác hẳn case game greenfield (0 hex/font/path = lỗ nghiêm trọng). Chỗ hở duy nhất: style component Create mới (badge color, button variant).

## 6. KẾT LUẬN then chốt
1. **Chất lượng spec không nhờ P1d/#3** (cả 2 bị né) — nhờ **xương sống vốn có**: evidence gate (scout source thật) + red-team 4-persona + input đại ca chi tiết. → **0.12.0 cũng cho spec tương đương**. Giá trị thật của hệ nằm ở evidence+red-team, không ở 2 validator-check đã thêm.
2. **Công sức 0.13.0 đặt sai trọng tâm**: tối ưu *context nạp* + xây *cơ chế chặt opt-in*. Thực địa: (a) context bị 935K output át, (b) cơ chế chặt bị né vì opt-in-có-downside. Chỉ P1b "thắng" mà case này AI vốn đã tự đúng.
3. **Đòn bẩy thật chưa đụng**: (a) output sinh ra (thinking + viết lại), (b) spec-size quá lớn = nguồn 1h38m + 8 timeout.

## 7. Việc đáng làm tiếp (theo thứ tự đòn bẩy thật)
1. **Sửa incentive P1d/#3** — đổi "opt-in im lặng" → SKILL.md chỉ thị BẮT BUỘC ("spec đa-tầng BE+FE PHẢI dùng contract marker"; "requirements PHẢI dùng literal Rx.y"). Biến né-được thành phải-dùng. Nếu không → gỡ luôn cho đỡ ảo tưởng an toàn.
2. **Chống spec-quá-lớn** — spec >10 task PHẢI tách hoặc song song hóa sinh task. Đây mới là thứ cắt 1h38m.
3. **Resume an toàn sau timeout** — 8 lần ngắt là rủi ro mất việc thật.
4. (phụ) Bổ sung style-reference 1 dòng cho component Create mới khi cần pixel-perfect.

## Điểm tổng
- Sản phẩm spec: **9/10**
- Cải tiến-v0.13.0-phát-huy: **4/10** (2/3 validator-check bị né)
- Vận hành: **4/10** (1h38m, 8 timeout)
32 changes: 32 additions & 0 deletions packages/spec/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.13.2] - 2026-06-21

### Added — Enforce scaffold on task creation
- **`task-scaffold-guard.cjs` (PreToolUse hook)**: hard-blocks any `Write` whose path matches `specs/<feature>/tasks/task-*.md`, so task files can only be created via `spec-scaffold.cjs` and then `Edit`-filled. Closes the dodge where the model hand-`Write`s task files and bypasses the (previously opt-in) scaffold step. Narrow scope: only the `Write` tool on a task-file path is blocked; `Edit`/`MultiEdit` and `Write` to any other file are untouched, and the scaffold script (writing via Node fs through Bash) is never blocked.
- **Three safety valves**: fail-open when `spec-scaffold.cjs` is absent (a hook shipped without its script must not deadlock task creation); actionable block message carrying the exact scaffold command; escape hatch via `"spec": { "scaffold_guard": false }` in `.claude/runtime.json`.
- **Validator placeholder gate (`validate-spec-output.cjs`)**: a task file that still carries an unfilled `{{...}}` scaffold placeholder now hard-fails (previously a prompt-only DoCT rule). A leftover `.../` path fragment warns. This is the fill-side complement to the guard — the hook forces task files through the scaffold; this proves the resulting stubs were actually completed, closing the "stub created but not filled" gap.

### Changed
- `settings/settings.json`: registered the guard under a dedicated `Write` matcher in `PreToolUse` (separate entry so the settings-merge dedupe does not swallow it).
- `migration-manifest.json`: declared `hooks/task-scaffold-guard.cjs` in `runtime.files` so the installer ships it alongside `spec-scaffold.cjs`.
- `skills/specs/SKILL.md` Step 7: scaffold is now stated as **mandatory** (raw `Write` to a task file is blocked), not a suggestion.

### Notes
- Enforces *process discipline* (every spec goes through scaffold; `task_files`/`task_registry` stay consistent), not a large token cut — task content is still emitted via `Edit`.

## [0.13.1] - 2026-06-21

### Added — Specs v2 (quality + grounding + output-cost)
- **Layer 2 grounding (`spec-ground.cjs`)**: deterministic check that greps the real work tree to verify every `Related Files` path a task cites (Modify/Delete/Read) exists, or is Created earlier in the spec. Active-grep, not opt-in. Wired into Step 8.5 + `--validate`. `--root` for monorepo specs.
- **Spec scaffolding (`spec-scaffold.cjs`)**: generates spec.json + doc templates + task stubs so the model Edit-fills placeholders instead of hand-Writing whole files (cuts the dominant output-token cost). `--tasks-only` merges into an existing spec without overwriting filled tasks.
- **Execution Tier (Light/Standard/Deep)**: auto-scales research/discovery/review depth so small specs skip full-pipeline overhead; recorded in `design_context.execution_tier`. Quality floor never skips.
- **Evidence-gated red-team**: findings without a concrete task/section citation are auto-rejected.
- **Definition of a Complete Task (DoCT)**: explicit quality bar mapping each completeness element to its enforcing mechanism.
- **Frontend Fidelity Rule**: FE tasks with a provided visual reference must reproduce it faithfully (concrete hex/font/spacing/verbatim text + `match <reference>` constraint). Conditional — brownfield reuse unaffected.
- **Complexity Smell Check (by numbers)**: quantitative YAGNI tripwires (>8 files / >2 services / >12 tasks → challenge; >15 → split).

### Changed
- **Literal `R{N}.{M}` is the default requirement format** (template + Step 5), waking per-criterion coverage that was previously dodged via bare numbered lists.

### Notes
- All new validator/grounding checks are opt-in/progressive or active-grep — legacy specs are unaffected and continue to pass. Spec structure (spec.json + requirements/design/research/tasks) unchanged.

## [0.13.0] - 2026-06-18

### Changed
Expand Down
2 changes: 1 addition & 1 deletion packages/spec/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@haposoft/cafekit",
"version": "0.13.0",
"version": "0.13.2",
"description": "Claude Code-first spec-driven workflow for AI coding assistants. Bundles CafeKit hapo: skills, runtime hooks, agents, and installer scaffolding.",
"author": "Haposoft <nghialt@haposoft.com>",
"license": "MIT",
Expand Down
9 changes: 6 additions & 3 deletions packages/spec/scripts/run-skill-self-tests.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,9 @@ async function runStaticSemanticTests() {
file: "src/claude/skills/specs/SKILL.md",
assert: (content) =>
content.includes("**MUST run deterministic validator.**") &&
content.includes("Script failure overrides any LLM checklist result") &&
// Substring is invariant to "Script failure" (single-layer) vs the
// Specs-v2 "Either script failing" (validator + grounding) wording.
content.includes("overrides any LLM checklist result") &&
content.includes("output MUST NOT suggest `/hapo:develop`"),
},
{
Expand All @@ -268,9 +270,10 @@ async function runStaticSemanticTests() {
{
label: "hapo:specs feature-description flow continues past init",
file: "src/claude/skills/specs/SKILL.md",
// Specs-v2 reworded this invariant; assert the current phrasing that
// still guarantees Init is not a stop point.
assert: (content) =>
content.includes("After user confirms scope, continue through Init") &&
content.includes("Do not stop after Init unless the user explicitly asks for init-only behavior."),
content.includes("Init is never a stop point"),
},
{
label: "legacy spec-init redirects to hapo specs resume",
Expand Down
91 changes: 91 additions & 0 deletions packages/spec/src/claude/hooks/task-scaffold-guard.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/usr/bin/env node
/**
* Copyright (c) 2026 Haposoft. MIT License.
*
* PreToolUse Hook — task-scaffold-guard.cjs
* Implements: https://docs.anthropic.com/en/docs/claude-code/hooks
*
* Forces spec task files to be GENERATED via spec-scaffold.cjs instead of
* hand-written. Field tests showed the model dodges the (opt-in) scaffold step
* and raw-`Write`s every task file — the single biggest output-token cost of a
* spec run. This hook blocks `Write` to `specs/<feature>/tasks/task-*.md`, so
* the only path to a task file is: scaffold (creates the stub) -> Edit (fills it).
*
* Scope is deliberately narrow — it only ever blocks the `Write` tool on a task
* file path. `Edit`/`MultiEdit` (filling a stub), and `Write` to any other file
* (requirements.md, design.md, source code) are never touched. The scaffold
* script itself writes via Node fs through a Bash call, not the `Write` tool, so
* it is never blocked.
*
* Safety valves:
* 1. fail-open when spec-scaffold.cjs is absent — you cannot force a tool that
* is not installed, so a hook shipped without its script must not deadlock.
* 2. actionable block message — prints the exact scaffold command to run.
* 3. escape hatch — "spec": { "scaffold_guard": false } in .claude/runtime.json.
*
* Exit: 0 = allow, 2 = block.
*/

try {
const fs = require('fs');
const path = require('path');

const stdin = fs.readFileSync(0, 'utf8').trim();
if (!stdin) process.exit(0);

const data = JSON.parse(stdin);
const toolName = data.tool_name || '';
const toolInput = data.tool_input || {};
const cwd = data.cwd || process.cwd();

// Only `Write` can create a file from scratch. Edit/MultiEdit require an
// existing file, so filling a scaffolded stub is always allowed.
if (toolName !== 'Write') process.exit(0);

const filePath = toolInput.file_path || toolInput.path || '';
if (!filePath) process.exit(0);

// Match a spec task file: .../specs/<feature>/tasks/task-<...>.md
const norm = filePath.replace(/\\/g, '/');
const isTaskFile = /(^|\/)specs\/[^/]+\/tasks\/task-[^/]+\.md$/.test(norm);
if (!isTaskFile) process.exit(0);

// Valve 3: explicit escape hatch via .claude/runtime.json (fail-closed: a
// missing/broken runtime.json keeps the guard ON).
let runtime = {};
try {
const rp = path.join(cwd, '.claude', 'runtime.json');
if (fs.existsSync(rp)) runtime = JSON.parse(fs.readFileSync(rp, 'utf8'));
} catch { /* malformed runtime.json -> keep guard on */ }
if (runtime.spec && runtime.spec.scaffold_guard === false) process.exit(0);

// Valve 1: fail-open if the scaffold script is not installed next to this hook
// (hook lives in .claude/hooks/, script in .claude/scripts/). Forcing a tool
// that does not exist would deadlock task creation.
const scaffold = path.join(__dirname, '..', 'scripts', 'spec-scaffold.cjs');
if (!fs.existsSync(scaffold)) process.exit(0);

// Valve 2: block with an actionable message carrying the exact command.
const m = norm.match(/(^|\/)specs\/([^/]+)\/tasks\//);
const feature = m ? m[2] : '<feature>';
console.log(
`TASK SCAFFOLD REQUIRED: task files must be generated, not hand-written.\n` +
`Blocked Write: ${filePath}\n\n` +
`Generate the stub(s), then Edit-fill the {{...}} placeholders:\n` +
` node .claude/scripts/spec-scaffold.cjs ${feature} --tasks "R0-01-slug,R1-01-slug,..." --tasks-only\n` +
`Then use Edit (not Write) on each tasks/task-*.md stub.\n` +
`Override: set "spec": { "scaffold_guard": false } in .claude/runtime.json`
);
process.exit(2);

} catch (e) {
// Never let a hook crash block the user — log and fail-open.
try {
const fs = require('fs'), p = require('path');
const d = p.join(__dirname, '.logs');
if (!fs.existsSync(d)) fs.mkdirSync(d, { recursive: true });
fs.appendFileSync(p.join(d, 'hook-log.jsonl'),
JSON.stringify({ ts: new Date().toISOString(), hook: 'task-scaffold-guard', status: 'crash', error: e.message }) + '\n');
} catch (_) {}
process.exit(0);
}
Loading