Claude: SRD Creator v2

.claude/skills/srd-creator/SKILL.md

@@ -10,47 +10,166 @@ style used in the existing Evaluation / Fast Evaluation / STT Evaluation SRDs.
 
 ## What an SRD is
 
-An SRD is the contract written *before* code. It states what to build, why,
-what is in/out of scope, the execution flow, the API surface, and the DB schema —
-specific enough that an engineer can build it and a reviewer can test it against
-the Functional Requirements table. It is not design prose; it is testable spec.
+An SRD is the contract written *before* code: what to build, why, what is in/out
+of scope, the execution flow, the API surface, and the DB schema, specific enough
+that an engineer can build it and a reviewer can test it against the Functional
+Requirements table. It is testable spec, not design prose.
+
+## A PRD is mandatory
+
+**The SRD is derived solely from a PRD.** If the user has not supplied a PRD, a
+file path or pasted content, stop and ask for one. Do not invent the problem,
+scope, endpoints, or schema from the conversation, the codebase, or assumption.
+No PRD → no SRD.
 
 ## Workflow
 
 1. **Read the reference material first.**
-   - `reference/srd-guide.md` — what each section means and what good content looks like.
-   - `reference/srd-template.md` — the skeleton to fill in.
-   - If unsure of house style, skim a prior SRD; the three originals live as
-     `*.docx` in the repo root.
+   - `reference/srd-guide.md`, what each section means and what good content looks like.
+   - `reference/srd-template.md`, the skeleton to fill in.
+   - If unsure of house style, skim a prior SRD; the originals live as `*.docx` in the repo root.
 
-2. **Gather the inputs.** Before writing, you need: the feature name, the problem
-   it solves, who the users are, what is in scope for Phase 1 vs later phases, the
-   endpoints, and the data model. If the user has not supplied these, ask — do not
-   invent endpoints or schema. Missing info is the #1 cause of a useless SRD.
+2. **Load the Quality Checklist into a `TodoWrite` list before writing.** Hold it
+   for the whole creation so each rule is checked once and not re-litigated per
+   section. Tick items as sections land; run a final pass against it before output.
 
-3. **Fill the template** section by section. Drop optional sections that don't
-   apply (e.g. Resources, Configuration) rather than leaving them empty.
+3. **Read the PRD and map it onto the template.** Problem → Introduction, Goals →
+   Goals, Users → Introduction/personas, scope → Assumptions & phasing, etc. Fill
+   section by section. Anything the PRD leaves open goes under *Design Decisions /
+   Known Limitations* as an open question, never fabricated. Drop optional sections
+   that don't apply (Resources, Configuration) rather than leaving them empty.
+   - **Resources links are never fabricated or guessed** (no inventing
+     `Fast Evaluation SRD.md`-style paths or PRD URLs). Ask the user for the actual
+     Google Docs / PRD / related-SRD links; if they have none, drop the section.
 
 4. **Write the Functional Requirements table** as the testable core. Every row =
    one user-facing behavior + a concrete acceptance criterion + a Status. If you
-   can't write an acceptance criterion for a requirement, the requirement is too
-   vague — sharpen it.
+   can't write a checkable acceptance criterion, the requirement is too vague;
+   sharpen it.
 
 5. **Output** as `features/<feature-slug>/SRD.md`, where `<feature-slug>` is a
    short kebab-case slug for the feature (e.g. `features/llm-judge-correctness/SRD.md`).
    Create the `features/<feature-slug>/` directory if it doesn't exist. Every feature
-   keeps its SRD and PRD together in this one folder — if a `PRD.md` already exists
+   keeps its SRD and PRD together in this one folder, if a `PRD.md` already exists
    for the same feature (written by the `start-prd` skill), write the `SRD.md`
    alongside it in the same folder rather than creating a new location. Reuse the
-   existing slug exactly; do not invent a parallel folder.
+   existing slug exactly; do not invent a parallel folder. The H1 inside the file is
+   the feature's display name (e.g. `# Native LLM as Judge Correctness Score SRD`).
+
+6. **Generate exactly one image: the primary execution-flow diagram. Render it into
+   `assets/`, do not embed it inline, and leave a placeholder band naming the file.**
+   The author exports the `.md` to Google Docs and manually pastes the image at its
+   band, so the band's job is to say *which* file goes *where*.
+   - **One image only.** The single SRD image is the main execution flow. Everything
+     else is words or tables, no second image: a second flow (e.g. plain config CRUD)
+     is prose plus the Endpoints section, and the **DB schema is always column tables,
+     never a diagram**.
+   - **Render it.** Write a mermaid source for the execution flow, then run the
+     skill's helper script (high-res mermaid render to a crisp png; no global
+     installs, it uses `npx`):
+     ```bash
+     scripts/render-diagram.sh flow-a.mmd features/<feature-slug>/assets/flow-a.png
+     ```
+   - **Placeholder band in the `.md`.** A horizontal-rule-fenced block (renders as a
+     distinct band in GitHub *and* after a Google Docs markdown import, which turns
+     `---` into a real line) naming the asset file:
+     ```markdown
+     ---
+
+     **>> PLACE IMAGE HERE: `assets/flow-a.png`, judged fast-eval run.**
+     System-level sequence: `User`, `Kaapi Backend`, `OpenAI`, `Langfuse`.
+
+     ---
+     ```
+   - **No HTML tags and no inline CSS** (e.g. no `<div style=...>`), the Google Docs
+     markdown importer strips them, so colored callouts do not survive. Plain
+     markdown only. **No emoji.** Do not use `![](...)` inline embeds (a relative
+     image path does not resolve after the Google Docs import; the band is the cue).
+
+## Quality Checklist
+
+Load these as todos at the start; verify each before output:
+
+- [ ] PRD supplied and read; every SRD claim traces back to the PRD.
+- [ ] Output at `features/<feature-slug>/SRD.md`, alongside the feature's `PRD.md`
+      (reuse the existing slug; don't create a parallel folder).
+- [ ] All required sections present (Introduction, Goals, Assumptions, Detailed
+      Design, Functional Requirements; Endpoints if the feature has an API; DB
+      Schema if there's a data model).
+- [ ] **No fact stated more than once**, each fact lives in one home section;
+      elsewhere reference it, don't restate it.
+- [ ] No filler, hedging, or restated bullets; every sentence adds new information.
+- [ ] Phasing explicit (Phase 1 vs Phase 2+).
+- [ ] Every FR row has a concrete, testable acceptance criterion + Status.
+- [ ] Every endpoint shows real request/response JSON, not just field lists.
+- [ ] DB schema follows Kaapi conventions (`inserted_at`/`updated_at`,
+      `organization_id` + `project_id`, snake_case, FK indexes).
+- [ ] High-level only, no function names, internal variables, or step internals
+      (DB schema, class/endpoint contracts, error codes, settings are fine).
+- [ ] Existing models reused, not duplicated, checked the codebase first; any new
+      shape is justified by no existing model fitting.
+- [ ] Exactly one image (the execution flow) rendered to
+      `features/<feature-slug>/assets/flow-a.png`, with a horizontal-rule-fenced band
+      naming it (no inline `![](...)` embed, no mermaid blocks, no HTML, no emoji).
+- [ ] DB schema is column tables, not a diagram; no second image anywhere.
+- [ ] Optional empty sections deleted, not left as placeholders.
 
 ## Rules
 
+- **No redundancy.** State each fact exactly once, in its home section. Later
+  sections reference it ("as in Goals"), they don't repeat it. Goals are outcomes,
+  not echoes of the Introduction. Assumptions are boundaries, not re-listed goals.
+  FR acceptance criteria are checkable conditions, not restatements of the behavior
+  column. Repetition across sections is the #1 cause of bloated SRDs.
+- **Keep it high-level. No variable/function-level detail.** An SRD describes
+  behavior and interfaces, not implementation internals. Allowed: DB schema (tables,
+  columns, constraints), object/class and endpoint contracts, error codes, settings.
+  Not allowed: specific function names, internal variable/field names, private
+  helpers, stage/step internals, concurrency-pool names, the reader sees those in
+  PR review. Name *what* a thing does and *which entity* holds it, not the symbol
+  that implements it (e.g. "the run's per-row score storage", not `per_item_scores`).
+- **Reuse existing models, don't invent new ones.** Before specifying any config,
+  request body, or pydantic/SQLModel class, search the codebase for an existing one
+  that fits (e.g. `TextLLMParams` for model + sampling + prompt). Default order:
+  (1) reuse as-is, (2) extend/compose the existing one, (3) only if nothing fits,
+  add a new shape, and call out explicitly why no existing model worked. Never
+  introduce a parallel config that duplicates an existing one. A reviewer asking
+  "can't we reuse X?" is a failure of this rule.
+- **Diagrams over prose for flows, but the SRD only holds a placeholder.** Both the
+  execution flow and the data model want a diagram, not a long numbered paragraph.
+  The skill does not draw it; it leaves an author image placeholder note (Workflow
+  step 6) and keeps the surrounding prose to what a diagram can't carry (failure
+  isolation, idempotency, resolution rules). Tell the author what the diagram should
+  depict so they can draw it:
+  - **Execution flow at system level, not internals.** The diagram should show the
+    user and the real systems that talk to each other (e.g. `User`, `Kaapi Backend`,
+    `OpenAI`, `Langfuse`), each with arrows in and out. It should not turn internal
+    pipeline steps or in-process helpers into separate lanes (e.g. a "Judge" lane
+    that is really an OpenAI call), which misleads readers into thinking they're
+    separate services.
+  - **Data model** should mark reused vs new entities.
+- **No em dashes.** Use commas, periods, or parentheses. Keep prose clean and
+  developer-readable; short sentences over long dash-joined clauses.
+- **Match existing system naming.** Column names, table names, request/response field
+  names, and class names must follow the codebase's conventions and reuse existing
+  names where one exists (snake_case columns, the same field names as the reused
+  model, e.g. `llm_params` carrying `TextLLMParams`). The schema must not read like it
+  came from a different system. Grep for an existing name before coining a new one.
 - Match Kaapi conventions: `inserted_at`/`updated_at` timestamps (not `created_at`),
   `organization_id` + `project_id` on every multi-tenant table, snake_case columns.
 - Phase the scope explicitly. State Phase 1 (build now) vs Phase 2+ (later) so
   scope creep is visible.
+- **Functional Requirements are PR-testable functional behaviors only.** Every FR row
+  is something a developer or client can verify works when reviewing the PR before it
+  ships. Drop rows that merely restate the Intro/Goals, describe internal mechanics,
+  or can't be checked against the running system. Lean beats exhaustive.
 - Show real request/response JSON for every endpoint, not just field lists.
+- **Error responses add value, don't restate the obvious.** Standard codes (409, 422,
+  404) are self-explanatory; the value is the readable client-facing message. Show the
+  actual message string (or a field-specific example for validation), not a paraphrase
+  of the status code.
 - DB schema as a table: Column / Type / Nullable / Default / Description, plus a
   Constraints list (unique keys, FKs, indexes).
+- Keep the reference style: consistent table headers, one blank line between
+  sections.
 - Don't pad. Each section earns its place; cut what doesn't apply.

.claude/skills/srd-creator/reference/srd-guide.md

@@ -1,20 +1,29 @@
-# SRD Guide — what each section is and how to write it
+# SRD Guide, what each section is and how to write it
 
-An SRD (Software Requirements Document) is written before implementation. It is the
-shared contract between whoever requested the feature and whoever builds it. Good
-SRDs are *testable*: a reviewer can take the Functional Requirements table and check
-each row against the running system.
+An SRD (Software Requirements Document) is the testable contract between whoever
+requested the feature and whoever builds it, written before implementation: a
+reviewer can take the Functional Requirements table and check each row against the
+running system.
 
 This guide describes each section, derived from the Kaapi Evaluation, Fast Evaluation,
 and STT Evaluation SRDs. Use `srd-template.md` as the fill-in skeleton.
 
+**The SRD is derived solely from a PRD**, no PRD, no SRD. Map PRD sections onto the
+template; record anything the PRD leaves open as an open question under *Design
+Decisions / Known Limitations*, never as invented detail.
+
+**No redundancy.** State each fact exactly once, in its home section; later sections
+reference it instead of repeating it. Goals are outcomes, not echoes of the
+Introduction. Assumptions are boundaries, not re-listed goals. FR acceptance criteria
+are checkable conditions, not restatements of the behavior column.
+
 ---
 
 ## 1. Introduction & Purpose  *(required)*
 
 The "what and why" in a few short paragraphs.
 - One sentence: what capability this SRD defines and for which system.
-- The problem / motivation — what's painful today, who feels it (name the early
+- The problem / motivation, what's painful today, who feels it (name the early
   users if known, e.g. "early users from Glific").
 - What the feature produces at minimum (the concrete outputs).
 - Explicit phasing: what is Phase 1, what is deferred to Phase 2/3. This is where
@@ -23,16 +32,17 @@ The "what and why" in a few short paragraphs.
 
 ## 2. Resources  *(optional)*
 
-Links to related SRDs, external API docs, research notes, design docs. Drop the
-section if there's nothing to link.
+Links to related SRDs, external API docs, research notes, design docs. **Ask the
+user for the real links** (Google Docs, PRD, related SRDs), never fabricate or
+guess paths/URLs. Drop the section if the user has nothing to link.
 
 ## 3. Goals  *(required)*
 
 A short bulleted list of what success looks like. Each goal is an outcome, not a
 task. Keep it to the handful that actually define done. Examples:
 - "Add a `run_mode="fast"` option so users can run a text evaluation synchronously."
 - "Identical scoring semantics to the batch path."
-- "Failure isolation — one item's failure must not fail the whole run."
+- "Failure isolation, one item's failure must not fail the whole run."
 
 ## 4. Assumptions & Constraints  *(required)*
 
@@ -47,16 +57,20 @@ Cover:
 
 ## 5. Detailed Design (Execution Flow)  *(required)*
 
-The step-by-step of how it actually runs. Number the steps in order. If there are
-multiple flows (upload flow, run flow, async/polling flow, feedback flow), give each
-its own numbered subsection. For multi-stage async work, describe each stage, its
-skip/idempotency marker, and its retry behavior. Mention the sequence diagram if one
-exists (the originals reference a "Sequence Flow Diagram" placeholder).
+How it runs, with brief supporting text, not a long numbered paragraph. Render each
+flow's diagram to a png in `assets/`, but do **not** embed it inline: where each
+diagram belongs, leave a horizontal-rule-fenced band naming the asset file the author
+pastes there (no mermaid, no HTML, no emoji). **Exactly one image per SRD: the primary
+execution flow.** A second flow (e.g. plain config CRUD) is prose plus the Endpoints
+section, not another image, and the DB schema is always column tables, never a diagram.
+Keep the text to what a diagram can't carry: failure isolation, idempotency,
+resolution rules.
 
-Good content here names the actual mechanics: which API is called, what's written to
-S3, which Celery task/queue runs it, what marks a stage complete, how retries dedupe.
+Stay high-level, actors and behavior ("Pipeline → Judge: question + answer",
+"persist both scores"), not internal function or variable names. Pipeline *stages*
+named by behavior are fine; private helpers and field names are not.
 
-## 6. Functional Requirements (Testing)  *(required — the core)*
+## 6. Functional Requirements (Testing)  *(required, the core)*
 
 A table, one row per user-facing behavior. Columns:
 
@@ -69,7 +83,7 @@ A table, one row per user-facing behavior. Columns:
   `dataset_too_large_for_fast` error and the actual unique-row count").
 - **Status**: `Not Started` / `In Progress` / `Done`.
 
-If you cannot write a crisp acceptance criterion, the requirement is too vague —
+If you cannot write a crisp acceptance criterion, the requirement is too vague;
 split or sharpen it. This table is what QA and review run against.
 
 ## 7. Endpoints  *(required when the feature has an API)*
@@ -88,7 +102,8 @@ field").
 
 ## 8. Database Schema / Tables  *(required when there's a data model)*
 
-For each table, a column table:
+Always column tables, never a diagram or image (the one SRD image is the execution
+flow). For each table, a column table:
 
 | Column | Type | Nullable | Default | Description |
 
@@ -98,7 +113,7 @@ Then a **Constraints** list: primary key, unique constraints (name them, e.g.
 Kaapi conventions to enforce:
 - `id INTEGER PK` auto-increment (or UUID where the domain calls for it).
 - `organization_id` + `project_id` on every multi-tenant table.
-- `inserted_at` / `updated_at TIMESTAMP NOT NULL DEFAULT now()` — **not** `created_at`.
+- `inserted_at` / `updated_at TIMESTAMP NOT NULL DEFAULT now()`, **not** `created_at`.
 - Filterable data as first-class columns; bag-of-attributes as `JSONB`.
 - Prefer reusing existing tables; if so, state "No new tables" and list only the
   added columns / constraints, with the backfill plan for new non-null columns.
@@ -121,3 +136,6 @@ so the next reader doesn't re-litigate it.
 Required: Introduction & Purpose · Goals · Assumptions & Constraints ·
 Detailed Design · Functional Requirements · Endpoints (if API) · DB Schema (if data).
 Optional: Resources · Configuration · Design Decisions / Known Limitations.
+
+Before output, verify the full **Quality Checklist** in `SKILL.md`, PRD-traceable,
+correctly named, no repeated facts, testable FRs, Kaapi DB conventions.