docs: add per-package module maps and fix three stale facts#33
Merged
Conversation
The docs/ tree explains how the system works; what was missing was the code map — where things live, what to reuse, what not to do — next to the code itself. Adds an index.md to every package (nine: the pipeline stack and its five packages, answer, evals, benchmark), indexed from docs/README.md. Each map covers: purpose, entry points, module map, "use these" (the shared primitives, so the next contributor doesn't write a second one), anti-patterns with the ADR that motivates them, data contracts, where the tests are, and a common-tasks table. Three factual drifts between docs and code, fixed: - README claimed ~390 tests; the suite is 421 (fetch 29, clean 248, corpus 48, graph 38, slack 13, answer 42, benchmark 3) -> ~420. - docs/architecture.md's stage table omitted slack, though the README and docs/pipeline/slack.md both cover it. - The README's "5 minutes" table displayed paths like `clean/src/verify.py` that do not exist (the hrefs were right, the visible text wasn't) — a reader copying them gets nothing. Also recorded, because it will bite whoever first evals a real model: the golden scorecard is 24 metrics offline but 23 with a live backend — eval_clean_and_trust swaps its two seeded-defect metrics for one when CLEAN_LLM is not fake-flawed, since there is nothing seeded to catch. Docs only; no code touched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/explains how the system works; what was missing was the code map — where things live, what to reuse, what not to do — sitting next to the code. This adds anindex.mdto every package (nine: the pipeline stack + its five packages,answer,evals,benchmark), indexed fromdocs/README.md. Written by the documentator agent; every factual claim in this PR was verified against the code before merging.Changes
~390 tests; the suite is 421 (fetch 29, clean 248, corpus 48, graph 38, slack 13, answer 42, benchmark 3) →~420.docs/architecture.md's stage table omitted slack, though the README anddocs/pipeline/slack.mdboth cover it.clean/src/verify.pythat don't exist — the hrefs were correct, the visible text wasn't, so anyone copying them got nothing.eval_clean_and_trustswaps its two seeded-defect metrics for one whenCLEAN_LLMisn'tfake-flawed, since there's nothing seeded to catch. Whoever first evals a real model would otherwise think a metric vanished.Testing
Docs only — no code touched. Claims spot-verified against the source: the
make eval→run-evals.sh→evals/.venvchain,service.refresh()being called perask_brain/search_brainrequest,index.superseded_pathsexisting, and the per-package test counts (run locally, all suites green).Notes
Deliberately not done, flagged for your call:
CLAUDE.md— the biggest remaining gap for agent onboarding (package layout,maketargets, per-package venvs, the bare-pytestpythonpathgotcha, single-writer-per-artifact, never read.env). It's agent configuration rather than project documentation, so it shouldn't land without you deciding you want it.make test-pkg PKG=...shortcut target — a Makefile change, not a docs one.facts.dbschema in one place — currently implicit inclean/factstore.pyand re-read byanswer/metrics.py, with parity enforced only byeval_contract_parity.