board: E-OCR-CTC-SIMPLETEXT-1 + E-OCR-P6-GOLDEN-1 — the CTC correction + P6 golden suite shipped

.claude/board/EPIPHANIES.md

@@ -1,3 +1,23 @@
+## 2026-07-08 — E-OCR-P6-GOLDEN-1 — P6 ships: hermetic golden corpus + tiered parity harness + perf report + CI — the plan's final phase, and the corpus paid for itself on first contact (it surfaced E-OCR-CTC-SIMPLETEXT-1)
+**Status:** FINDING (shipped tesseract-rs `7dc8dea`+`3f8c445`+`df4a641`+`16b5ba6`; PR #20)
+
+**D6.1 hermetic corpus:** the six eng model components committed (Apache-2.0 tessdata provenance + SHA256s) so the golden suite needs NO network / C++ tesseract / combine_tessdata in CI; the arc's proven line fixtures (incl. the roomy/tight stacked layouts — the position-invariance anchor is now a hermetic unit test); 10 deterministic rendered pages (DejaVu ×3 faces, 70 authored ASCII lines, seeded-noise arm; generator double-run byte-determinism proven) + 5 hand-authored PDF-1.4 files (two-page + no-text-layer arms, xref self-checked); all ground truth authored in-repo — license-clean by construction.
+
+**D6.2 tier discipline (the plan's "NEVER blur which is which", operationalized):** Tier A = hard byte gates vs OWN goldens (`golden_lines` 4/4, `golden_pages` 2/2 incl. determinism, `golden_pdfs` 2/2; `UPDATE_GOLDEN=1` refresh; goldens bake the CORRECT CTC decode). Tier B = CLI comparison as CER/WER REPORT ONLY with the untranscoded framing named per row — pages mean CER ours-vs-gt **0.0153** (CLI 0.0016; page_06 = 0.0000; residual classes: trailing-period drops + the `Acool` space-drop the CLI makes identically), lines psm13 with img_8/img_16 byte-equal to the CLI. Tier C = perf report, no gate (ours 1.06 pages/s release / VmHWM 20.4 MB vs CLI 5.38 pages/s / 36.7 MB child RSS; harness-shape difference documented; a contention-contaminated first run documented and DISCARDED — idle-machine rule in the banked doc).
+
+**The meta-finding:** the corpus's very first rendered page exposed the arc's biggest correctness bug (E-OCR-CTC-SIMPLETEXT-1, one entry down) — after ~40 byte-parity-green leaves on noise fixtures. Rendered-text fixtures falsify DECODE SEMANTICS; ramps only falsify data paths. P6's golden suite is now the standing guard against that whole bug class. CI (D6.4): scoped build/test/clippy/fmt for tesseract-ocr/-pdf/-ogar wired additively (pages golden ≈10 min debug — flagged for a future release-mode golden job, not gated today). Artifacts: `tesseract-rs/.claude/harvest/p6-parity-report.md` + `p6-perf-report.md`. **The pdf-to-text-ocr-v1 plan is COMPLETE through P6.** Branch `claude/happy-hamilton-0azlw4`.
+
+## 2026-07-08 — E-OCR-CTC-SIMPLETEXT-1 — the SimpleTextOutput/LT_CTC correction: every recognizer text anchor was decoded with the WRONG beam mode — found by the P6 corpus smoke, pinned by a 9-stage live bisect, re-anchored 8/8
+**Status:** FINDING (root-caused, fixed, re-anchored byte-identical vs the corrected libtesseract oracle; `tesseract-ocr`, tested)
+
+**The bug (one boolean, arc-wide blast radius):** eng.lstm's head is `O1c111` — `NT_SOFTMAX` = softmax **activation** with **CTC loss** (`fullyconnected.cpp:47-58` maps `NT_SOFTMAX → LT_CTC`, `NT_SOFTMAX_NO_CTC → LT_SOFTMAX`), so the real `LSTMRecognizer::SimpleTextOutput()` (`lstmrecognizer.h:84-86`: `OutputLossType() == LT_SOFTMAX`) is **FALSE** and `RecodeBeamSearch` must run full CTC semantics (top-n flags, duplicate collapse). The A6b-era oracle hardcoded `simple_text = true` with a comment claiming "OutputLossType()==LT_SOFTMAX for eng.lstm" — the activation/loss conflation — and the Rust transcode mirrored it. Every downstream "byte-parity green" (A6b image→text, 7b beam configs, B3-core, C1 dict, B3-full words, pixScale-height sweeps) carried the SAME wrong flag on BOTH sides of the diff, so oracle==rust stayed green while both diverged from real Tesseract. **On noise fixtures the bug was UNFALSIFIABLE** — gibberish in, "plausible" gibberish out; the P6 corpus's first RENDERED TEXT page exposed it instantly as systematic character repeats (`The old clock` → `TTTThhheee ooolllldd cccclllooocccckk`, the signature of re-emitting every per-timestep spike).
+
+**The bisect (how it was pinned, transferable method):** CLI clean vs ours repeating on identical input → eliminated in order: scaling (identity-scale band still repeats; H-sweep 36..150 never cleans), binarization, model file (sha256 equal to system tessdata), fixture polarity/inversion (inverted = garbage), randomizer seed conventions (3 variants, repeats stable), missing dawgs (component inventory identical, 3 dawgs), ABI skew (rebuilt oracle against true 5.3.4 worktree headers — same repeats), dict beam + `ExtractBestPathAsWords` with the REAL `Dict` via `TessBaseAPI::Init` (still repeats). Live gdb on the CLI: `PreparePixInput` pix **byte-identical** to our input (pixWrite dump + compare), `StaticShape{1,36,0,1}` identical, ONE `pixScale` (f=1.0), production Decode params captured live (`dict_ratio=2.25`, `cert_offset=-0.085`, `worst_dict_cert=-25/7`, `lstm_choice_mode=0`), logits **identical** (full-matrix sum + per-timestep ARGMAX fingerprint equal). The argmax stream itself (`110,110,19,19,110,95,95,110,…` — label PAIRS between nulls) is textbook CTC emission; flipping `simple_text=false` in the oracle reproduced the CLI byte-for-byte, dict and non-dict.
+
+**The fix:** `Network::simple_text_output()` (tesseract-rs `network.rs`) derives the flag faithfully from the loaded tree — the `OutputShape` loss walk: final output-producing node decides; FC family maps `SoftmaxNoCtc → LT_SOFTMAX` (simple), `Softmax → LT_CTC` (not simple); `Series` → last child, wrappers → child. All three beam call sites consume it. **Re-anchored byte-identical vs the corrected oracle 8/8** (6 ramp fixtures + 2 real-text bands, non-dict), dict path re-anchored on real text (`The old clock ticked all night.` == real-Dict oracle == our DictLite) and on `line36` (`i,` == the CLI — closing the earlier `Ly,`-vs-`i,` discrepancy, same root cause). New ramp anchors: `img_24 → "y,"` (the old `qLLiy,,` was bug inflation). Full-page E2E through the makerow+typographic-feeding path now reads the rendered corpus pages at near-ground-truth (e.g. page_01: 7/7 lines, errors limited to a missing period class the CLI shares, incl. the identical `Acool` space-drop).
+
+**Transferable lessons:** (1) decode-SEMANTICS bugs need TEXT falsifiers — ramp/noise fixtures prove data-path parity but cannot falsify emission semantics; every future beam/LM leaf gets a rendered-text fixture in its sweep. (2) An oracle's config comment is a CLAIM, not a fact — verify flag derivations against the real call chain (`SimpleTextOutput()` is derived state, not a constant). (3) The live-CLI gdb bisect (dump pix at entry, argmax-fingerprint the logits, capture production params from registers) is the escalation path when composition-oracle parity and end-binary behavior disagree. Corrected oracle banked at `tesseract-rs/.claude/harvest/oracles/image_text_oracle_ctc.cpp` (nodict self-check arm + real-`Dict` arm). Commit tesseract-rs `50fbab5`. Branch `claude/happy-hamilton-0azlw4`.
+
 ## 2026-07-07 — E-OCR-LINEFEED-1 — the typographic line feeding lands: `LSTMRecognizeWord`'s band transcoded, position invariance PROVEN — 3F₂'s boundary #2 is closed
 **Status:** FINDING (falsifier green on the roomy stacked fixture; `tesseract-ocr`, tested)