Skip to content

fix(lpc-engine): 16-bit gamma — remove the 8-bit choke in the 16-bit pipeline - #252

Merged
Yona-Appletree merged 2 commits into
mainfrom
claude/bold-rhodes-82a88f
Aug 1, 2026
Merged

fix(lpc-engine): 16-bit gamma — remove the 8-bit choke in the 16-bit pipeline#252
Yona-Appletree merged 2 commits into
mainfrom
claude/bold-rhodes-82a88f

Conversation

@Yona-Appletree

Copy link
Copy Markdown
Member

The defect

The render pipeline is unorm16 end to end — shader read_sample_out returns Vec<u16> → brightness in u16 → gamma → power limit in u16 → Unorm16 control product → DisplayPipeline ("16-bit in, 8-bit out", interpolation + temporal dithering at the wire) — but the gamma step truncated to 8 bits internally:

r = apply_gamma((r >> 8) as u8).to_q32().to_u16_saturating();

apply_gamma indexed the canonical Adafruit GAMMA8: [u8; 256] table (exactly round(255·(i/255)^2.8)), a survivor from before the pipeline went 16-bit. Measured on 2026-08-01:

  • only 163 distinct output levels survive the u8 round-trip
  • the bottom 28/256 of input collapses to hard 0
  • at fixture brightness 38 — the real desk project quad-strips-v3 — full white lands on GAMMA8[38]=1, so the whole project's post-gamma range was {0, 257}: binary pixels
  • the 16-bit dithering downstream cannot recover information gamma already destroyed

Full write-up: docs/defects/2026-08-01-gamma-8bit-choke.md.

The fix

apply_gamma16(u16) -> u16 in lpc-engine/src/nodes/fixture/gamma.rs: a const LUT evaluated by linear interpolation, the same shape as the white-point LUT in display_pipeline/lut.rs (index = high bits, alpha = low bits, final entry one segment step past the domain so the top lands exactly without an edge clamp). γ=2.8 kept for visual continuity with the legacy table. Shared .rodata, no heap, no per-channel storage. All three call sites in fixture_node.rs updated; ordering untouched (brightness → gamma → power scale → color order — power scale after gamma, see power_limit).

One deliberate deviation from the agreed sketch: the table is [u32; 513] with 2 fractional bits (~2 KB) instead of [u16; 257] (~514 B). The 257×u16 shape cannot meet its own accuracy bar: the top table node sits at input 65536 whose true curve value is ~65537.8 — doesn't fit u16, and clamping sags the whole top segment ~3 counts; separately, 256-segment chord error (0.63) plus node and output rounding measures ~1.4 counts. With 512 segments + fractional bits, measured max error is 0.75 counts in 65535, monotone, endpoints exact. u32 entries also match the white-point LUT's shape exactly.

Tests

  • exact regeneration of the checked-in literal from the formula (hand-edits impossible)
  • full-domain oracle vs libm pow: max abs error ≤ 1 count across all 65536 inputs
  • monotone non-decreasing, exact endpoints (0→0, 65535→65535)
  • regression for the motivating case: brightness 38 keeps ≥250 distinct post-gamma levels (measured 318; old path produced 2)

Baseline audit

This intentionally changes output for any fixture with gamma_correction=true (engine default when unspecified). A thorough sweep found no checked-in expected values need regeneration: every harness asserting numeric channel output pins gamma off (engine test helpers bind gamma_correction=false; ProjectBuilder::fixture() hard-codes Some(false); shader-oracle's examples/shader-oracle/fixture.json sets false and stores no golden bytes; the lp-ws281x goldens encode a fixed byte array that never touches the engine; story PNGs never render through the engine gamma path; filetests are shader-compilation only). The S3 device-vs-host comparison computes both sides live, so it shifts together and stays bit-exact. The one gamma-on artifact in the tree is examples/fast/fixture.json, a manual demo not referenced by any test.

Relation to D7

Independent of D7 (whether to remove DisplayPipeline's per-channel white-point tables — still open with Yona in the classic-ESP32 bring-up plan). This PR doesn't touch DisplayPipeline, Esp32OutputProvider, or anything on PR #239. It does strengthen D7 option A: gamma turns out to want a const shared table, not the per-channel heap machinery.

🤖 Generated with Claude Code

Yona-Appletree and others added 2 commits August 1, 2026 11:34
…pipeline

The pipeline is unorm16 end to end, but the gamma step truncated to 8
bits internally via the legacy Adafruit GAMMA8 table: only 163 distinct
output levels survived, the bottom 28/256 of input collapsed to hard 0,
and at fixture brightness 38 (the real desk project quad-strips-v3) the
whole post-gamma range was {0, 257} — binary pixels. The 16-bit temporal
dithering downstream cannot recover what gamma already destroyed.

Replace it with apply_gamma16(u16) -> u16: a 513-entry const LUT
([u32; 513], 16.2 fixed point, ~2 KB .rodata, shared, no heap) evaluated
by linear interpolation — the same shape as the white-point LUT in
display_pipeline/lut.rs. γ=2.8 kept for visual continuity. Max error vs
the analytic curve is 0.75 counts in 65535, monotone, exact endpoints —
all asserted over the full 65536-input domain, plus an exact
regeneration test on the checked-in literal and a regression test for
the brightness-38 case (318 distinct levels, was 2).

Ordering unchanged: brightness → gamma → power scale → color order.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Yona-Appletree
Yona-Appletree merged commit 54d8b8d into main Aug 1, 2026
10 checks passed
Yona-Appletree added a commit that referenced this pull request Aug 1, 2026
… regression

Merged origin/main (brings PR #252's 16-bit gamma) and re-measured on the
desk DOM-Z-102. Two findings, only one of them expected.

**16-bit gamma is free on this chip.** Isolated by flipping
gamma_correction on identical firmware, same board, same project:

    gamma on   fps 19  tick 49ms  free 6,268 B
    gamma off  fps 20  tick 48ms  free 6,264 B

4 bytes of heap and ~1 fps — noise. GAMMA16 is a const in .rodata, so it
costs flash (image 1,720,448 B, still 45% headroom) and no heap. Added
projects/test/quad-gamma-v3 (quad-strips-v3 with gamma on at brightness
38 — the defect's own motivating case) as the hardware fixture for it.

**Main regressed per-project heap by 8,136 B**, which is the finding
worth acting on. Same project, clean boot, before vs after the merge:
18,128 B free -> 9,992 B, while tick dropped 69ms -> 47ms. Idle heap is
unchanged (102,156 -> 102,144), so it is per-project allocation, not
static growth — faster AND fatter, the signature of caching. quad60-v3
(240 LEDs) ran with 7,384 B spare before and now OOMs at a 360-byte
alloc.

At ~89.5 B/LED that is ~91 LEDs of capacity, roughly a third of this
chip's usable budget. Gamma is ruled out by the measurement above; the
three f32 PRs (#249/#251/#253) are the candidates. NOT bisected — each
point needs a build+flash+upload cycle and the test projects do not exist
on main, so it is left for whoever owns that work rather than guessed at.

Recorded in docs/defects/2026-08-01-classic-heap-regression-after-f32-merge.md,
indexed (along with the RMT open fault, which was never indexed), and the
M6 ADR now carries a warning that its absolute ceiling is optimistic by
~91 LEDs until this is resolved. The ADR's method and conclusion stand:
heap binds, flash does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant