Skip to content

fix(fw-esp32c6): give catch_unwind recovery enough stack (heap 300K -> 250K) - #187

Open
Yona-Appletree wants to merge 1 commit into
mainfrom
claude/esp32c6-unwind-stack-budget
Open

fix(fw-esp32c6): give catch_unwind recovery enough stack (heap 300K -> 250K)#187
Yona-Appletree wants to merge 1 commit into
mainfrom
claude/esp32c6-unwind-stack-budget

Conversation

@Yona-Appletree

Copy link
Copy Markdown
Member

Problem

catch_unwind-based panic/OOM recovery has been non-functional on device. With --features test_oom, Test 1 raised its panic and then unwinding tripped esp-hal's stack guard:

Detected a write to the main stack's guard value at 0x40030cf6, possibly called by 0x4223f3f4
  0x4223f3f4 - <unwinding::unwinder::frame::Frame>::from_context

Test 1 OK never printed, boot never completed, and the recovery ledger latched safe_mode=true (27 consecutive incomplete boots on my board). This also meant lpa-server's node-level panic-recovery feature was not actually recovering on hardware.

Root cause

A stack budget failure — not a toolchain, esp-hal, or linker regression.

The main stack is whatever DRAM is left after .bss, and .bss holds the esp_alloc heap, so every heap byte costs a stack byte. At heap_allocator!(size: 300_000) the stack was 32,392 B total, with only 25,976 B free at the panic site.

Unwinding a single panic needs ~41 KB, concentrated in two frames (measured from the disassembled prologues):

Symbol Frame size
<unwinding::unwinder::frame::Frame>::from_context 20,736 B
with_context::delegate::<…_Unwind_RaiseException::{closure#0}> 8,448 B

from_context builds a gimli UnwindContext on the stack per frame, and StoreOnStack rounds MAX_REG_RULES (65) up to 128 register-rule slots — twice over in [UnwindTableRow; 2], plus the cloned row. LLVM doesn't merge those slots, so one call costs 20 KB. The unwinder ran off the bottom of the stack and wrote __stack_chk_guard (at _stack_end + 60).

Everything reported downstream follows from that single overflow:

  • esp-hal's ExceptionHandler is extern "C" and therefore nounwind, so panicking inside it and unwinding hits panic_cannot_unwind (core/src/panicking.rs:225).
  • That panic re-enters esp_println's already-held esp-sync lock → the unbounded lock is not reentrant cascade.

Ruled out along the way: stack-guard-offset: 60 and stack-guard-monitoring: true are byte-identical defaults across esp-hal 1.0.0 → 1.1.0 → 1.1.1, so the 1.1.x bumps are not implicated; .cargo/config.toml (-C panic=unwind, force-unwind-tables, build-std) is unchanged from the known-good configuration in docs/reports/2026-03-13-esp32-unwinding-implementation.md. The emulator test fw-tests/tests/unwind_emu.rs passes precisely because it runs with a RAM-sized stack, which is why this never showed up off-device.

Fix

One line: heap 300_000250_000, which grows the stack to 85,784 B. The added comment records the numbers so nobody raises it back without re-testing.

Verification (on hardware, at this commit)

ESP32-C6, XIAO, MAC a0:f2:62:87:b4:8c:

[RECOVERY] boot: cause=user-reset level=green safe_mode=false prior_boot_complete=true
[test_oom] Test 1 OK: simple panic caught
[test_oom] Test 2 OK: OOM caught, recovery works
[test_oom] Tests complete, continuing boot...
Boot: auto-loaded project /projects/studio
[RECOVERY] boot complete (first frame served)
  • Instrumented stack high-water for a boot-depth panic: 47,224 B of 85,784 — ~38 KB spare for panics raised deeper in node render.
  • Heap is not tight afterwards: 142,628 B free of 250,000 with the studio project loaded.
  • Flash unaffected (the heap is .bss): image is 2,862,848 / 3,145,728 bytes, 91.01%.

Follow-ups (not in this PR)

  1. The esp-sync reentrancy guard in the panic handler is dead code. fw-esp32-common's panic_handler checks is_esp_sync_reentrant_lock_panic after four esp_println! calls, but printing is what re-takes the lock — so the recursion starts before the check can run. Hoisting the check above the prints would turn this class of failure from "bricked boot" into a clean reset.
  2. The 50 KB of heap is recoverable by patching unwinding's riscv32 MAX_REG_RULES from 65 to 32 (next_value then picks 32 slots instead of 128, shrinking from_context roughly 4×). Rust frames save ~13 registers and riscv32imac is soft-float, so DWARF regs 32–63 never appear in unwind rules. Needs a vendored fork of the crate, so it's out of scope here.

🤖 Generated with Claude Code

…> 250K)

`catch_unwind`-based panic/OOM recovery has been non-functional on device:
Test 1 of `--features test_oom` raised its panic, then unwinding tripped
esp-hal's stack guard and the device never finished booting.

Root cause is a stack budget failure, not a toolchain or esp-hal regression.
The main stack is whatever DRAM is left after .bss, and .bss holds the
esp_alloc heap, so every heap byte costs a stack byte. At 300_000 the stack
was 32,392 B total and only 25,976 B were free at the panic site. Unwinding
one panic needs ~41 KB, concentrated in two frames:

  <unwinding::unwinder::frame::Frame>::from_context             20,736 B
  with_context::delegate::<..._Unwind_RaiseException::{c#0}>     8,448 B

`from_context` builds a gimli `UnwindContext` on the stack per frame;
`StoreOnStack` rounds MAX_REG_RULES (65) up to 128 register-rule slots,
twice over in `[UnwindTableRow; 2]`, plus the cloned row. So the unwinder
ran off the bottom of the stack and wrote `__stack_chk_guard`.

Everything downstream followed from that. esp-hal's `ExceptionHandler` is
`extern "C"` and therefore nounwind, so panicking inside it and unwinding
hits `panic_cannot_unwind`; that panic re-enters esp_println's already-held
esp-sync lock, producing an unbounded "lock is not reentrant" cascade.

Dropping the heap to 250_000 leaves an 85,784-byte stack. Measured
high-water for a boot-depth panic is 47,224 B, so ~38 KB remains for panics
raised deeper in node render. Heap is not tight afterwards: 142 KB of
250 KB still free with the studio project loaded. Flash is unaffected --
the heap lives in .bss.

Verified on an ESP32-C6 (XIAO, a0:f2:62:87:b4:8c) at this commit:

  [test_oom] Test 1 OK: simple panic caught
  [test_oom] Test 2 OK: OOM caught, recovery works
  [test_oom] Tests complete, continuing boot...
  Boot: auto-loaded project /projects/studio
  [RECOVERY] boot complete (first frame served)

with recovery level green, safe_mode=false, and steady frame service.

Refs docs/reports/2026-03-13-esp32-unwinding-implementation.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant