Skip to content

perf: stop spurious mesh epoch churn; drop per-chunk zero-fill in silkd bulk path - #53

Merged
CMGS merged 4 commits into
mainfrom
perf/static-trims
Jul 26, 2026
Merged

perf: stop spurious mesh epoch churn; drop per-chunk zero-fill in silkd bulk path#53
CMGS merged 4 commits into
mainfrom
perf/static-trims

Conversation

@CMGS

@CMGS CMGS commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Fixes out of a static performance pass over sandboxd, the Rust crates, the SDKs, protocol/wire, and mcp.

sandboxd: sort template hashes so the mesh unchanged-guard holds

TemplateHashes returned map-iteration order while UpdateSelf compares the
template list order-sensitively (slices.Equal). With two or more promoted
templates the "unchanged view" guard misfired on a large fraction of 1s gossip
ticks — each misfire a spurious epoch bump: a durable double-fsync persist plus
a cluster-wide republish and re-merge. One-line fix, regression test pinning
the canonical order, and UpdateSelf now documents the sorted contract.

Measured (mesh of one, 3 promoted templates, 60s): epoch delta +19 → 0.

silkd: drop the per-chunk zero-fill in write_chunk_frame

The scratch buffer was cleared and resize(.., 0)-extended per chunk,
memsetting the whole base64 region (~349KB per 256KB bulk chunk) that
encode_slice immediately overwrites — on every stdout/stderr/data frame of
every streaming verb. The buffer is now grow-only: the zero-fill runs once at
high-water and each frame fully overwrites buf[..total].

Micro-bench (bench_chunk_render_old_vs_new, release): 32768 x 256KB chunks
4.08s → 2.31s (~1.77x). Bare-metal A/B (Ryzen 9700X, same sandboxd, old vs new
silkd in the same rt:24.04 base, two rounds each): fs_pull 963.2/969.0 vs
948.6/842.5 MiB/s — new ≥ old in every pairing and materially tighter; exec
RTT, claim tiers, cold boot, refill recovery unchanged.

sdk/go + protocol/wire: trim the per-RPC dial and read path

  • dialAgent writes the fixed-shape upgrade request directly instead of
    driving http.Request.Write (measured 16 → 2 allocs/op), with the same
    control-character guard the Python SDK already had (plus a regression test).
  • Frame scanning consolidates into wire.NewFrameScanner, shared by the SDK
    and sandboxd's engine — both copies previously pre-allocated a zeroed 64KB
    buffer per connection that bulk frames outgrew anyway.
  • ReadFile retains decoded chunks and concatenates once instead of paying
    append-growth re-copies; the retention contract is documented on
    DecodeResponse, and the silkdtest fake now chunks downloads at silkd's
    real 256KB frame size (multi-chunk path is now tested).

Bare-metal A/B (same daemon and template, old vs new SDK bench binaries, two
rounds each): exec RTT p50 0.21/0.17ms vs 0.22/0.21ms, fs_pull 792.6/975.8 vs
768.6/907.0 MiB/s — new ≤/≥ old respectively in every pairing, no regression.

sdk/python: TCP_NODELAY + 256K fs chunks

Raw data-plane sockets left Nagle enabled (Go's net.Dial disables it by
default); small back-to-back frames risked delayed-ACK stalls once client and
node are not co-located. FS_CHUNK rises 32K → 256K to match silkd's frame
chunk — 8x fewer frames on large writes.

Measured (8MiB write_file, bare metal, median of 6): 27.9ms → 25.5ms (~9%).

Gates

go test -race across sandboxd, sdk/go, protocol/wire, e2e; golangci-lint
linux+darwin 0 issues; asl clean both GOOS; cargo fmt --check + clippy -D warnings + cargo test clean; ruff check clean; 136 Python tests pass.

CMGS added 2 commits July 27, 2026 01:06
Map-iteration order made UpdateSelf's order-sensitive compare misfire on
most 1s ticks once two or more promoted templates existed: each misfire
cost a spurious epoch bump, a double-fsync persist, and a cluster-wide
re-merge of this node's state.
The scratch buffer is now grow-only: resize's zero-fill ran on every
output chunk (~349KB memset per 256KB chunk, immediately overwritten by
encode_slice) and is now paid once at high-water. bench_chunk_render
32768 x 256KB chunks: 4.08s -> 2.31s (~1.77x) on the hand-rendered path.
CMGS added 2 commits July 27, 2026 01:56
dialAgent writes the fixed-shape upgrade request directly (16 -> 2
allocs/op) with the same control-character guard as the Python SDK;
frame scanning moves to wire.NewFrameScanner shared with sandboxd's
engine, dropping the 64KB pre-sized buffer both copies paid per conn;
ReadFile retains decoded chunks instead of re-copying through append
growth, with the retention contract documented on DecodeResponse.
The silkdtest fake now chunks downloads at silkd's real frame size.
Raw data-plane sockets left Nagle on (Go's net.Dial disables it);
small back-to-back frames risked delayed-ACK stalls off-loopback.
FS_CHUNK now matches silkd's 256K frame chunk instead of 32K —
8x fewer frames on large writes.
@CMGS
CMGS merged commit 9661c50 into main Jul 26, 2026
4 checks passed
@CMGS
CMGS deleted the perf/static-trims branch July 26, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant