fix(vanilla-epoll): lazy json-comp gz cache — drop ~135 MiB -gc none boot leak#959
Open
enghitalo wants to merge 2 commits into
Open
fix(vanilla-epoll): lazy json-comp gz cache — drop ~135 MiB -gc none boot leak#959enghitalo wants to merge 2 commits into
enghitalo wants to merge 2 commits into
Conversation
…boot leak The json-comp lock-free precompute (from MDA2AV#956) compressed the full (count x m) grid (~800 keys) at boot. On this `-gc none` entry V's gzip.compress leaks ~157 KiB of scratch per call (vlang/v#27606), so the boot precompute leaked ~135 MiB of permanent RSS — CI showed baseline mem jump 77 -> 213 MiB across every profile. Restore the original LAZY, process-shared, RwMutex-guarded gz cache: compress only the ~8 (count,m) pairs the profile actually sends (~1.3 MiB) and append the cached bytes on a hit. This is the exact json-comp code CI measured at 77 MiB in the MDA2AV#956 combo commit (pre-cleanup), plus a guard comment so the precompute is not reintroduced. The lock-free change had no upside: it was meant to fix the io_uring json-comp@16384 collapse (lock-contention hypothesis, enghitalo/vanilla#89), but CI refuted it — epoll is healthy at 16384c WITH the lazy lock. Validated (benchmarked as MDA2AV#958): baseline 213 -> 77 MiB (-63.8%), json-comp@16384 2.31M rps / CPU 6389% (saturated) / 185 MiB (-50.9%), no throughput regression. Split out of MDA2AV#958 (one PR per engine). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
/benchmark -f vanilla-epoll --save |
Contributor
|
👋 |
Contributor
Benchmark ResultsFramework:
Full log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Restore the lazy, shared, RwMutex-guarded json-comp gz cache on
vanilla-epoll, reverting the lock-free boot precompute that shipped in #956.(Split of #958 — this is the epoll half; the io_uring half is a separate PR.)
Why
The precompute compressed the full
(count × m)grid (~800 keys) at boot. On this-gc nonebuild, V'sgzip.compressleaks ~157 KiB of internal scratch per call (vlang/v#27606), so the boot precompute leaked ~135 MiB of permanent RSS — CI showed baseline memory jump 77 → 213 MiB across every profile.The lazy cache compresses only the 8
(count,m)pairs the profile actually sends (~1.3 MiB). This restores the exact json-comp code CI measured at 77 MiB in the #956 combo commit (pre-cleanup), plus a guard comment so the precompute isn't reintroduced.The lock-free change had no upside: it was meant to fix the io_uring
json-comp@16384collapse (lock-contention hypothesis, enghitalo/vanilla#89), but CI refuted it — epoll is healthy at 16384c with the lazy lock.Benchmark (already run on #958, vanilla-epoll)
Memory dropped 30–67% across every profile. Happy to re-run
/benchmark -f vanilla-epollon this split PR to re-validate.🤖 Generated with Claude Code