build: per-variant objdirs — variants coexist, rebuilds go incremental (#740) - #808
Merged
Merged
Conversation
#740) Every runtime variant (build/full/http/zlib/net/gfx/asan/asan-http/tsan/ valgrind/poison) was one whole-program gcc invocation writing src/eigenscript: an ASan build destroyed the release binary, any edit cost a full 22-TU rebuild, and the project paid for the collision twice (the #681 fingerprint guard, a standing CLAUDE.md prohibition). Each variant now compiles into build/<variant>/ with -MMD/-MP header dependency tracking and links build/<variant>/eigenscript; the phony target re-points src/eigenscript at it as a HARD link, so every existing consumer of that path works unchanged. Hard, not symbolic: the runtime resolves lib/ relative to /proc/self/exe, which dereferences a symlink into build/ and loses the executable-relative stdlib — the symlink prototype failed 58 suite checks exactly there. Measured: make asan -> make went from a ~90s full rebuild to a 0.2s relink; touching vm.c recompiles 1 TU; touching vm.h recompiles its 12 includers. Target names, per-variant flags, and output messages are unchanged. pgo/coverage (whole-program by nature) and build.sh now rm -f the alias first so they write a fresh file instead of truncating the shared inode under a variant's binary. The fingerprint guard stays — re-pointing the alias or relinking the same variant mid-suite is still a mid-run swap — with two symlink-hardened touches found by a real guard trip: the guard's stat now dereferences (-L) so its metadata half describes the file its cksum half reads, and [99d]'s restore re-creates a symlink alias instead of materializing a copy. What is retired is the cross-variant half of the "never rebuild while a suite runs" prohibition, not the guard. Gate: release suite 3364/3364; ASan/UBSan detect_leaks=1 3368/3368, leak tally 0; all 10 locally-buildable variants + make -n full clean; build.sh interop verified (writes its own file, variant binaries untouched). Closes #740 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR reworks the build system so each runtime variant builds incrementally into its own build/<variant>/ object directory, while keeping the historical src/eigenscript path working via a retargeted hard-link alias. It also updates the test runner’s binary-fingerprint guard and supporting docs/scripts to account for the new alias behavior.
Changes:
- Introduces a per-variant objdir engine in
Makefile(pattern rules +-MMD/-MPdeps) and retargetssrc/eigenscriptto the last-built variant via hard link. - Updates
tests/run_all_tests.shfingerprint guard to usestat -L(symlink-safe) and adjusts the [99d] self-test restore logic. - Updates supporting documentation (
CLAUDE.md,.claude/rules/...,CHANGELOG.md) andbuild.shto avoid truncating a hard-linked alias.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/run_all_tests.sh | Makes the binary fingerprint guard symlink-aware and updates the [99d] self-test logic for the new alias behavior. |
| Makefile | Implements per-variant build/<variant>/ objdirs with dependency tracking and hard-link retargeting for src/eigenscript. |
| CLAUDE.md | Updates the build guidance to reflect per-variant builds and the src/eigenscript hard-link alias. |
| CHANGELOG.md | Documents the per-variant objdir change and its operational implications. |
| build.sh | Removes src/eigenscript before compiling to avoid writing through a hard link. |
| .claude/rules/c-runtime-memory.md | Updates project guidance about variant coexistence and mid-suite alias swaps. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+38
to
+40
| # -L: dereference — cksum reads through a symlink, so the size/mtime | ||
| # half must describe the same file the cksum half does (#740 made | ||
| # src/eigenscript a symlink to build/<variant>/eigenscript). |
Comment on lines
+3901
to
3905
| # If the binary is the #740 variant symlink, remember its target so the | ||
| # restore can re-create the link (the swap's mv replaces it with a | ||
| # regular file; the link's target file itself is never touched). | ||
| SELFTEST_LINK_TARGET=$(readlink "$EIGS_BIN" 2>/dev/null || true) | ||
| cp -p "$EIGS_BIN" "$SELFTEST_BIN_BAK" |
Merged
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #740
What
Every runtime variant was a single whole-program
gccinvocation writingsrc/eigenscript— an ASan build destroyed the release binary, every edit cost a full 22-TU rebuild, and the project was absorbing the collision through the #681 fingerprint guard plus a standing CLAUDE.md prohibition.Now each of the 11 variants compiles into its own
build/<variant>/objdir with-MMD/-MPheader-dependency tracking and linksbuild/<variant>/eigenscript; the phony target re-pointssrc/eigenscriptat it, so every existing consumer of that path — 35 test/tool scripts, CI, install, consumers — works unchanged. Target names, per-variant flags, and output messages are byte-identical.Measured:
make asan→make: ~90 s full rebuild → 0.2 s relinkvm.c→ 1 TU recompiles; touchvm.h→ exactly its 12 includersbuild/The hard-link subtlety (a symlink prototype failed 58 suite checks)
The alias is a hard link, not a symlink. The runtime resolves the stdlib relative to
/proc/self/exe, which dereferences a symlink intobuild/<variant>/— andlib/stops being a sibling. The symlink prototype failed every executable-relative-stdlib test (bundle, replay, import). The hard link keeps the exec'd path atsrc/eigenscript, and coexistence is preserved because each variant's objdir file keeps its own name.Consequences handled:
pgo/coverage(whole-program by nature) andbuild.shrm -fthe alias before writing, so they never truncate the shared inode under a variant's binary.statdidn't dereference — its metadata half described the link while its cksum half read the target. It now usesstat -L, and [99d]'s restore re-creates a symlink alias instead of materializing a copy. (Both are no-ops for the hard link; they make the guard correct under any alias form.)What's retired vs kept
make <other-variant>) or relinking the same variant mid-suite is still a mid-run swap, and the guard is what catches it.Gate
detect_leaks=13368/3368, leak tally 0make -n fullverified (libpq absent locally — thedb extensionCI leg is the real build)build.shinterop verified: writes its own file, variant binaries untouched, stdlib resolves🤖 Generated with Claude Code