Skip to content

build: per-variant objdirs — variants coexist, rebuilds go incremental (#740) - #808

Merged
InauguralPhysicist merged 1 commit into
mainfrom
fix-740-per-variant-objdir
Aug 2, 2026
Merged

build: per-variant objdirs — variants coexist, rebuilds go incremental (#740)#808
InauguralPhysicist merged 1 commit into
mainfrom
fix-740-per-variant-objdir

Conversation

@InauguralPhysicist

Copy link
Copy Markdown
Collaborator

Closes #740

What

Every runtime variant was a single whole-program gcc invocation writing src/eigenscript — an ASan build destroyed the release binary, every edit cost a full 22-TU rebuild, and the project was absorbing the collision through the #681 fingerprint guard plus a standing CLAUDE.md prohibition.

Now each of the 11 variants compiles into its own build/<variant>/ objdir with -MMD/-MP header-dependency tracking and links build/<variant>/eigenscript; the phony target re-points src/eigenscript at it, so every existing consumer of that path — 35 test/tool scripts, CI, install, consumers — works unchanged. Target names, per-variant flags, and output messages are byte-identical.

Measured:

  • make asanmake: ~90 s full rebuild → 0.2 s relink
  • touch vm.c → 1 TU recompiles; touch vm.h → exactly its 12 includers
  • all 11 variant objdirs coexist under build/

The hard-link subtlety (a symlink prototype failed 58 suite checks)

The alias is a hard link, not a symlink. The runtime resolves the stdlib relative to /proc/self/exe, which dereferences a symlink into build/<variant>/ — and lib/ stops being a sibling. The symlink prototype failed every executable-relative-stdlib test (bundle, replay, import). The hard link keeps the exec'd path at src/eigenscript, and coexistence is preserved because each variant's objdir file keeps its own name.

Consequences handled:

  • pgo/coverage (whole-program by nature) and build.sh rm -f the alias before writing, so they never truncate the shared inode under a variant's binary.
  • A real guard trip during verification exposed that the guard's stat didn't dereference — its metadata half described the link while its cksum half read the target. It now uses stat -L, and [99d]'s restore re-creates a symlink alias instead of materializing a copy. (Both are no-ops for the hard link; they make the guard correct under any alias form.)

What's retired vs kept

Gate

  • release suite 3364/3364; ASan/UBSan detect_leaks=1 3368/3368, leak tally 0
  • all 10 locally-buildable variants built through the engine; make -n full verified (libpq absent locally — the db extension CI leg is the real build)
  • build.sh interop verified: writes its own file, variant binaries untouched, stdlib resolves

🤖 Generated with Claude Code

#740)

Every runtime variant (build/full/http/zlib/net/gfx/asan/asan-http/tsan/
valgrind/poison) was one whole-program gcc invocation writing
src/eigenscript: an ASan build destroyed the release binary, any edit
cost a full 22-TU rebuild, and the project paid for the collision twice
(the #681 fingerprint guard, a standing CLAUDE.md prohibition).

Each variant now compiles into build/<variant>/ with -MMD/-MP header
dependency tracking and links build/<variant>/eigenscript; the phony
target re-points src/eigenscript at it as a HARD link, so every existing
consumer of that path works unchanged. Hard, not symbolic: the runtime
resolves lib/ relative to /proc/self/exe, which dereferences a symlink
into build/ and loses the executable-relative stdlib — the symlink
prototype failed 58 suite checks exactly there.

Measured: make asan -> make went from a ~90s full rebuild to a 0.2s
relink; touching vm.c recompiles 1 TU; touching vm.h recompiles its 12
includers. Target names, per-variant flags, and output messages are
unchanged.

pgo/coverage (whole-program by nature) and build.sh now rm -f the alias
first so they write a fresh file instead of truncating the shared inode
under a variant's binary. The fingerprint guard stays — re-pointing the
alias or relinking the same variant mid-suite is still a mid-run swap —
with two symlink-hardened touches found by a real guard trip: the
guard's stat now dereferences (-L) so its metadata half describes the
file its cksum half reads, and [99d]'s restore re-creates a symlink
alias instead of materializing a copy. What is retired is the
cross-variant half of the "never rebuild while a suite runs"
prohibition, not the guard.

Gate: release suite 3364/3364; ASan/UBSan detect_leaks=1 3368/3368,
leak tally 0; all 10 locally-buildable variants + make -n full clean;
build.sh interop verified (writes its own file, variant binaries
untouched).

Closes #740

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 2, 2026 10:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reworks the build system so each runtime variant builds incrementally into its own build/<variant>/ object directory, while keeping the historical src/eigenscript path working via a retargeted hard-link alias. It also updates the test runner’s binary-fingerprint guard and supporting docs/scripts to account for the new alias behavior.

Changes:

  • Introduces a per-variant objdir engine in Makefile (pattern rules + -MMD/-MP deps) and retargets src/eigenscript to the last-built variant via hard link.
  • Updates tests/run_all_tests.sh fingerprint guard to use stat -L (symlink-safe) and adjusts the [99d] self-test restore logic.
  • Updates supporting documentation (CLAUDE.md, .claude/rules/..., CHANGELOG.md) and build.sh to avoid truncating a hard-linked alias.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/run_all_tests.sh Makes the binary fingerprint guard symlink-aware and updates the [99d] self-test logic for the new alias behavior.
Makefile Implements per-variant build/<variant>/ objdirs with dependency tracking and hard-link retargeting for src/eigenscript.
CLAUDE.md Updates the build guidance to reflect per-variant builds and the src/eigenscript hard-link alias.
CHANGELOG.md Documents the per-variant objdir change and its operational implications.
build.sh Removes src/eigenscript before compiling to avoid writing through a hard link.
.claude/rules/c-runtime-memory.md Updates project guidance about variant coexistence and mid-suite alias swaps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/run_all_tests.sh
Comment on lines +38 to +40
# -L: dereference — cksum reads through a symlink, so the size/mtime
# half must describe the same file the cksum half does (#740 made
# src/eigenscript a symlink to build/<variant>/eigenscript).
Comment thread tests/run_all_tests.sh
Comment on lines +3901 to 3905
# If the binary is the #740 variant symlink, remember its target so the
# restore can re-create the link (the swap's mv replaces it with a
# regular file; the link's target file itself is never touched).
SELFTEST_LINK_TARGET=$(readlink "$EIGS_BIN" 2>/dev/null || true)
cp -p "$EIGS_BIN" "$SELFTEST_BIN_BAK"
@InauguralPhysicist
InauguralPhysicist merged commit cd9f82e into main Aug 2, 2026
19 checks passed
@InauguralPhysicist
InauguralPhysicist deleted the fix-740-per-variant-objdir branch August 2, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build: every variant writes src/eigenscript, which is why the test runner needs a binary fingerprint guard — a per-variant objdir retires both

2 participants