chore(docker): jemalloc-accurate local image + contrib/Dockerfile cleanup#9774
Open
matthewmcneely wants to merge 1 commit into
Open
chore(docker): jemalloc-accurate local image + contrib/Dockerfile cleanup#9774matthewmcneely wants to merge 1 commit into
matthewmcneely wants to merge 1 commit into
Conversation
…anup Add `make local-image-jemalloc`, which compiles a jemalloc-enabled, CGO, native-Linux dgraph binary inside a Linux container and packages it via contrib/Dockerfile -- the same runtime image CI/CD publishes. The plain local-image target cross-compiles from macOS with BUILD_TAGS= empty, so on a Mac it ships a non-jemalloc binary that misrepresents production memory behavior. Building natively in a container sidesteps the macOS->Linux jemalloc cross-compile entirely. - contrib/Dockerfile.build: new multi-stage builder; exports the compiled binary to ./linux via BuildKit for contrib/Dockerfile's `ADD linux`. - Makefile: local-image-jemalloc target plus BUILD_PLATFORM/GO_VERSION knobs (defaults to the host arch so the jemalloc compile runs natively). - contrib/Dockerfile: drop libjemalloc-dev -- the binary statically links jemalloc, so no runtime .so is loaded -- and remove the no-op GODEBUG=madvdontneed=1 (Go's default since 1.16, and it never governed jemalloc's off-heap arenas). Document JE_MALLOC_CONF as the real tuning knob, prefixed because jemalloc is built --with-jemalloc-prefix=je_. - .dockerignore: exclude dgraphtest/datafiles (~24GB) and cached binaries so the build context is ~1GB instead of ~25GB. Keep linux/ so `ADD linux` works. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
make local-image-jemalloc, a target that builds a jemalloc-enableddgraph/dgraph:localimage, and clean upcontrib/Dockerfileand.dockerignorealongthe way.
packages it through the same
contrib/Dockerfilethat CI uses.contrib/Dockerfile: drop an unused apt package and a no-op env var, and document thereal jemalloc tuning knob.
.dockerignore: stop shipping a ~25GB build context.Why
On macOS,
make local-imagecross-compiles withBUILD_TAGS=empty, so it ships anon-jemalloc binary. Production links jemalloc statically, so the local image is a
poor stand-in for anything memory-related. Cross-compiling jemalloc from macOS to Linux is
the hard part, so this sidesteps it by building natively in a container.
Two smaller issues in
contrib/Dockerfilesurfaced while I was in there.Changes
make local-image-jemalloc(+contrib/Dockerfile.build)Multi-stage builder. Compiles the binary in a
golangLinux container (jemalloc builtfrom source and statically linked, CGO on, default
BUILD_TAGS=jemalloc), exports it viaBuildKit, then packages it through
contrib/Dockerfile. Defaults to the host arch so thejemalloc compile runs natively; set
BUILD_PLATFORM=linux/amd64for amd64-CI parity(qemu-emulated).
contrib/Dockerfilecleanuplibjemalloc-dev. The binary links jemalloc statically(
/usr/local/lib/libjemalloc.avia the ristrettozpackage cgo directive), so nolibjemalloc.soloads at runtime. Confirmed withreadelf -dand grep on the builtimage, so the runtime package was dead weight.
GODEBUG=madvdontneed=1. It has been the Go runtime default since Go 1.16, andit only ever governed the Go heap, never jemalloc's off-heap arenas where the large
allocations live. It was a no-op here.
JE_MALLOC_CONFas the tuning knob (prefixed, because jemalloc is built--with-jemalloc-prefix=je_, so plainMALLOC_CONFis ignored), plus thenarenas/percpu_arenacaveat for CPU-limited containers and a link to the jemallocdocs.
.dockerignoreExclude
dgraphtest/datafiles(~24GB of LDBC/test fixtures) and cached binaries. Buildcontext drops from ~25GB to under 1GB.
linux/stays included socontrib/Dockerfile'sADD linuxstill works. This also fixes the existinglocal-imagetarget and anydocker build .in the repo, which were all shipping the full 25GB.Testing
Built the image and verified in a container:
dgraph versionreportsjemalloc enabled : truelibjemalloc.soinreadelf -d)malloc_confpresent,libjemalloc-devabsent from the imageNote
The default
make local-imageon macOS stays non-jemalloc on purpose (fast iteration, noper-change container build). jemalloc is opt-in through the new target. Everyday CI on
Linux already builds jemalloc, so this mainly helps macOS developers doing memory-related
work.