Skip to content

chore(docker): jemalloc-accurate local image + contrib/Dockerfile cleanup#9774

Open
matthewmcneely wants to merge 1 commit into
mainfrom
matthewmcneely/docker-updates
Open

chore(docker): jemalloc-accurate local image + contrib/Dockerfile cleanup#9774
matthewmcneely wants to merge 1 commit into
mainfrom
matthewmcneely/docker-updates

Conversation

@matthewmcneely

Copy link
Copy Markdown
Contributor

Summary

Add make local-image-jemalloc, a target that builds a jemalloc-enabled
dgraph/dgraph:local image, and clean up contrib/Dockerfile and .dockerignore along
the way.

  • New target compiles a jemalloc, CGO, native-Linux binary inside a Linux container, then
    packages it through the same contrib/Dockerfile that CI uses.
  • contrib/Dockerfile: drop an unused apt package and a no-op env var, and document the
    real jemalloc tuning knob.
  • .dockerignore: stop shipping a ~25GB build context.

Why

On macOS, make local-image cross-compiles with BUILD_TAGS= empty, so it ships a
non-jemalloc binary. Production links jemalloc statically, so the local image is a
poor stand-in for anything memory-related. Cross-compiling jemalloc from macOS to Linux is
the hard part, so this sidesteps it by building natively in a container.

Two smaller issues in contrib/Dockerfile surfaced while I was in there.

Changes

make local-image-jemalloc (+ contrib/Dockerfile.build)

Multi-stage builder. Compiles the binary in a golang Linux container (jemalloc built
from source and statically linked, CGO on, default BUILD_TAGS=jemalloc), exports it via
BuildKit, then packages it through contrib/Dockerfile. Defaults to the host arch so the
jemalloc compile runs natively; set BUILD_PLATFORM=linux/amd64 for amd64-CI parity
(qemu-emulated).

contrib/Dockerfile cleanup

  • Drop libjemalloc-dev. The binary links jemalloc statically
    (/usr/local/lib/libjemalloc.a via the ristretto z package cgo directive), so no
    libjemalloc.so loads at runtime. Confirmed with readelf -d and grep on the built
    image, so the runtime package was dead weight.
  • Remove GODEBUG=madvdontneed=1. It has been the Go runtime default since Go 1.16, and
    it only ever governed the Go heap, never jemalloc's off-heap arenas where the large
    allocations live. It was a no-op here.
  • Document JE_MALLOC_CONF as the tuning knob (prefixed, because jemalloc is built
    --with-jemalloc-prefix=je_, so plain MALLOC_CONF is ignored), plus the
    narenas/percpu_arena caveat for CPU-limited containers and a link to the jemalloc
    docs.

.dockerignore

Exclude dgraphtest/datafiles (~24GB of LDBC/test fixtures) and cached binaries. Build
context drops from ~25GB to under 1GB. linux/ stays included so contrib/Dockerfile's
ADD linux still works. This also fixes the existing local-image target and any
docker build . in the repo, which were all shipping the full 25GB.

Testing

Built the image and verified in a container:

  • dgraph version reports jemalloc enabled : true
  • statically linked (no libjemalloc.so in readelf -d)
  • baked malloc_conf present, libjemalloc-dev absent from the image
  • version string stamped correctly

Note

The default make local-image on macOS stays non-jemalloc on purpose (fast iteration, no
per-change container build). jemalloc is opt-in through the new target. Everyday CI on
Linux already builds jemalloc, so this mainly helps macOS developers doing memory-related
work.

…anup

Add `make local-image-jemalloc`, which compiles a jemalloc-enabled, CGO,
native-Linux dgraph binary inside a Linux container and packages it via
contrib/Dockerfile -- the same runtime image CI/CD publishes. The plain
local-image target cross-compiles from macOS with BUILD_TAGS= empty, so on a
Mac it ships a non-jemalloc binary that misrepresents production memory
behavior. Building natively in a container sidesteps the macOS->Linux jemalloc
cross-compile entirely.

- contrib/Dockerfile.build: new multi-stage builder; exports the compiled
  binary to ./linux via BuildKit for contrib/Dockerfile's `ADD linux`.
- Makefile: local-image-jemalloc target plus BUILD_PLATFORM/GO_VERSION knobs
  (defaults to the host arch so the jemalloc compile runs natively).
- contrib/Dockerfile: drop libjemalloc-dev -- the binary statically links
  jemalloc, so no runtime .so is loaded -- and remove the no-op
  GODEBUG=madvdontneed=1 (Go's default since 1.16, and it never governed
  jemalloc's off-heap arenas). Document JE_MALLOC_CONF as the real tuning knob,
  prefixed because jemalloc is built --with-jemalloc-prefix=je_.
- .dockerignore: exclude dgraphtest/datafiles (~24GB) and cached binaries so
  the build context is ~1GB instead of ~25GB. Keep linux/ so `ADD linux` works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@matthewmcneely matthewmcneely requested a review from a team as a code owner July 8, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant