Skip to content

Fix pthread_create self-recursion on glibc >= 2.34#1

Open
Zeleph25 wants to merge 1 commit into
UTSASRG:masterfrom
Zeleph25:fix-pthread-create-glibc234
Open

Fix pthread_create self-recursion on glibc >= 2.34#1
Zeleph25 wants to merge 1 commit into
UTSASRG:masterfrom
Zeleph25:fix-pthread-create-glibc234

Conversation

@Zeleph25

@Zeleph25 Zeleph25 commented Jul 8, 2026

Copy link
Copy Markdown

Fix: pthread_create self-recursion on glibc >= 2.34 (source/real.cpp)

Bug

real.cpp::initializer() resolves the real pthread_create via:

void *pthread_handle = dlopen("libpthread.so.0", RTLD_NOW | RTLD_GLOBAL | RTLD_NOLOAD);
INIT_WRAPPER(pthread_create, pthread_handle);

This assumes pthread functions live in a separate libpthread.so.0, which was true before glibc 2.34 (2021). On any system with glibc >= 2.34 (e.g. Ubuntu with glibc 2.42), pthread functions are merged into libc.so.6, so libpthread.so.0 is never loaded into the process. RTLD_NOLOAD then makes dlopen return NULL.

In dlsym(NULL, "pthread_create"), NULL means "search the global default scope." Since this library is loaded via LD_PRELOAD, its own interposed pthread_create is first in that scope, so Real::pthread_create ends up pointing at itself. Every call to xthread::thread_create() then recurses through its own pthread_create wrapper infinitely, never reaching a real OS thread, until _alives blows past MAX_ALIVE_THREADS and trips the assertion in allocThreadIndex():

threadspawn_test: ./xthread.hh:213: int xthread::allocThreadIndex(): Assertion `_alives < _max' failed.

Confirmed with a gdb backtrace on the crash - repeating pattern all the way down:

xthread::thread_create (xthread.cpp:62)
  -> pthread_create (libnumalloc.cpp:343)   <- this library's own wrapper
    -> xthread::thread_create (xthread.cpp:62)
      -> pthread_create (libnumalloc.cpp:343)
        -> ... (repeats)

Also confirmed on this system:

  • ldd <any binary preloading libnumalloc.so> | grep pthread → empty (libpthread.so.0 never loaded)
  • nm -D libnumalloc.so | grep pthread_createT pthread_create (confirms it's first in global scope)

Reproduces on an unmodified build (NUMA_NODES=2, default config) - unrelated to node count or topology.

Fix

Resolve pthread_create/pthread_join/pthread_kill via RTLD_NEXT instead - the same mechanism this file already uses correctly for malloc/free/calloc/realloc two lines above. RTLD_NEXT finds the next definition after the calling library in the normal search order, regardless of whether pthread symbols live in a separate libpthread.so.0 or in libc.so.6.

 void initializer() {
   INIT_WRAPPER(malloc, RTLD_NEXT);
   INIT_WRAPPER(free, RTLD_NEXT);
   INIT_WRAPPER(calloc, RTLD_NEXT);
   INIT_WRAPPER(realloc, RTLD_NEXT);

-  void *pthread_handle = dlopen("libpthread.so.0", RTLD_NOW | RTLD_GLOBAL | RTLD_NOLOAD);
-  INIT_WRAPPER(pthread_create, pthread_handle);
-  INIT_WRAPPER(pthread_join, pthread_handle);
-  INIT_WRAPPER(pthread_kill, pthread_handle);
+  INIT_WRAPPER(pthread_create, RTLD_NEXT);
+  INIT_WRAPPER(pthread_join, RTLD_NEXT);
+  INIT_WRAPPER(pthread_kill, RTLD_NEXT);
 }

Testing

Multithreaded smoke test (6 threads, malloc/free per thread) via LD_PRELOAD:

  • Before: crashes on the 2nd thread creation, every time, regardless of NUMA_NODES.
  • After: all 6 threads complete, exit code 0.

Tested on Ubuntu, glibc 2.42, clang++ build per the existing Makefile.

Scope

This PR only touches source/, since that's the copy matching the published ISMM'23 paper. The identical dlopen("libpthread.so.0", ...) pattern also exists in source-0104, source-48-class-size, source-fasterthantcmalloc-raytrace, source-fullinterleaved, source-linklist, source-reducememory, source-remmove-jumplist, source-scalability, source-smallbag, and (with minor unrelated differences) binding/ and try/ - all 12 copies in the repo are affected. Happy to open follow-up PRs for those if useful.

Resolve pthread_create/join/kill via RTLD_NEXT instead of
dlopen(libpthread.so.0, RTLD_NOLOAD), which returns NULL on
glibc >= 2.34 (pthread merged into libc.so.6), causing
dlsym(NULL, ...) to resolve back to this library's own
interposed pthread_create and recurse infinitely.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant