Fix pthread_create self-recursion on glibc >= 2.34#1
Open
Zeleph25 wants to merge 1 commit into
Open
Conversation
Resolve pthread_create/join/kill via RTLD_NEXT instead of dlopen(libpthread.so.0, RTLD_NOLOAD), which returns NULL on glibc >= 2.34 (pthread merged into libc.so.6), causing dlsym(NULL, ...) to resolve back to this library's own interposed pthread_create and recurse infinitely.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix:
pthread_createself-recursion on glibc >= 2.34 (source/real.cpp)Bug
real.cpp::initializer()resolves the realpthread_createvia:This assumes pthread functions live in a separate
libpthread.so.0, which was true before glibc 2.34 (2021). On any system with glibc >= 2.34 (e.g. Ubuntu with glibc 2.42), pthread functions are merged intolibc.so.6, solibpthread.so.0is never loaded into the process.RTLD_NOLOADthen makesdlopenreturnNULL.In
dlsym(NULL, "pthread_create"),NULLmeans "search the global default scope." Since this library is loaded viaLD_PRELOAD, its own interposedpthread_createis first in that scope, soReal::pthread_createends up pointing at itself. Every call toxthread::thread_create()then recurses through its ownpthread_createwrapper infinitely, never reaching a real OS thread, until_alivesblows pastMAX_ALIVE_THREADSand trips the assertion inallocThreadIndex():Confirmed with a gdb backtrace on the crash - repeating pattern all the way down:
Also confirmed on this system:
ldd <any binary preloading libnumalloc.so> | grep pthread→ empty (libpthread.so.0 never loaded)nm -D libnumalloc.so | grep pthread_create→T pthread_create(confirms it's first in global scope)Reproduces on an unmodified build (
NUMA_NODES=2, default config) - unrelated to node count or topology.Fix
Resolve
pthread_create/pthread_join/pthread_killviaRTLD_NEXTinstead - the same mechanism this file already uses correctly formalloc/free/calloc/realloctwo lines above.RTLD_NEXTfinds the next definition after the calling library in the normal search order, regardless of whether pthread symbols live in a separatelibpthread.so.0or inlibc.so.6.void initializer() { INIT_WRAPPER(malloc, RTLD_NEXT); INIT_WRAPPER(free, RTLD_NEXT); INIT_WRAPPER(calloc, RTLD_NEXT); INIT_WRAPPER(realloc, RTLD_NEXT); - void *pthread_handle = dlopen("libpthread.so.0", RTLD_NOW | RTLD_GLOBAL | RTLD_NOLOAD); - INIT_WRAPPER(pthread_create, pthread_handle); - INIT_WRAPPER(pthread_join, pthread_handle); - INIT_WRAPPER(pthread_kill, pthread_handle); + INIT_WRAPPER(pthread_create, RTLD_NEXT); + INIT_WRAPPER(pthread_join, RTLD_NEXT); + INIT_WRAPPER(pthread_kill, RTLD_NEXT); }Testing
Multithreaded smoke test (6 threads, malloc/free per thread) via
LD_PRELOAD:NUMA_NODES.Tested on Ubuntu, glibc 2.42, clang++ build per the existing Makefile.
Scope
This PR only touches
source/, since that's the copy matching the published ISMM'23 paper. The identicaldlopen("libpthread.so.0", ...)pattern also exists insource-0104,source-48-class-size,source-fasterthantcmalloc-raytrace,source-fullinterleaved,source-linklist,source-reducememory,source-remmove-jumplist,source-scalability,source-smallbag, and (with minor unrelated differences)binding/andtry/- all 12 copies in the repo are affected. Happy to open follow-up PRs for those if useful.