gh-151627: protect OrderedDict iterator creation#151688
Conversation
|
A bit more context on why this approach was chosen: The reported race happens while The mutation paths already update the OrderedDict linked-list state while holding the OrderedDict critical section. This PR makes iterator creation use the same synchronization boundary when taking its initial snapshot. The change is intentionally small and localized: it does not change iteration semantics, and it does not hold the lock while iterating. It only protects the short section where the iterator captures the first key/state from the OrderedDict. This is why the fix is in I also checked the related gh-151633/gh-151634 path. That fix targets Locally, the reproducer crashed before this change with a negative refcount/assertion failure, and no longer crashed after the change. Tests run: ./python -m test test_free_threading.test_collections -v |
Fixes gh-151627.
In free-threaded builds,
OrderedDictiterator creation could read the ordered linked-list head/tail, the current node key, the size, andod_statewithout holding theOrderedDictcritical section. A concurrentclear()orupdate()could mutate or free the linked-list nodes whileodictiter_new()was taking that initial iterator snapshot, leading to a data race and a crash/use-after-free.This change protects the initial iterator snapshot in
odictiter_new()with theOrderedDictcritical section. The mutation paths already update the linked-list state while holding theOrderedDictlock, so taking the iterator's initial current key, size, state, and object reference under the same critical section avoids racing with concurrentclear()/update().This is separate from gh-151633/gh-151634, which address
Counter.update()in_collectionsmodule.c. This PR targetsOrderedDictiterator construction inObjects/odictobject.c.A free-threading regression test was added that repeatedly creates forward and reversed
OrderedDictiterators while another thread concurrently clears and updates the sameOrderedDict.Tests run:
./python -m test test_free_threading.test_collections -v
./python -m test test_free_threading.test_collections -v -m test_iterator_update_clear_race
./python -m test test_ordered_dict -v
./python ../gh151627_reproducer.py