Skip to content

test: fix flaky PTY env test#30

Merged
knep merged 1 commit into
masterfrom
fix/flaky-pty-env-test
Jun 25, 2026
Merged

test: fix flaky PTY env test#30
knep merged 1 commit into
masterfrom
fix/flaky-pty-env-test

Conversation

@knep

@knep knep commented Jun 25, 2026

Copy link
Copy Markdown
Owner

Summary

Stabilises process_pty_test.py::TestEnvironmentVariables::test_PYTHONUNBUFFERED, which intermittently failed in CI (most recently on Python 3.14, PR #29) with:

AssertionError: '1' != None

i.e. the PYTHONUNBUFFERED line was missing from the captured environment.

Root cause — a read race against the output stream

start() reads the process output on a separate thread that pushes chunks into a ReplayObservable and close()s it when done. The tests read via read_until_closed() while that thread was still pushing, after only a thread.join(timeout=0.1).

ReplayObservable.subscribe() (replay buffered chunks, then register the observer) is not synchronised against push() (append chunk, fire to observers). A chunk pushed in the window between "replay finished" and "observer registered" is neither replayed nor delivered live — so the last line of output could be dropped. The tiny, fast printenv.sh output made this most likely, and fast/loaded runners (3.14) hit it.

Fix (test-only)

Wait for output_stream.wait_close() before reading. Once the stream is closed there is no concurrent pusher, and close() keeps the replay buffer (only dispose() clears it, which these tests don't call), so the late subscribe replays every chunk deterministically. Extracted as a shared _run_and_read_all() helper and applied to the env tests and the unicode/encoding tests (which had the same latent race).

No production code changed.

Testing

  • process_pty_test.py run 20× locally: 20/20 pass (and the fix is deterministic by construction, not just lucky).
  • Full backend suite: 1740 passed (×2 runs).

Follow-up (out of scope)

The underlying ReplayObservable.subscribe/push race is a real (if rare) concurrency gap that could also affect a production subscriber connecting mid-execution. Hardening it would mean adding synchronisation to that core class — deliberately left out of this test-only fix.

🤖 Generated with Claude Code

test_PYTHONUNBUFFERED intermittently failed on CI with "'1' != None": the
PYTHONUNBUFFERED line was missing from the captured env.

Root cause is a race in how the test read the output. start() reads the process
output on a separate thread that pushes chunks into a ReplayObservable and
close()s it when done. Reading via read_until_closed() while that thread is still
pushing races ReplayObservable.subscribe (replay buffered chunks, then register)
against push (append chunk, fire to observers): a chunk pushed in the gap is
neither replayed nor delivered, so the last env line could be dropped. The tiny,
fast printenv.sh output made this most likely (and 3.14/loaded runners hit it).

Fix the tests to wait for output_stream.wait_close() before reading: once closed
there is no concurrent pusher, and close() keeps the replay buffer, so the late
subscribe replays every chunk deterministically. Applied via a shared
_run_and_read_all() helper to the env tests and the unicode/encoding tests
(same latent race). No production code changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@knep knep merged commit fc9bb76 into master Jun 25, 2026
8 checks passed
@knep knep deleted the fix/flaky-pty-env-test branch June 25, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants