test: fix two flaky serverless tests#513
Open
deanq wants to merge 1 commit into
Open
Conversation
Both tests fail intermittently regardless of source changes, blocking make quality-check and CI. - TestRunWorker invoked run_worker -> run_fitness_checks, whose _memory_check probes real host memory and exits when <4GB. Stub run_fitness_checks in asyncSetUp; fitness checks are covered by tests/test_serverless/test_modules/test_fitness/. - test_download_files_from_urls asserted get() call order, but downloads run in parallel threads (executor.map preserves result order, not call order). Assert the set of requested URLs instead. Refs SLS-268
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two tests fail intermittently regardless of source changes, blocking
make quality-checkand CI.1.
TestRunWorker(test_worker.py) — real memory proberun_worker()callsasyncio.run(run_fitness_checks())(runpod/serverless/worker.py:40). The_memory_checkprobe reads real host memory andsys.exit(1)s when <4GB. NoTestRunWorkertest mocked it, so the class failed whenever the host was under memory pressure (RuntimeError: Insufficient memory: 3.66GB available, 4.0GB required).Fix: stub
run_fitness_checksinasyncSetUp. Fitness checks keep dedicated coverage intests/test_serverless/test_modules/test_fitness/.2.
test_download_files_from_urls— parallel call orderingdownload_files_from_urlsdownloads viaThreadPoolExecutor.map. The test assertedmock_get.call_args_list[index]matchedurls[index]positionally, but call order across threads is non-deterministic (executor.mappreserves result order, not call order).Fix: assert the set of requested URLs equals the input set. Result-order checks (extension,
open()call) are unchanged sinceexecutor.mappreserves result order.Test plan
Verified deterministic across repeated runs before fixing intent:
test_download_files_from_urls: 10/10 passed (was ~4/5)TestRunWorker: 9/9 passed across 10 runsmake quality-check: 478 passed, coverage 94.08% (gate 90%)