fix: avoid retaining a completed task per protocol message on Python 3.14 (#41357)#3114
fix: avoid retaining a completed task per protocol message on Python 3.14 (#41357)#3114yen0304 wants to merge 1 commit into
Conversation
…3.14 Each protocol send awaited the long-lived transport `on_error_future` together with its per-message callback future via `asyncio.wait(..., FIRST_COMPLETED)`. On Python 3.14 the new asyncio await-graph keeps strong references between a task and the futures it awaits, so every completed send task stayed registered in `on_error_future._asyncio_awaited_by` until that future resolved, i.e. until the whole Connection closed. Intercepting requests (context.route / page.route) spawns one such task per intercepted request, so memory grew linearly with the number of requests and was not released by closing the page, context, or browser. Forward transport errors to the pending callback future via a done callback instead of awaiting `on_error_future` directly, so each send's await graph stays local to its own short-lived future. The transport-error behaviour is preserved: a send still raises the transport error if the transport dies mid-flight, and the callback is removed in a finally block. Adds a regression test that intercepts many requests and asserts completed asyncio tasks are not retained after the contexts are closed.
|
Nice fix. I tried this branch locally against the #41357 shape on Python 3.14.4 / Chromium. It removes the accumulating I think there is one more It may also be worth tightening the regression to count |
Fixes microsoft/playwright#41357.
Summary
Registering
context.route(...)/page.route(...)caused the async client to retain one completedasyncio.Taskper intercepted request until the Connection closed (end ofasync_playwright()). Closing the page, context, or browser did not release them, so memory grew roughly linearly with the number of intercepted requests on a long-lived connection.Root cause
Every protocol send in
Channel._inner_sendawaited the long-lived transporton_error_futurealongside its per-messagecallback.future:on_error_futureis created once per connection (Transport.__init__) and only resolves when the transport errors/closes. On Python 3.14, asyncio added an await-graph (Future._asyncio_awaited_by) that keeps strong references between a task and the futures it awaits. Because each send runs in its own task (route handlers are dispatched as tasks) and awaits the never-resolvingon_error_future, every completed send task stayed registered inon_error_future._asyncio_awaited_byand was only released when that future — and the wholeConnection— went away.This is interception-agnostic (a handler that
continue_()s leaks the same as one thatabort()s) because the leak is in the generic send path, which route handling exercises once per request.Reproduction
Pinned down with
gc.get_referrers; the retention chain is:Minimal, browserless confirmation (distinct task per wait against a long-lived future):
Full repro from the issue (80 sub-resources/page, intercept-and-abort):
browser.close()The fix
Forward transport errors to the pending
callback.futurethrough a done callback instead of awaitingon_error_futuredirectly, so each send's await graph stays local to its own short-lived future and is released as soon as the send completes. Behaviour is preserved:finallyblock;callback.future(avoids theInvalidStateErrorthat a naiveset_exceptionforward would race withConnection.cleanup).Test plan
test_route_should_not_leak_done_tasks— intercepts 100 requests across 5 contexts and asserts retained completed tasks stay bounded. Verified it fails on unpatched 3.14 (leaked 200) and passes after the fix (leaked 0); passes on 3.12 too.black/isort/flake8clean on the changed files.