Skip to content

reuse SSL context and API http client instead of rebuilding per request - #535

Draft
lilyydu wants to merge 1 commit into
mainfrom
lilyydu/fix-async-client
Draft

reuse SSL context and API http client instead of rebuilding per request#535
lilyydu wants to merge 1 commit into
mainfrom
lilyydu/fix-async-client

Conversation

@lilyydu

@lilyydu lilyydu commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

This is a performance fix concerning redundant HTTP client construction. Callers can now share a pre-built SSL context instead of paying ~2.4ms of CA parsing per client, and the API client is built once instead of on every inbound activity - which is what restores TCP connection reuse.

Problem

Client.__init__ always builds its own httpx.AsyncClient, which constructs a fresh ssl.SSLContext - parsing the full CA bundle (~150 certs) every time.

  • ssl.create_default_context() - 2.37ms
  • httpx.AsyncClient() - 2.45ms (almost entirely the line above)

Two consequences:

  1. No way to avoid it. ClientOptions had no hook for supplying a context, and clone() rebuilt from options only.
  2. We do it more than we need to. ActivityProcessor._build_context cloned the http client on every inbound activity, even though the clone was identical each time: token_manager.get_bot_token is the same bound method, takes no arguments, and is resolved per request anyway. Every activity therefore got a brand-new connection pool, so no outbound call to Teams ever reused a TCP connection.

Changes

ClientOptions.verify: Optional[ssl.SSLContext] - opt-in. Callers may pass a pre-built context; clone() propagates it.

ActivityProcessor - build the API http client once in __init__ rather than per activity.

Impact

before after
100 clients w/ per-turn token 303ms 3.8ms
20 sequential API calls 176ms / 20 conns 9.9ms / 1 conn

Why verify= and not transport=

Passing a shared transport also skips SSL setup, but silently disables httpx's env-based proxy detection and collapses pool isolation. verify= keeps both.

proxy honored own pool shared SSL ctx
default yes yes no
verify=ctx yes yes yes
transport= no no yes

Compatibility

Default behaviour is unchanged - verify defaults to None, which maps to httpx's own True. The field is typed Optional[ssl.SSLContext], deliberately not accepting bool, so verify=False can't be smuggled through to disable TLS verification.

Reviewer caveats

  • Handler mutations now persist. ctx.api.http.use_interceptor(...) used to be discarded with the per-activity clone; it now accumulates across activities. Nothing in the SDK or examples calls it, but it is a real semantic change.
  • A cached context is a snapshot of the CA bundle; long-lived processes should rebuild it to pick up CA rotation. This is why it's opt-in rather than default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant