fix(litellm): store bookkeeping span off-band, not in forwarded metadata

[jgreer013]

Summary

Fixes #6596 (Linear PY-2540).

With LiteLLMIntegration enabled, any call that passes caller metadata could crash during request serialization:

TypeError: Object of type Span is not JSON serializable

Root cause

_input_callback stored the live span via _get_metadata_dict(kwargs)["_sentry_span"] = span. litellm threads the caller's metadata dict through to litellm_params["metadata"] — it's the same dict object the caller passed — and some providers (e.g. Anthropic's /v1/messages passthrough) forward that dict into the outbound request body. So the live Span landed at request_body["metadata"]["_sentry_span"] and json.dumps(request_body) failed before the request was sent.

Separately, under send_default_pii=True + include_prompts=True, that span's gen_ai.request.messages holds the verbatim prompt, so any sink that serialized the injected metadata could leak prompt content to the provider.

Fix

Store the bookkeeping span off-band in a module-level registry keyed by litellm_call_id (a per-request UUID that is stable across the input/success/failure callbacks), falling back to the identity of the shared callback kwargs dict for direct callback invocations that omit it. The span no longer lives in any litellm-visible dict, so it can't be forwarded, serialized, or deep-copied by litellm — fixing both the crash and the prompt-leak vector. The registry entry is removed by the terminal success/failure callback (streaming success peeks and pops only on the final call).

Testing

• New regression test test_caller_metadata_stays_json_serializable asserts the forwarded metadata stays free of _sentry_span and JSON-serializable at the serialization moment, while the span is still recorded off-band. It fails on master and passes with this change.
• Full litellm suite: 160 passed (litellm v1.81.16), including streaming/async/failure paths.
• Regression test also green on the oldest (v1.77.7) and latest litellm matrix entries.
• mypy clean; ruff check + format clean.

Related (downstream, defense-in-depth)

litellm forwards/serializes this injected metadata into the provider body. Companion issue: BerriAI/litellm#30662. litellm has stripped such span objects in other logging paths (BerriAI/litellm#15728, BerriAI/litellm#12354).

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5a5f39b. Configure here.}

[alexander-alderman-webb]

Thanks for the PR!
I'll think about approaches because the _spans_by_call will grow if spans aren't finished correctly and everything is dropped if _MAX_TRACKED_SPANS is reached.

Generally, if there's a public shared reference between input and output callbacks, we should use that. If not, your approach could be the best we can do.

[jgreer013]

It seems like there isn't one in this case, but perhaps there's an opportunity to add one with a bigger rework?

[jgreer013]

Actually, turns out there is one — litellm threads the same model_call_details dict through all three callbacks, so I moved the span onto that (off the forwarded metadata path) instead of the module-level registry. No cap or eviction now, and the span's lifetime is tied to the request, like the clickhouse/dramatiq integrations do it. I double-checked the Anthropic passthrough doesn't forward it — the request body only gets the recognized params, not model_call_details. Lmk if you had a different reference in mind!