feat: Add LiteLLM provider instrumentation for cross-provider LLM tracking#1410
Closed
icohangar-ops wants to merge 1 commit into
Closed
feat: Add LiteLLM provider instrumentation for cross-provider LLM tracking#1410icohangar-ops wants to merge 1 commit into
icohangar-ops wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a new
litellmprovider instrumentation module to AgentOps that wrapslitellm.completion()andlitellm.acompletion()directly at the LiteLLM entry point. This ensures LLM calls are tracked regardless of which underlying provider LiteLLM routes to.Problem
When using LiteLLM with
litellm.completion(model="anthropic/claude-3-5-sonnet-..."), LiteLLM uses its own internal Anthropic handler that makes direct HTTP calls. It does NOT call the officialanthropicPython SDK, so AgentOps' Anthropic instrumentor cannot capture those calls. The same issue affects other non-OpenAI providers like Bedrock, Gemini, Cohere, etc.For OpenAI models, this works because LiteLLM internally uses the
openaiPython SDK, which AgentOps' OpenAI instrumentor wraps successfully.Solution
Add LiteLLM as its own provider in AgentOps' instrumentation system. Instead of wrapping individual provider SDKs, the LiteLLM instrumentor wraps the LiteLLM entry points (
completion()andacompletion()) directly, using attribute extraction that handles:"anthropic/..."→ provider="anthropic")/v1/responsesresponse structureFiles Changed
agentops/instrumentation/providers/litellm/__init__.py- Module initagentops/instrumentation/providers/litellm/attributes.py- Attribute extraction for LiteLLM requests/responsesagentops/instrumentation/providers/litellm/instrumentor.py- Instrumentor class with sync/async wrappersagentops/instrumentation/__init__.py- Registerlitellmin PROVIDERS dictRelated Issues
Fixes #1079