Trace-level LLM (GenAI) enrichment for OpenRouter OTel spans#202
Open
Makisuo wants to merge 1 commit into
Open
Trace-level LLM (GenAI) enrichment for OpenRouter OTel spans#202Makisuo wants to merge 1 commit into
Makisuo wants to merge 1 commit into
Conversation
OpenRouter Broadcast already ingests into Maple via /v1/traces using the OTel gen_ai.* semantic conventions, but those attributes landed in the generic SpanAttributes map with no LLM-aware rendering (roadmap #13). This makes every LLM span read as a first-class LLM call across the trace views and span detail. - Project the gen_ai.* keys into TREE_SPAN_ATTR_KEYS so the waterfall/timeline/ flow views can badge LLM spans without the per-span lazy detail fetch. - Add getGenAiInfo (packages/ui/src/lib/gen-ai.ts) normalizing gen_ai.* into a single shape, coalescing gen_ai.system -> gen_ai.provider.name and gen_ai.response.model -> gen_ai.request.model, and tolerating OpenRouter Privacy Mode (absent prompt/completion/finish reasons). - Add a genAiAdapter to the cloud-platform registry so the sparkle/provider badge lights up across every trace view and the span-detail summary block (provider, operation, model, tokens, cost, params, finish reason) with no component edits. - Coalesce gen_ai.system <-> gen_ai.provider.name in attribute filters (traces-shared.ts) so a provider filter matches either spelling. - Format gen_ai model/token/cost values in the attribute table. - Add SparklesIcon; unit tests for the detector, adapter, formatters, the provider-alias SQL, and the tree projection; update the OpenRouter tracing doc and roadmap #13. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdSppGz2eTovugvSVRQeBq
Contributor
|
Your Pullfrog Router balance is exhausted. You have a card on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer. Top up balance → · Enable auto-reload →
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Context
OpenRouter's Broadcast feature already ingests into Maple: it POSTs OTLP/HTTP-JSON traces to
/v1/traces(documented indocs/openrouter-tracing.md) using the OTelgen_ai.*semantic conventions (gen_ai.operation.name,gen_ai.provider.name/legacygen_ai.system,gen_ai.request.model,gen_ai.response.model,gen_ai.usage.input_tokens/output_tokens,gen_ai.usage.cost,gen_ai.response.finish_reasons). But those attributes landed in the genericSpanAttributesmap with no LLM-aware rendering — this is roadmap item #13.This PR closes that gap for trace-level LLM enrichment: open any trace with OpenRouter/LLM spans and immediately see which spans are LLM calls, on what model/provider, how many tokens, and at what cost — with prompts/completions inspectable inline and graceful handling of OpenRouter Privacy Mode. It is pure query-engine projection + frontend work — no ingest or storage changes.
What changed
gen_ai.*keys toTREE_SPAN_ATTR_KEYS(packages/query-engine/src/ch/queries/errors.ts) so the waterfall/timeline/flow views can badge LLM spans without the per-span lazy detail fetch.getGenAiInfo(packages/ui/src/lib/gen-ai.ts) normalizesgen_ai.*into one shape, coalescinggen_ai.system→gen_ai.provider.nameandgen_ai.response.model→gen_ai.request.model, and tolerating Privacy Mode (absent prompt/completion/finish reasons). Detection keys on a real non-empty value, never key-presence (the trimmed-projection footgun).genAiAdapterin the cloud-platform registry (packages/ui/src/lib/cloud-platforms/gen-ai.ts) lights up the sparkle/provider badge across every trace view and the span-detail summary block (provider, operation, model, tokens, cost, params, finish reason) with no component edits, since both are driven bygetCloudPlatform.gen_ai.system↔gen_ai.provider.namenow coalesce in attribute filters (packages/query-engine/src/traces-shared.ts), so a provider filter matches either spelling.gen_ai.request.model/response.modelrender as model chips, token counts are grouped, andgen_ai.usage.costrenders as currency in the attribute table (apps/web/src/components/attributes/commit-sha-attribute.tsx).SparklesIcon; updateddocs/openrouter-tracing.mdand roadmap Alerts #13.Testing
packages/ui— 14 new unit tests for the detector, adapter, and formatters (full OpenRouter span, provider coalescing, empty-value rejection, Privacy Mode, finish-reason parsing, bad-outcome flagging, no shadowing of DB/non-LLM spans). All 103 ui-lib tests pass.packages/query-engine— extended the span-hierarchy projection test to assert thegen_ai.*keys, and added atracesTimeseriesQuerytest asserting thegen_ai.provider.name/gen_ai.systemcoalescing compiles. All 105 ch tests pass.turbo typecheckclean across@maple/ui,@maple/query-engine,@maple/web;oxfmt+oxlintclean.gen_aispan → view enrichment); the pure logic is unit-covered and the React integration is typecheck-covered.Follow-ups (deferred)
Aggregate "AI-first ops" surfaces — a dedicated AI page, token/cost dashboards + numeric-attribute aggregation, a
gen_ai_usage_hourlyrollup, MCP tools, and cost/budget alerts — are scoped in the plan for a phased follow-up (query-time first, typed rollup MV as a fast-follow).🤖 Generated with Claude Code
https://claude.ai/code/session_01QdSppGz2eTovugvSVRQeBq
Generated by Claude Code