feat(routing): add evidence-based smart modes#409
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Warning Review limit reached
Next review available in: 56 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (30)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1acca4cb2a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| body: JSON.stringify(postBody), | ||
| body: JSON.stringify({ | ||
| ...postBody, | ||
| provider: selectedSubset ? { ...postBody.provider, selectedModels: selectedSubset } : postBody.provider, |
There was a problem hiding this comment.
When a user edits the Provider name before saving a preset with static discovery, such as github-models or cloudflare-ai, the /api/providers save path is keyed by that edited name, so registry enrichment no longer copies the preset's models and liveModels:false. This branch only attaches selectedModels for a partial selection and sends neither field when all discovered static models remain selected, causing the saved renamed provider to fall back to live /models discovery instead of exposing the static catalog. Include the preset's static catalog fields in the payload or pass the preset id so the server can enrich by the selected preset rather than the final name.
Useful? React with 👍 / 👎.
| @@ -684,6 +711,37 @@ export default function Models({ apiBase }: { apiBase: string }) { | |||
| <span>{t("models.orderHint")}</span> | |||
There was a problem hiding this comment.
This adds a user-facing Smart routing card that creates auto-* combo entries, but this commit does not include any docs-site/ update explaining what the modes do, what config they write, or how users should edit/disable the generated combos afterward. Add docs coverage alongside the UI/API change so the documented behavior stays in sync.
AGENTS.md reference: AGENTS.md:L78-L79
Useful? React with 👍 / 👎.
| baseUrlChoices: [ | ||
| { id: "china-mainland", label: "China mainland", baseUrl: "https://api.siliconflow.cn/v1" }, | ||
| { id: "international", label: "International", baseUrl: "https://api.siliconflow.com/v1" }, | ||
| ], |
There was a problem hiding this comment.
Selecting the new international SiliconFlow endpoint saves https://api.siliconflow.com/v1, but routed provider config only honors saved base URLs for registry entries with allowBaseUrlOverride or a template base URL. Because this entry adds baseUrlChoices without that flag, runtime routing rewrites the saved international choice back to the default China endpoint, so users cannot actually route through the endpoint they selected. Set allowBaseUrlOverride: true or otherwise make these official choices honored at route time.
Useful? React with 👍 / 👎.
| if (!mode) { | ||
| return jsonResponse({ error: `mode must be one of: ${SMART_ROUTING_MODES.join(", ")}` }, 400); | ||
| } | ||
| const models = filterCatalogVisibleModels(await fetchAllModels(config), config); |
There was a problem hiding this comment.
When a configured OAuth provider is logged out, fetchAllModels still returns its configured/static models for catalog visibility, and this endpoint passes those rows directly into buildSmartRoutingCombo. That can generate an auto-* combo containing Anthropic/Kimi/etc. targets that immediately fail requests with OAuthLoginRequiredError until the user logs in. Filter OAuth providers by usable login status before building the generated combo.
Useful? React with 👍 / 👎.
| const bestQuality = Math.max(0, ...scored.map(candidate => candidate.quality)); | ||
| const capable = mode === "cost" | ||
| ? scored.filter(candidate => candidate.cost !== undefined && candidate.quality >= Math.max(.48, bestQuality - .18)) |
There was a problem hiding this comment.
In cost mode, bestQuality is computed across all candidates, including models with unknown pricing that are excluded on the next line. If an unpriced frontier model is present, it can raise the quality cutoff enough to drop every priced cheap/local model and make cost mode return null even though verified-cost targets exist. Compute the cutoff from cost-eligible candidates, or apply the quality floor after excluding unpriced models.
Useful? React with 👍 / 👎.
| normalized.push({ | ||
| id, |
There was a problem hiding this comment.
For Google AI Studio, /v1beta/models includes embeddings and other non-chat models distinguished by fields such as supportedGenerationMethods, but this normalization accepts any row with a name and then drops those capability fields. Those ids are advertised to Codex even though the Google adapter always calls :generateContent, so selecting something like text-embedding-004 will fail at request time. Filter Google rows to generative methods before returning them.
Useful? React with 👍 / 👎.
| discoveryRequestRef.current += 1; | ||
| discoveryAbortRef.current?.abort(); | ||
| discoveryAbortRef.current = null; | ||
| setDiscoveryBusy(false); |
There was a problem hiding this comment.
After a successful model discovery, changing the endpoint, base URL, adapter, API key, or private-network toggle calls this invalidation helper, but it only aborts the in-flight request and leaves the old discoveredModels, selectedModels, source label, and default model in place. Users can then save an allowlist/default from the previous endpoint without rediscovering the new one. Clear the discovery state when invalidating due to input changes.
Useful? React with 👍 / 👎.
| const candidateUrl = new URL(suppliedBaseUrl); | ||
| const escapedPath = templateUrl.pathname.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); | ||
| const pathPattern = new RegExp(`^${escapedPath.replaceAll(marker, "[A-Za-z0-9_-]+")}/?$`); | ||
| matchesResolvedTemplate = candidateUrl.origin === templateUrl.origin |
There was a problem hiding this comment.
For presets whose placeholder is in the hostname, such as Azure OpenAI's https://{resource}.openai.azure.com/openai, the template origin contains the marker host, so a real resource URL can never satisfy this origin equality check. The new discovery endpoint therefore rejects every valid resolved Azure resource URL as not matching the trusted preset. Handle hostname placeholders explicitly or mark these presets as discovery-unsupported.
Useful? React with 👍 / 👎.
|
|
||
| function modelCost(config: OcxConfig, model: CatalogModel): number | undefined { | ||
| const provider = config.providers[model.provider]; | ||
| if (provider?.authMode === "local") return 0; |
There was a problem hiding this comment.
Local presets saved through the registry/UI omit authMode:"local", so this check does not recognize Ollama, vLLM, or LM Studio as zero-cost unless the user hand-edits the config. In cost mode those local models get undefined pricing and are excluded from routing, which is the opposite of the intended cheap/local preference. Check the registry auth kind or loopback destination as well as authMode.
Useful? React with 👍 / 👎.
Summary
Dependencies
Merge after #405 and #407.
Validation