Skip to content

Add Novita AI provider support#697

Open
Alex-yang00 wants to merge 2 commits into
Zoo-Code-Org:mainfrom
Alex-yang00:add-novita-ai-provider
Open

Add Novita AI provider support#697
Alex-yang00 wants to merge 2 commits into
Zoo-Code-Org:mainfrom
Alex-yang00:add-novita-ai-provider

Conversation

@Alex-yang00

@Alex-yang00 Alex-yang00 commented Jun 23, 2026

Copy link
Copy Markdown

Summary

  • Add Novita AI as a first-class OpenAI-compatible LLM provider
  • Wire Novita into provider settings, model metadata, API handler, Settings UI, validation, secret storage, and selected-model handling
  • Add CLI support via --provider novita and NOVITA_API_KEY
  • Add Novita docs wording and e2e fixture/test coverage

Testing

  • git diff --check
  • vitest packages/types/src/__tests__/provider-settings.test.ts
  • vitest apps/cli/src/lib/utils/__tests__/provider.test.ts
  • vitest webview-ui/src/utils/__tests__/validate.spec.ts
  • vitest src/shared/__tests__/ProfileValidator.spec.ts src/api/providers/__tests__/novita.spec.ts
  • tsc -p apps/vscode-e2e/tsconfig.esm.json --noEmit
  • Live Novita API smoke: models, chat completion, and tool calling passed with moonshotai/kimi-k2.7-code

Summary by CodeRabbit

Release Notes

  • New Features
    • Added Novita AI as a new provider option for running models and building AI agents.
    • Added Novita API key and base URL configuration to the settings UI.
  • Documentation
    • Updated the main release notes and expanded CLI/VS Code docs with Novita setup details and example commands.
  • Tests
    • Added CLI, API handler, configuration validation, and VS Code end-to-end coverage for Novita (including fixtures in mock record mode).
  • Localization
    • Added Novita UI labels/tooling text across multiple languages.

@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 08617c86-f651-41a6-aa19-74a30e8be295

📥 Commits

Reviewing files that changed from the base of the PR and between 9024681 and dbb482a.

📒 Files selected for processing (24)
  • apps/cli/README.md
  • apps/cli/src/lib/utils/__tests__/provider.test.ts
  • apps/vscode-e2e/fixtures/novita.json
  • apps/vscode-e2e/src/suite/providers/novita.test.ts
  • src/api/providers/__tests__/novita.spec.ts
  • src/api/providers/novita.ts
  • webview-ui/src/components/settings/providers/__tests__/Novita.spec.tsx
  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
  • webview-ui/src/i18n/locales/ca/settings.json
  • webview-ui/src/i18n/locales/de/settings.json
  • webview-ui/src/i18n/locales/es/settings.json
  • webview-ui/src/i18n/locales/fr/settings.json
  • webview-ui/src/i18n/locales/hi/settings.json
  • webview-ui/src/i18n/locales/id/settings.json
  • webview-ui/src/i18n/locales/it/settings.json
  • webview-ui/src/i18n/locales/ja/settings.json
  • webview-ui/src/i18n/locales/ko/settings.json
  • webview-ui/src/i18n/locales/nl/settings.json
  • webview-ui/src/i18n/locales/pl/settings.json
  • webview-ui/src/i18n/locales/pt-BR/settings.json
  • webview-ui/src/i18n/locales/ru/settings.json
  • webview-ui/src/i18n/locales/tr/settings.json
  • webview-ui/src/i18n/locales/vi/settings.json
  • webview-ui/src/i18n/locales/zh-TW/settings.json
✅ Files skipped from review due to trivial changes (18)
  • webview-ui/src/i18n/locales/nl/settings.json
  • webview-ui/src/i18n/locales/es/settings.json
  • webview-ui/src/i18n/locales/ja/settings.json
  • webview-ui/src/i18n/locales/pl/settings.json
  • webview-ui/src/i18n/locales/pt-BR/settings.json
  • webview-ui/src/i18n/locales/ca/settings.json
  • webview-ui/src/i18n/locales/hi/settings.json
  • webview-ui/src/i18n/locales/ko/settings.json
  • webview-ui/src/i18n/locales/de/settings.json
  • webview-ui/src/i18n/locales/id/settings.json
  • apps/cli/README.md
  • webview-ui/src/i18n/locales/fr/settings.json
  • webview-ui/src/i18n/locales/tr/settings.json
  • webview-ui/src/i18n/locales/ru/settings.json
  • webview-ui/src/i18n/locales/it/settings.json
  • webview-ui/src/components/ui/hooks/tests/useSelectedModel.spec.ts
  • webview-ui/src/i18n/locales/vi/settings.json
  • webview-ui/src/i18n/locales/zh-TW/settings.json
🚧 Files skipped from review as they are similar to previous changes (6)
  • apps/cli/src/lib/utils/tests/provider.test.ts
  • apps/vscode-e2e/fixtures/novita.json
  • src/api/providers/tests/novita.spec.ts
  • src/api/providers/novita.ts
  • webview-ui/src/components/settings/providers/tests/Novita.spec.tsx
  • apps/vscode-e2e/src/suite/providers/novita.test.ts

📝 Walkthrough

Walkthrough

This PR integrates Novita AI as a new provider across the full stack: type definitions (model registry, Zod schemas, secrets), API handler implementation and routing, CLI configuration and documentation, webview settings UI with 18-locale translations and validation, component unit tests, and a comprehensive VSCode E2E test suite with mock fixtures.

Changes

Novita AI Provider Integration

Layer / File(s) Summary
Novita model registry and provider type contracts
packages/types/src/providers/novita.ts, packages/types/src/provider-settings.ts, packages/types/src/global-settings.ts, packages/types/src/providers/index.ts, packages/types/src/__tests__/provider-settings.test.ts
Defines NovitaModelId, novitaModels (4 models with full ModelInfo), novitaDefaultModelId/Info. Adds novitaApiKey to SECRET_STATE_KEYS, extends providerNames, defines novitaSchema (Zod), wires it into discriminated union, providerSettingsSchema, modelIdKeysByProvider, and MODELS_BY_PROVIDER. Re-exports from providers index and extends getProviderDefaultModelId. Tests assert protocol and default model ID.
NovitaHandler implementation and routing
src/api/providers/novita.ts, src/api/providers/index.ts, src/api/index.ts, src/shared/ProfileValidator.ts, src/shared/__tests__/ProfileValidator.spec.ts, src/api/providers/__tests__/novita.spec.ts
Adds NovitaHandler extending OpenAICompatibleHandler with constructor building OpenAICompatibleConfig and getModel() override. Wires into buildApiHandler switch and ProfileValidator.getModelIdFromProfile. Unit tests cover explicit options, default fallback, unknown model ID, and custom model parameters.
CLI provider support and documentation
apps/cli/src/types/types.ts, apps/cli/src/lib/utils/provider.ts, apps/cli/src/lib/utils/__tests__/provider.test.ts, apps/cli/README.md
Adds novita to supportedProviders, maps NOVITA_API_KEY in envVarMap, adds getProviderSettings novita case. CLI README documents setup instructions, environment variables, and pnpm dev examples. Unit tests verify getApiKeyFromEnv and getProviderSettings behavior.
Novita settings component and wiring
webview-ui/src/components/settings/providers/Novita.tsx, webview-ui/src/components/settings/providers/__tests__/Novita.spec.tsx, webview-ui/src/components/settings/providers/index.ts, webview-ui/src/components/settings/ApiOptions.tsx
Adds Novita.tsx component with base URL and API key fields, conditional API key management link, and unit test coverage for empty and populated configurations. Exports from providers index and renders conditionally in ApiOptions.tsx when provider is novita.
Webview UI provider registries and model selection
webview-ui/src/components/settings/constants.ts, webview-ui/src/components/settings/utils/providerModelConfig.ts, webview-ui/src/components/ui/hooks/useSelectedModel.ts, webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
Extends PROVIDERS, MODELS_BY_PROVIDER, PROVIDER_SERVICE_CONFIG, PROVIDER_DEFAULT_MODEL_IDS, PROVIDER_MODEL_CONFIG. Adds novita case to useSelectedModel for model info lookup from novitaModels. Includes unit test coverage for default and configured model cases.
Webview validation and i18n support
webview-ui/src/utils/validate.ts, webview-ui/src/utils/__tests__/validate.spec.ts, webview-ui/src/i18n/locales/*/settings.json
Adds novitaApiKey validation gate in validateModelsAndKeysProvided. Validation test covers missing and present API key. Adds i18n strings across 18 locales (Catalan, German, English, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Russian, Turkish, Vietnamese, Simplified Chinese, Traditional Chinese) with API key label, get API key label, and base URL label.
VSCode E2E test suite with fixture and aimock wiring
apps/vscode-e2e/src/suite/providers/novita.test.ts, apps/vscode-e2e/fixtures/novita.json, apps/vscode-e2e/src/runTest.ts, apps/vscode-e2e/AGENTS.md
Adds novita.json fixture with chained read_fileattempt_completion → error-retry tool-call flow. Extends runTest.ts with isNovitaTargetedRun() detection, record-mode API key guard, and aimock URL routing to https://api.novita.ai/openai. Full E2E suite includes fetch interception, runNovitaToolProbe orchestration, and assertions on model identity, completion status, marker text, and tool usage. Documents fixture workflow.
Release notes
README.md
Adds Novita AI support bullet to v3.62.0 What's New section.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • Zoo-Code-Org/Zoo-Code#319: Extends packages/types provider plumbing (SECRET_STATE_KEYS, provider-settings discriminated schemas) to add Opencode Go provider branch, parallel to Novita addition.
  • Zoo-Code-Org/Zoo-Code#344: Modifies the same provider-registry plumbing in packages/types/src/provider-settings.ts by adding a new provider branch (zoo-gateway vs novita) into shared schema/lookup structures.
  • Zoo-Code-Org/Zoo-Code#345: Extends webview-ui/src/utils/validate.ts provider-switch logic by adding zoo-gateway validation, similar pattern to novita addition.

Suggested labels

awaiting-review

Suggested reviewers

  • JamesRobert20
  • navedmerchant
  • hannesrudolph
  • taltas

Poem

🐇 Hop hop, a new platform joins the fold,
Novita AI, fast and bold!
novitaModels defined with care,
API keys floating through the air.
The rabbit wired each handler tight—
E2E fixtures shining bright! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 6.25% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding Novita AI provider support, which is the primary focus of all file modifications.
Description check ✅ Passed The description includes a summary of changes, testing steps with specific command examples, and validation details, covering the main implementation aspects.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.81818% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
apps/cli/src/lib/utils/provider.ts 33.33% 0 Missing and 2 partials ⚠️
src/api/providers/novita.ts 84.61% 0 Missing and 2 partials ⚠️
src/api/index.ts 0.00% 1 Missing ⚠️
webview-ui/src/components/settings/ApiOptions.tsx 0.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 17

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/cli/README.md`:
- Around line 79-80: The example documentation for Novita AI configuration is
incomplete because it exports the NOVITA_API_KEY environment variable but the
accompanying command example does not include the --provider novita flag
required to actually use Novita as the provider. Add the --provider novita flag
to the example command that follows the NOVITA_API_KEY export statement so users
understand they must specify the provider to use the Novita API key.

In `@webview-ui/src/i18n/locales/ca/settings.json`:
- Around line 411-413: The Catalan locale file contains three untranslated
English strings for Novita AI settings that need to be localized. Replace the
English values for the keys novitaApiKey, getNovitaApiKey, and novitaBaseUrl
with their proper Catalan translations to ensure Catalan users see consistent
localized UI text instead of mixed-language content.

In `@webview-ui/src/i18n/locales/de/settings.json`:
- Around line 411-413: The three Novita AI related strings in the German locale
file are currently in English instead of German. Translate the values for the
keys novitaApiKey, getNovitaApiKey, and novitaBaseUrl from English to their
appropriate German equivalents to ensure the German UI displays properly
localized labels.

In `@webview-ui/src/i18n/locales/es/settings.json`:
- Around line 411-413: The three Novita provider settings strings novitaApiKey,
getNovitaApiKey, and novitaBaseUrl in the Spanish localization file have English
values instead of Spanish translations. Translate each of these string values
from English to their appropriate Spanish equivalents to maintain consistent
localization across the settings.

In `@webview-ui/src/i18n/locales/fr/settings.json`:
- Around line 411-413: The French locale file contains three untranslated
English strings for Novita AI settings. In the fr/settings.json file, locate the
properties "novitaApiKey", "getNovitaApiKey", and "novitaBaseUrl" and replace
their English values with appropriate French translations. Ensure the
translations are contextually appropriate and consistent with the rest of the
French localization in the file.

In `@webview-ui/src/i18n/locales/hi/settings.json`:
- Around line 411-413: The three Novita provider strings (novitaApiKey,
getNovitaApiKey, and novitaBaseUrl) in the Hindi settings.json file are still in
English while neighboring provider entries are properly localized to Hindi.
Translate these three string values from English to their appropriate Hindi
equivalents to maintain consistency with the rest of the localized content in
the file.

In `@webview-ui/src/i18n/locales/id/settings.json`:
- Around line 411-413: The Indonesian locale file contains three Novita AI
settings labels (novitaApiKey, getNovitaApiKey, and novitaBaseUrl) with English
text values instead of Indonesian translations. Translate these three label
values from English to Indonesian to ensure consistency throughout the
Indonesian locale file and provide a cohesive user interface experience for
Indonesian users.

In `@webview-ui/src/i18n/locales/it/settings.json`:
- Around line 411-413: The values for the Novita provider keys (novitaApiKey,
getNovitaApiKey, and novitaBaseUrl) in the Italian locale file are currently in
English instead of Italian. Translate these three key-value pairs to Italian to
maintain consistency with the rest of the Italian settings locale file. Replace
the English text values for each of these keys with appropriate Italian
translations while keeping the key names unchanged.

In `@webview-ui/src/i18n/locales/ja/settings.json`:
- Around line 411-413: The Novita AI configuration strings (novitaApiKey,
getNovitaApiKey, and novitaBaseUrl) in the Japanese locale file
(ja/settings.json) are currently in English, which creates an inconsistent user
experience. Translate these three string values to Japanese to match the rest of
the Japanese locale while keeping the property keys unchanged.

In `@webview-ui/src/i18n/locales/ko/settings.json`:
- Around line 411-413: The three Novita provider label strings in the Korean
locale file have values in English instead of Korean, creating mixed-language
content. Translate the values for the keys novitaApiKey, getNovitaApiKey, and
novitaBaseUrl from English to Korean to maintain consistency with the rest of
the Korean settings locale file.

In `@webview-ui/src/i18n/locales/nl/settings.json`:
- Around line 411-413: Translate the three Novita AI related entries in the
Dutch locale file to Dutch. The keys "novitaApiKey", "getNovitaApiKey", and
"novitaBaseUrl" currently have English values in the nl/settings.json file, but
should be translated to Dutch to maintain consistency with the locale and
provide a seamless user experience for Dutch-speaking users. Replace each
English value with its appropriate Dutch translation.

In `@webview-ui/src/i18n/locales/pl/settings.json`:
- Around line 411-413: The Novita provider entries (novitaApiKey,
getNovitaApiKey, and novitaBaseUrl) in the Polish locale file are currently in
English, breaking locale consistency. Translate the values for these three
entries from English to Polish to maintain consistency with the rest of the
Polish settings locale file.

In `@webview-ui/src/i18n/locales/pt-BR/settings.json`:
- Around line 411-413: The Novita AI label values in the pt-BR locale file are
currently in English instead of Portuguese, causing mixed language display.
Translate the values for the keys novitaApiKey, getNovitaApiKey, and
novitaBaseUrl from English to their Portuguese (Brazil) equivalents to maintain
consistent localization throughout the settings interface.

In `@webview-ui/src/i18n/locales/ru/settings.json`:
- Around line 411-413: The Russian locale file (ru/settings.json) contains
English strings for the Novita provider labels in keys "novitaApiKey",
"getNovitaApiKey", and "novitaBaseUrl". These values need to be translated to
Russian to maintain consistency with other localized strings in the Russian UI.
Replace each English value with its appropriate Russian translation for these
three keys.

In `@webview-ui/src/i18n/locales/tr/settings.json`:
- Around line 411-413: The three Novita-related entries (novitaApiKey,
getNovitaApiKey, and novitaBaseUrl) in the Turkish locale file are currently in
English while the rest of the file is in Turkish. Translate these three string
values to Turkish to maintain consistency with the rest of the locale and
provide a proper localized user interface.

In `@webview-ui/src/i18n/locales/vi/settings.json`:
- Around line 411-413: The properties novitaApiKey, getNovitaApiKey, and
novitaBaseUrl in the Vietnamese locale file have English values instead of
Vietnamese translations. Translate each of these three property values from
English to their appropriate Vietnamese equivalents to maintain consistency with
the rest of the Vietnamese locale and ensure the settings panel displays
properly in Vietnamese.

In `@webview-ui/src/i18n/locales/zh-TW/settings.json`:
- Around line 438-440: In the zh-TW/settings.json locale file, the three new
Novita-related keys (novitaApiKey, getNovitaApiKey, and novitaBaseUrl) currently
have English string values instead of Traditional Chinese translations. Replace
the English values for each of these three keys with their appropriate
Traditional Chinese translations to maintain consistency with the rest of the
Traditional Chinese locale file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3bf02952-44d5-4692-9d81-6ac83e9c7ce8

📥 Commits

Reviewing files that changed from the base of the PR and between e8acc6a and f761431.

📒 Files selected for processing (46)
  • README.md
  • apps/cli/README.md
  • apps/cli/src/lib/utils/__tests__/provider.test.ts
  • apps/cli/src/lib/utils/provider.ts
  • apps/cli/src/types/types.ts
  • apps/vscode-e2e/AGENTS.md
  • apps/vscode-e2e/fixtures/novita.json
  • apps/vscode-e2e/src/runTest.ts
  • apps/vscode-e2e/src/suite/providers/novita.test.ts
  • packages/types/src/__tests__/provider-settings.test.ts
  • packages/types/src/global-settings.ts
  • packages/types/src/provider-settings.ts
  • packages/types/src/providers/index.ts
  • packages/types/src/providers/novita.ts
  • src/api/index.ts
  • src/api/providers/__tests__/novita.spec.ts
  • src/api/providers/index.ts
  • src/api/providers/novita.ts
  • src/shared/ProfileValidator.ts
  • src/shared/__tests__/ProfileValidator.spec.ts
  • webview-ui/src/components/settings/ApiOptions.tsx
  • webview-ui/src/components/settings/constants.ts
  • webview-ui/src/components/settings/providers/Novita.tsx
  • webview-ui/src/components/settings/providers/index.ts
  • webview-ui/src/components/settings/utils/providerModelConfig.ts
  • webview-ui/src/components/ui/hooks/useSelectedModel.ts
  • webview-ui/src/i18n/locales/ca/settings.json
  • webview-ui/src/i18n/locales/de/settings.json
  • webview-ui/src/i18n/locales/en/settings.json
  • webview-ui/src/i18n/locales/es/settings.json
  • webview-ui/src/i18n/locales/fr/settings.json
  • webview-ui/src/i18n/locales/hi/settings.json
  • webview-ui/src/i18n/locales/id/settings.json
  • webview-ui/src/i18n/locales/it/settings.json
  • webview-ui/src/i18n/locales/ja/settings.json
  • webview-ui/src/i18n/locales/ko/settings.json
  • webview-ui/src/i18n/locales/nl/settings.json
  • webview-ui/src/i18n/locales/pl/settings.json
  • webview-ui/src/i18n/locales/pt-BR/settings.json
  • webview-ui/src/i18n/locales/ru/settings.json
  • webview-ui/src/i18n/locales/tr/settings.json
  • webview-ui/src/i18n/locales/vi/settings.json
  • webview-ui/src/i18n/locales/zh-CN/settings.json
  • webview-ui/src/i18n/locales/zh-TW/settings.json
  • webview-ui/src/utils/__tests__/validate.spec.ts
  • webview-ui/src/utils/validate.ts

Comment thread apps/cli/README.md
Comment thread webview-ui/src/i18n/locales/ca/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/de/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/es/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/fr/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/pt-BR/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/ru/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/tr/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/vi/settings.json Outdated
Comment thread webview-ui/src/i18n/locales/zh-TW/settings.json Outdated
@Alex-yang00 Alex-yang00 force-pushed the add-novita-ai-provider branch from f761431 to 9024681 Compare June 23, 2026 08:05
@Alex-yang00 Alex-yang00 force-pushed the add-novita-ai-provider branch from 9024681 to dbb482a Compare June 23, 2026 08:14

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/vscode-e2e/src/suite/providers/novita.test.ts`:
- Around line 184-187: The setConfiguration call for the novita provider needs
to guard against undefined API keys when in record mode with AIMOCK_URL. The
current logic uses NOVITA_API_KEY! without ensuring it's defined; update the
novitaApiKey assignment to check that NOVITA_API_KEY exists before using it, or
skip the configuration entirely if it's absent. Additionally, apply the same
defensive check to any teardown configuration that uses OPENROUTER_API_KEY to
avoid passing undefined environment variables into the teardown.
- Around line 323-326: The assertion in the Novita test is too permissive
because it allows result.completionText to be missing or empty via the
`!result.completionText ||` condition. Remove the `!result.completionText ||`
portion and require that result.completionText strictly equals the marker
variable, ensuring the test validates that the expected final marker is actually
returned instead of passing when the completion is missing.
- Around line 247-249: The filter condition for requests in the
requests.filter() call is too permissive by allowing requests without a probeTag
through the condition `!request.probeTag || request.probeTag === probeTag`. This
can include stale or background requests from previous tests. Remove the
`!request.probeTag ||` part from the filter condition so that only requests with
a probeTag matching the current probeTag are included, changing the condition to
strictly check `request.model === modelId && request.probeTag === probeTag`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7691b242-0238-4ad8-8b3b-b4ee575ef110

📥 Commits

Reviewing files that changed from the base of the PR and between f761431 and 9024681.

📒 Files selected for processing (48)
  • README.md
  • apps/cli/README.md
  • apps/cli/src/lib/utils/__tests__/provider.test.ts
  • apps/cli/src/lib/utils/provider.ts
  • apps/cli/src/types/types.ts
  • apps/vscode-e2e/AGENTS.md
  • apps/vscode-e2e/fixtures/novita.json
  • apps/vscode-e2e/src/runTest.ts
  • apps/vscode-e2e/src/suite/providers/novita.test.ts
  • packages/types/src/__tests__/provider-settings.test.ts
  • packages/types/src/global-settings.ts
  • packages/types/src/provider-settings.ts
  • packages/types/src/providers/index.ts
  • packages/types/src/providers/novita.ts
  • src/api/index.ts
  • src/api/providers/__tests__/novita.spec.ts
  • src/api/providers/index.ts
  • src/api/providers/novita.ts
  • src/shared/ProfileValidator.ts
  • src/shared/__tests__/ProfileValidator.spec.ts
  • webview-ui/src/components/settings/ApiOptions.tsx
  • webview-ui/src/components/settings/constants.ts
  • webview-ui/src/components/settings/providers/Novita.tsx
  • webview-ui/src/components/settings/providers/__tests__/Novita.spec.tsx
  • webview-ui/src/components/settings/providers/index.ts
  • webview-ui/src/components/settings/utils/providerModelConfig.ts
  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
  • webview-ui/src/components/ui/hooks/useSelectedModel.ts
  • webview-ui/src/i18n/locales/ca/settings.json
  • webview-ui/src/i18n/locales/de/settings.json
  • webview-ui/src/i18n/locales/en/settings.json
  • webview-ui/src/i18n/locales/es/settings.json
  • webview-ui/src/i18n/locales/fr/settings.json
  • webview-ui/src/i18n/locales/hi/settings.json
  • webview-ui/src/i18n/locales/id/settings.json
  • webview-ui/src/i18n/locales/it/settings.json
  • webview-ui/src/i18n/locales/ja/settings.json
  • webview-ui/src/i18n/locales/ko/settings.json
  • webview-ui/src/i18n/locales/nl/settings.json
  • webview-ui/src/i18n/locales/pl/settings.json
  • webview-ui/src/i18n/locales/pt-BR/settings.json
  • webview-ui/src/i18n/locales/ru/settings.json
  • webview-ui/src/i18n/locales/tr/settings.json
  • webview-ui/src/i18n/locales/vi/settings.json
  • webview-ui/src/i18n/locales/zh-CN/settings.json
  • webview-ui/src/i18n/locales/zh-TW/settings.json
  • webview-ui/src/utils/__tests__/validate.spec.ts
  • webview-ui/src/utils/validate.ts
✅ Files skipped from review due to trivial changes (22)
  • src/api/providers/index.ts
  • webview-ui/src/i18n/locales/nl/settings.json
  • webview-ui/src/i18n/locales/fr/settings.json
  • webview-ui/src/i18n/locales/ko/settings.json
  • webview-ui/src/i18n/locales/ca/settings.json
  • webview-ui/src/components/settings/providers/index.ts
  • webview-ui/src/i18n/locales/pt-BR/settings.json
  • webview-ui/src/i18n/locales/ja/settings.json
  • webview-ui/src/i18n/locales/zh-CN/settings.json
  • webview-ui/src/i18n/locales/hi/settings.json
  • webview-ui/src/i18n/locales/ru/settings.json
  • apps/cli/README.md
  • webview-ui/src/i18n/locales/pl/settings.json
  • webview-ui/src/i18n/locales/vi/settings.json
  • README.md
  • webview-ui/src/i18n/locales/zh-TW/settings.json
  • webview-ui/src/i18n/locales/it/settings.json
  • webview-ui/src/i18n/locales/en/settings.json
  • webview-ui/src/i18n/locales/es/settings.json
  • webview-ui/src/i18n/locales/de/settings.json
  • webview-ui/src/i18n/locales/id/settings.json
  • webview-ui/src/i18n/locales/tr/settings.json
🚧 Files skipped from review as they are similar to previous changes (21)
  • webview-ui/src/components/settings/constants.ts
  • webview-ui/src/utils/validate.ts
  • webview-ui/src/utils/tests/validate.spec.ts
  • packages/types/src/tests/provider-settings.test.ts
  • packages/types/src/providers/novita.ts
  • src/api/providers/tests/novita.spec.ts
  • packages/types/src/global-settings.ts
  • apps/vscode-e2e/src/runTest.ts
  • webview-ui/src/components/settings/ApiOptions.tsx
  • src/shared/ProfileValidator.ts
  • webview-ui/src/components/settings/providers/Novita.tsx
  • src/api/index.ts
  • src/shared/tests/ProfileValidator.spec.ts
  • apps/cli/src/lib/utils/provider.ts
  • packages/types/src/providers/index.ts
  • webview-ui/src/components/settings/utils/providerModelConfig.ts
  • webview-ui/src/components/ui/hooks/useSelectedModel.ts
  • src/api/providers/novita.ts
  • packages/types/src/provider-settings.ts
  • apps/cli/src/types/types.ts
  • apps/vscode-e2e/AGENTS.md

Comment on lines +184 to +187
await api.setConfiguration({
apiProvider: "novita" as const,
novitaApiKey: aimockUrl && !isRecord ? "mock-key" : NOVITA_API_KEY!,
...(aimockUrl && { novitaBaseUrl: `${aimockUrl}/v1` }),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Guard the record/live key paths before configuring providers.

Record mode with AIMOCK_URL still needs a real Novita key, but the current setup() does not skip that case and line 186 can pass undefined. Also avoid passing OPENROUTER_API_KEY! into teardown when the env var is absent.

Based on learnings, “Record mode uses record-on-miss: if an existing fixture already matches a request, aimock serves it and does not re-record. Only unmatched requests are proxied to the real API and saved as openai-*.json files.”

🛠️ Proposed fix
 setup(function () {
-	if (!process.env.AIMOCK_URL && !NOVITA_API_KEY) {
+	const needsRealNovitaKey = process.env.AIMOCK_RECORD === "true" || !process.env.AIMOCK_URL
+
+	if (needsRealNovitaKey && !NOVITA_API_KEY) {
 		this.skip()
 	}
 })
@@
 		const aimockUrl = process.env.AIMOCK_URL
 		const isRecord = process.env.AIMOCK_RECORD === "true"
+		const openRouterApiKey = aimockUrl
+			? isRecord
+				? (process.env.OPENROUTER_API_KEY ?? "mock-key")
+				: "mock-key"
+			: process.env.OPENROUTER_API_KEY
+
 		await globalThis.api.setConfiguration({
 			apiProvider: "openrouter" as const,
-			openRouterApiKey: aimockUrl
-				? isRecord
-					? (process.env.OPENROUTER_API_KEY ?? "mock-key")
-					: "mock-key"
-				: process.env.OPENROUTER_API_KEY!,
+			...(openRouterApiKey && { openRouterApiKey }),
 			openRouterModelId: "openai/gpt-4.1",
 			...(aimockUrl && { openRouterBaseUrl: `${aimockUrl}/v1` }),
 		})

Also applies to: 277-305

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/vscode-e2e/src/suite/providers/novita.test.ts` around lines 184 - 187,
The setConfiguration call for the novita provider needs to guard against
undefined API keys when in record mode with AIMOCK_URL. The current logic uses
NOVITA_API_KEY! without ensuring it's defined; update the novitaApiKey
assignment to check that NOVITA_API_KEY exists before using it, or skip the
configuration entirely if it's absent. Additionally, apply the same defensive
check to any teardown configuration that uses OPENROUTER_API_KEY to avoid
passing undefined environment variables into the teardown.

Source: Learnings

Comment on lines +247 to +249
requests: requests.filter(
(request) => request.model === modelId && (!request.probeTag || request.probeTag === probeTag),
),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Filter captured requests strictly to this probe tag.

Allowing !request.probeTag can include untagged background or stale same-model requests in firstRequest. Since the prompt injects novita-e2e:tool-use, keep assertions scoped to that tag.

As per coding guidelines, “In provider tests using fetch interceptors, scope request-shape assertions to the current probe or test tag only; do not pull in older requests from previous tests.”

🛠️ Proposed fix
-				requests: requests.filter(
-					(request) => request.model === modelId && (!request.probeTag || request.probeTag === probeTag),
-				),
+				requests: requests.filter((request) => request.model === modelId && request.probeTag === probeTag),
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/vscode-e2e/src/suite/providers/novita.test.ts` around lines 247 - 249,
The filter condition for requests in the requests.filter() call is too
permissive by allowing requests without a probeTag through the condition
`!request.probeTag || request.probeTag === probeTag`. This can include stale or
background requests from previous tests. Remove the `!request.probeTag ||` part
from the filter condition so that only requests with a probeTag matching the
current probeTag are included, changing the condition to strictly check
`request.model === modelId && request.probeTag === probeTag`.

Source: Coding guidelines

Comment on lines +323 to +326
assert.ok(
!result.completionText || result.completionText === marker,
`Novita should not return an incorrect marker.\n${diagnostics}`,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Require the final marker instead of allowing a missing completion.

!result.completionText || ... lets the test pass without verifying the expected final answer. The probe asks for the exact marker, so assert that directly.

🛠️ Proposed fix
-		assert.ok(
-			!result.completionText || result.completionText === marker,
-			`Novita should not return an incorrect marker.\n${diagnostics}`,
-		)
+		assert.strictEqual(
+			result.completionText,
+			marker,
+			`Novita should return the exact marker.\n${diagnostics}`,
+		)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/vscode-e2e/src/suite/providers/novita.test.ts` around lines 323 - 326,
The assertion in the Novita test is too permissive because it allows
result.completionText to be missing or empty via the `!result.completionText ||`
condition. Remove the `!result.completionText ||` portion and require that
result.completionText strictly equals the marker variable, ensuring the test
validates that the expected final marker is actually returned instead of passing
when the completion is missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant