feat(claude): modernize model defaults and bound inline-completion output

[herikwebb]

Changes

• Chat fallback moves claude-sonnet-4-5 → claude-sonnet-5 (current Sonnet tier).
• Inline completion fallback moves to claude-haiku-4-5: a suggestion has to beat the user's next keystroke and fires on every pause in typing, so the fastest/cheapest tier is the right default.
• Inline completion max_tokens capped at 1024 (was 10000), overridable via NBI_CLAUDE_INLINE_COMPLETION_MAX_TOKENS. The old ceiling let a rambling response generate for many seconds before the extraction regex threw most of it away.
• fetch_claude_models prefers the Models API's max_input_tokens (validated as a positive int) over litellm's static database, which lags new model releases; litellm remains the fallback.
• 6 tests pinning the defaults, the bounded request, and the context-window preference.

Testing

Full Python suite passes (1298 tests).

[github-actions]

Automated PR Review

Model: gpt-5.5
Verdict: CHANGES_REQUESTED

Changed files:

notebook_intelligence/claude.py
tests/test_claude_models.py

Review result:

Automated Review Findings

Medium — notebook_intelligence/claude.py

CLAUDE_INLINE_COMPLETION_MAX_TOKENS is parsed directly from NBI_CLAUDE_INLINE_COMPLETION_MAX_TOKENS and then passed to Anthropic as max_tokens without validation.

This undermines the new bounded-autocomplete behavior:

• NBI_CLAUDE_INLINE_COMPLETION_MAX_TOKENS=10000 restores the old slow/costly behavior the PR is trying to prevent.
• 0 or negative values can cause API request failures.
• A non-integer value causes an import-time ValueError, potentially preventing the extension/backend from starting.

The added test only verifies the default value, not the configured path. Consider parsing defensively and enforcing a sane range, e.g. clamp or reject values outside 1..4096, with tests for invalid, too-low, and too-high environment values.

[github-actions]

Automated PR Review

Model: gpt-5.5
Verdict: CHANGES_REQUESTED

Changed files:

notebook_intelligence/claude.py
tests/test_claude_models.py

Review result:

Automated findings

High — notebook_intelligence/claude.py — Default chat model appears to be an unsupported Anthropic model ID

The fallback chat model was changed from claude-sonnet-4-5 to claude-sonnet-5:

CLAUDE_DEFAULT_CHAT_MODEL = "claude-sonnet-5"

When users have not configured a model, ClaudeChatModel.__init__ sends this string directly to messages.create. If the configured Anthropic endpoint does not recognize claude-sonnet-5, the default chat path will fail with an API error for new/default installations.

The added tests only assert that the constant equals this value; they do not verify that the model ID is available from Anthropic or from the fetched model list.

Recommendation: Use a currently supported Anthropic alias/versioned model ID, or resolve the default from the fetched models list with a safe fallback to a known available Sonnet-tier model.

[herikwebb]

Re the finding on claude-sonnet-5: that alias is a currently served first-party Anthropic model (the current Sonnet tier — see the models list at platform.claude.com/docs). The reviewer's model catalog appears to predate it. The underlying robustness point is fair for custom base_url endpoints though, so defaults are now resolved against the fetched model list via resolve_default_model — the constant is used only when the endpoint actually serves it (or when the cache is empty), with tier-prefix and first-listed fallbacks. Covered by 5 new tests.

[github-actions]

Automated PR Review

Model: gpt-5.5
Verdict: CHANGES_REQUESTED

Changed files:

notebook_intelligence/claude.py
tests/test_claude_models.py

Review result:

Findings

Medium — notebook_intelligence/claude.py — Default model resolution uses a global, endpoint-agnostic cache

resolve_default_model() validates defaults against _claude_models_cache, but that cache is global and not tied to the api_key / base_url passed to ClaudeChatModel or ClaudeCodeInlineCompletionModel.

This can select a model unsupported by the current endpoint:

• Server fetches models from endpoint A and populates _claude_models_cache.
• User switches to a custom base_url endpoint B, or the refresh for B fails / is still in flight.
• Constructing ClaudeChatModel("", base_url=B) resolves the default against endpoint A’s cached model list.
• Requests are then sent to endpoint B with a model ID from endpoint A, causing failures.

The new tests cover cache-based fallback behavior, but not cache staleness across endpoint changes.

Suggested fix: associate the model cache with the credentials/base URL it was fetched from, clear it on endpoint changes, or pass endpoint identity into resolve_default_model() and only use cached models known to belong to the current endpoint. If the cache is stale or from another endpoint, avoid treating it as authoritative.

[github-actions]

Automated PR Review

Model: gpt-5.5
Verdict: CHANGES_REQUESTED

Changed files:

notebook_intelligence/claude.py
tests/test_claude_models.py

Review result:

Findings

High — notebook_intelligence/claude.py

Default model validation is bypassed when the cache is empty or belongs to another endpoint.

resolve_default_model() returns the hard-coded preferred model whenever get_claude_models() is empty or _claude_models_cache_endpoint does not match the requested (api_key, base_url). That means the constructors can still send CLAUDE_DEFAULT_CHAT_MODEL / CLAUDE_DEFAULT_INLINE_COMPLETION_MODEL directly to a custom endpoint before its model fetch completes, after a fetch failure, or immediately after credentials/base URL change.

This undermines the stated purpose of validating defaults against endpoint-specific catalogs and can produce failing API requests for custom/proxy endpoints that do not serve those hard-coded IDs. Consider synchronously fetching/awaiting the catalog for custom endpoints before resolving an empty model id, or falling back only when the cache is known authoritative for that endpoint.

Medium — notebook_intelligence/claude.py

The “newest same tier” fallback uses lexicographic sorting of model IDs.

resolve_default_model() does:

tier_matches = sorted(i for i in ids if i.startswith(tier_prefix))
fallback = tier_matches[-1]

Lexicographic order can choose the wrong “newest” model for version-like IDs, e.g. claude-sonnet-4-6 sorts after claude-sonnet-4-10. If this fallback is used for endpoint compatibility, selecting the wrong model can silently downgrade users or choose an older alias. Prefer API-provided ordering/metadata if available, or parse known version/date suffixes explicitly.

[herikwebb]

Round-4 follow-ups pushed: tier fallback now sorts by numeric version components (fixes the 4-10 vs 4-6 lexicographic issue), and a stale cross-endpoint cache triggers a background catalog refresh so resolutions converge after a credentials change. On the suggestion to fetch synchronously before resolving: deliberately not done — resolve_default_model runs inside model constructors on the request path, and blocking there on a slow/dead custom endpoint would turn one possibly-failing request into a hung chat turn. The constant remains the fallback until the endpoint's catalog lands.

[github-actions]

Automated PR Review

Model: gpt-5.5
Verdict: APPROVE

Changed files:

notebook_intelligence/claude.py
tests/test_claude_models.py

Review result:

No automated findings were found. Residual risks: the new default model IDs and endpoint-aware fallback behavior depend on live Anthropic/custom-endpoint catalogs, so integration coverage against real or contract-tested model-list responses would still be valuable.

[herikwebb]

Manual test results (2026-07-07)

All four scenarios pass against the live first-party Anthropic API (credits now available):

• Catalog fetch — 10 models fetched, context windows sourced from the API's max_input_tokens.
• Chat default — resolves to claude-sonnet-5 (matched verbatim in the catalog).
• Version-aware fallback — a bogus claude-sonnet-99 correctly resolves to claude-sonnet-5 as newest-in-tier over claude-sonnet-4-6/claude-sonnet-4-5-…, confirming numeric (not lexicographic) version sort.
• Inline completions — Haiku default drives ghost-completions in-notebook; NBI_CLAUDE_INLINE_COMPLETION_MAX_TOKENS clamp holds end-to-end.

Observation (not a defect)

The Models API lists Haiku only under its dated id claude-haiku-4-5-20251001, not the rolling claude-haiku-4-5 alias it uses for Sonnet. So the inline default claude-haiku-4-5 isn't matched verbatim and the fallback resolves it to the dated snapshot — the same model, correct outcome. Side effect: this INFO line prints on every startup, which is expected/healthy, not an error:

Default model 'claude-haiku-4-5' is not served by the configured endpoint; using 'claude-haiku-4-5-20251001'

Leaving the constant as the rolling alias deliberately — resolving it via the fallback is exactly what this change is for.

[herikwebb]

Raised upstream as plmbr#389; closing this fork PR. Review history retained here.