What happened:
Changing the LLM model in Settings while a task is running breaks caching for that task. Even when the provider stays the same and only the model name changes, all accumulated per-task cache state is discarded — cache hit rates drop to zero and the cache has to re-warm from scratch mid-task. On providers with stateless chat-completions APIs, the wiped session history is also the task's conversational memory, so the impact is worse than cost: the in-flight task can lose earlier context.
What I expected:
A model-only change within the same provider should not destroy per-task session state. The session message histories are provider-agnostic message lists that the new model can consume as-is — the prefix cache would take one miss and re-warm, but accumulated history (and correctness for stateless providers) should be preserved. A full reset is only justified when the provider actually changes.
Steps to reproduce:
- Start a long-running task so the session cache warms up (watch for
[CACHE METRICS] ... HIT lines in the logs).
- While the task is running, open Settings → Models and change only the model (keep the same provider). Save.
- Observe the logs:
[LLM] Reinitializing with provider: ... fires, and subsequent calls for the running task show cache misses / rebuilt session history. On BytePlus, chat_with_session can raise No session cache found for <task_id>:<call_type>.
Environment: Browser UI, macOS, commit 07ddba44, v0.1.0
Logs / screenshots:
[LLM] Reinitializing with provider: <same provider>, model: <new model> followed by [CACHE METRICS] ... MISS for a task that previously had hits.
Root cause (from reading the code):
- The frontend always includes
llmProvider in the model_settings_update payload, even when only the model changed (src/pages/Settings/ModelSettings.tsx:254).
- The handler calls
agent.reinitialize_llm(new_provider) whenever llmProvider is present — it does not distinguish "provider changed" from "model changed within the same provider" (app/ui_layer/adapters/browser_adapter.py:5831).
LLMInterface.reinitialize() unconditionally resets _session_system_prompts, all per-provider session message histories (_anthropic_session_messages, _openai_compat_session_messages, etc.), and replaces the BytePlus/Gemini cache managers — dropping the BytePlus _session_cache_registry keyed by task_id:call_type (agent_core/core/impl/llm/interface.py:332-346).
So any Settings save — including a no-op save — nukes all mid-task cache state.
Suggested fix direction:
In reinitialize(), compare the old (provider, model) against the new one:
- Provider changed → full reset (current behavior is correct).
- Model-only change → keep
_session_system_prompts and the session message histories (they are plain message lists usable by the new model); recreate the BytePlus/Gemini cache managers with the new model since their server-side caches are model-bound.
- Nothing changed → skip the reset entirely, so an unrelated Settings save doesn't disturb running tasks.
What happened:
Changing the LLM model in Settings while a task is running breaks caching for that task. Even when the provider stays the same and only the model name changes, all accumulated per-task cache state is discarded — cache hit rates drop to zero and the cache has to re-warm from scratch mid-task. On providers with stateless chat-completions APIs, the wiped session history is also the task's conversational memory, so the impact is worse than cost: the in-flight task can lose earlier context.
What I expected:
A model-only change within the same provider should not destroy per-task session state. The session message histories are provider-agnostic message lists that the new model can consume as-is — the prefix cache would take one miss and re-warm, but accumulated history (and correctness for stateless providers) should be preserved. A full reset is only justified when the provider actually changes.
Steps to reproduce:
[CACHE METRICS] ... HITlines in the logs).[LLM] Reinitializing with provider: ...fires, and subsequent calls for the running task show cache misses / rebuilt session history. On BytePlus,chat_with_sessioncan raiseNo session cache found for <task_id>:<call_type>.Environment: Browser UI, macOS, commit
07ddba44, v0.1.0Logs / screenshots:
[LLM] Reinitializing with provider: <same provider>, model: <new model>followed by[CACHE METRICS] ... MISSfor a task that previously had hits.Root cause (from reading the code):
llmProviderin themodel_settings_updatepayload, even when only the model changed (src/pages/Settings/ModelSettings.tsx:254).agent.reinitialize_llm(new_provider)wheneverllmProvideris present — it does not distinguish "provider changed" from "model changed within the same provider" (app/ui_layer/adapters/browser_adapter.py:5831).LLMInterface.reinitialize()unconditionally resets_session_system_prompts, all per-provider session message histories (_anthropic_session_messages,_openai_compat_session_messages, etc.), and replaces the BytePlus/Gemini cache managers — dropping the BytePlus_session_cache_registrykeyed bytask_id:call_type(agent_core/core/impl/llm/interface.py:332-346).So any Settings save — including a no-op save — nukes all mid-task cache state.
Suggested fix direction:
In
reinitialize(), compare the old(provider, model)against the new one:_session_system_promptsand the session message histories (they are plain message lists usable by the new model); recreate the BytePlus/Gemini cache managers with the new model since their server-side caches are model-bound.