Vision Proxy for Text-Only Models + Context Overflow Fix + Output Popup Fix#76
Vision Proxy for Text-Only Models + Context Overflow Fix + Output Popup Fix#76Wallacy wants to merge 3 commits into
Conversation
- Vision proxy (ltmoerdani#74): new setting opencodego.visionModel lets users specify a vision-capable Copilot model (e.g. copilot:gpt-5.5). When a text-only OpenCode model receives an image, the extension transparently forwards it to the vision model, gets a text description, and feeds it to the original model. Uses vscode.lm.selectChatModels + sendRequest — no extra deps. - Context overflow fix: added 64-token safety margin to promptReserve in modelLimits() to prevent 400 errors when estimateTokenCount underestimates by 1-2 tokens on large prompts. - CHANGELOG/README/docs updated.
…gin (ltmoerdani#74) - Vision proxy: new opencodego.visionModel setting and 'OpenCode Go: Configure Vision Proxy' command to pick from available vision-capable models. When a text-only model receives an image, the extension forwards it to the vision model, gets a description, and feeds it to the original model. Model lookup is flexible: accepts full model IDs, vendor:name, or name substring. - Context overflow fix: added 64-token safety margin to promptReserve in modelLimits() to prevent 400 errors when estimateTokenCount underestimates by 1-2 tokens on large prompts (reported with GLM-5.2). - CHANGELOG/README/docs updated.
|
hey Wallacy, nice work on this one. the vision proxy idea is solid, and the circular-dependency fix (caching a few things to sort out before merge: 🔴 the context overflow fix isn't in the codethe CHANGELOG and PR description both say you added "a 64-token safety margin to did a commit get lost during the rebase, or was this never implemented? if it's the latter, just drop the CHANGELOG entry so users don't expect a fix that isn't there. 🟡 README table is brokenthis row collapsed into one line: github will render those three features as a single mangled row. needs to be split back into separate lines. 🟡 CHANGELOG lost the 0.3.7 headingthe diff removes 🟢 non-blockingtwo small things, feel free to ignore for this PR:
the once the blocker and the two formatting issues are sorted, i'll merge with a regular merge commit. no squash, your commits stay intact. |
PR: Vision Proxy for Text-Only Models + Context Overflow Fix + Output Popup Fix
Branch:
feat/vision-proxy-and-context-fixIssues: Closes #74, Fix #67
Files changed: 6 files, +129/−59
What it does
🖼️ Vision Proxy for Text-Only Models (#74)
When a non-vision OpenCode model receives an image, the extension transparently forwards it to a configured vision-capable model, receives a text description, and feeds that to the original model — so text-only models can "see" images with zero extra steps.
Ctrl+Shift+P→ OpenCode Go: Configure Vision Proxy — no settings to toggle, no config to hunt forsupportsVision(frommodels.devmetadata +VISION_CAPABLE_MODELSset)opencodego.visionProxyboolean, noopencodego.visionModelstring — model ID is stored in extensionglobalState, set exclusively via the commandKey change:
modelCapabilities()now reportsimageInput: truefor all models when the proxy is active (so VS Code doesn't strip images).🛡️ Context Overflow Safety Margin (#68)
estimateTokenCountcan underestimate token counts by 0–2%. On large prompts (~130K tokens on a 262K window), this could push the payload 1 token past the limit, causing a 400. Added a 64-token safety margin topromptReserveinmodelLimits(). (I got this bug few times)🔇 Output Pane Focus Steal (#67)
Removed the stray
.show(true)call instreamChatCompletionsthat popped the Output panel over the chat when the response was empty.Tests Added
VISION_CAPABLE_MODELSset correctness — known vision models included, text-only excludedmodelCapabilitiesflag behavior, circular-regression guardTotal: 107 tests, 0 failing across all test files.
Architecture
Summary
This PR adds a transparent vision proxy that lets text-only OpenCode models
"see" images by relaying them to a configured vision-capable model. It also
fixes a context overflow edge case and a focus-stealing Output popup.
Changes
Vision proxy
opencodego.configureVisionProxywith a QuickPick that shows:models.devmetadata)globalState— no settings to toggle. If a model is saved, the proxy is on.actuallySupportsVisionbeforemodelCapabilities()overrides it, fixing a circular condition that made the proxy dead code.Bug fixes
modelLimits()..show(true)instreamChatCompletionsempty-response warning.Tests
VISION_CAPABLE_MODELScoverage tometadata.test.tsvisionProxy.test.tswith 9 tests covering the proxy condition andmodelCapabilitiesflag