Skip to content

Vision Proxy for Text-Only Models + Context Overflow Fix + Output Popup Fix#76

Open
Wallacy wants to merge 3 commits into
ltmoerdani:mainfrom
Wallacy:feat/vision-proxy-and-context-fix
Open

Vision Proxy for Text-Only Models + Context Overflow Fix + Output Popup Fix#76
Wallacy wants to merge 3 commits into
ltmoerdani:mainfrom
Wallacy:feat/vision-proxy-and-context-fix

Conversation

@Wallacy

@Wallacy Wallacy commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

PR: Vision Proxy for Text-Only Models + Context Overflow Fix + Output Popup Fix

Branch: feat/vision-proxy-and-context-fix
Issues: Closes #74, Fix #67
Files changed: 6 files, +129/−59

What it does

🖼️ Vision Proxy for Text-Only Models (#74)

When a non-vision OpenCode model receives an image, the extension transparently forwards it to a configured vision-capable model, receives a text description, and feeds that to the original model — so text-only models can "see" images with zero extra steps.

  • Single command UX: Ctrl+Shift+POpenCode Go: Configure Vision Proxy — no settings to toggle, no config to hunt for
  • Smart picker: Filters models by supportsVision (from models.dev metadata + VISION_CAPABLE_MODELS set)
  • "None" option at the top to disable the proxy
  • Prompt customization: A "Customize prompt..." entry in the picker lets users edit the description instruction sent to the vision model
  • Zero config cruft: No opencodego.visionProxy boolean, no opencodego.visionModel string — model ID is stored in extension globalState, set exclusively via the command

Key change: modelCapabilities() now reports imageInput: true for all models when the proxy is active (so VS Code doesn't strip images).

🛡️ Context Overflow Safety Margin (#68)

estimateTokenCount can underestimate token counts by 0–2%. On large prompts (~130K tokens on a 262K window), this could push the payload 1 token past the limit, causing a 400. Added a 64-token safety margin to promptReserve in modelLimits(). (I got this bug few times)

🔇 Output Pane Focus Steal (#67)

Removed the stray .show(true) call in streamChatCompletions that popped the Output panel over the chat when the response was empty.

Tests Added

File Tests What's tested
metadata.test.ts 3 new VISION_CAPABLE_MODELS set correctness — known vision models included, text-only excluded
visionProxy.test.ts 9 new Proxy condition logic (all 6 boolean combinations), modelCapabilities flag behavior, circular-regression guard

Total: 107 tests, 0 failing across all test files.

Architecture

User attaches image → text-only model
  → modelCapabilities() reports imageInput:true (isVisionProxyEnabled())
  → VS Code keeps images in the request
  → provideLanguageModelChatResponse() detects:
      hasImageInput && !actuallySupportsVision (cached raw flag) && visionProxyModelId
  → proxyVision(): vscode.lm.selectChatModels + model.sendRequest
  → Replaces image_url[] parts with text description
  → Original model processes the text normally

Summary

This PR adds a transparent vision proxy that lets text-only OpenCode models
"see" images by relaying them to a configured vision-capable model. It also
fixes a context overflow edge case and a focus-stealing Output popup.

Changes

Vision proxy

  • New command opencodego.configureVisionProxy with a QuickPick that shows:
    • A "None" option to disable
    • A "Customize prompt..." option to edit the description instruction
    • All vision-capable models (filtered by models.dev metadata)
  • Model ID stored in globalState — no settings to toggle. If a model is saved, the proxy is on.
  • Caches actuallySupportsVision before modelCapabilities() overrides it, fixing a circular condition that made the proxy dead code.

Bug fixes

  • Context overflow 400: 64-token safety margin in modelLimits().
  • Output pane focus steal: removed .show(true) in streamChatCompletions empty-response warning.

Tests

  • Added VISION_CAPABLE_MODELS coverage to metadata.test.ts
  • Created visionProxy.test.ts with 9 tests covering the proxy condition and modelCapabilities flag

Wallacy added 3 commits July 12, 2026 16:23
- Vision proxy (ltmoerdani#74): new setting opencodego.visionModel lets users specify
  a vision-capable Copilot model (e.g. copilot:gpt-5.5). When a text-only
  OpenCode model receives an image, the extension transparently forwards it
  to the vision model, gets a text description, and feeds it to the original
  model. Uses vscode.lm.selectChatModels + sendRequest — no extra deps.
- Context overflow fix: added 64-token safety margin to promptReserve in
  modelLimits() to prevent 400 errors when estimateTokenCount underestimates
  by 1-2 tokens on large prompts.
- CHANGELOG/README/docs updated.
…gin (ltmoerdani#74)

- Vision proxy: new opencodego.visionModel setting and
  'OpenCode Go: Configure Vision Proxy' command to pick from
  available vision-capable models. When a text-only model receives
  an image, the extension forwards it to the vision model, gets a
  description, and feeds it to the original model. Model lookup is
  flexible: accepts full model IDs, vendor:name, or name substring.
- Context overflow fix: added 64-token safety margin to promptReserve
  in modelLimits() to prevent 400 errors when estimateTokenCount
  underestimates by 1-2 tokens on large prompts (reported with GLM-5.2).
- CHANGELOG/README/docs updated.
@ltmoerdani

Copy link
Copy Markdown
Owner

hey Wallacy, nice work on this one. the vision proxy idea is solid, and the circular-dependency fix (caching actuallySupportsVision before modelCapabilities() overrides it) is genuinely clever.

a few things to sort out before merge:

🔴 the context overflow fix isn't in the code

the CHANGELOG and PR description both say you added "a 64-token safety margin to promptReserve in modelLimits()". i checked the diff twice and modelLimits() isn't touched anywhere. the claim only lives in the CHANGELOG text.

did a commit get lost during the rebase, or was this never implemented? if it's the latter, just drop the CHANGELOG entry so users don't expect a fix that isn't there.

🟡 README table is broken

this row collapsed into one line:

| 🧠 Thinking controls | ... || 🖼️ Vision proxy | ... || 📊 Live usage tracking | ... |

github will render those three features as a single mangled row. needs to be split back into separate lines.

🟡 CHANGELOG lost the 0.3.7 heading

the diff removes ## [0.3.7] — 2026-07-09, so the ### Added block underneath is now orphaned. all the 0.3.7 entries end up floating inside the 0.4.0 section.

🟢 non-blocking

two small things, feel free to ignore for this PR:

  • proxyVision builds a dummy CancellationToken instead of passing options.token through. means if the user cancels the chat, the proxy call keeps running. easy to wire up.
  • the proxy calls model.sendRequest() on a copilot model, which quietly eats the user's copilot quota. a log line in the output channel, or a note in the README, would help people understand what's happening.

the .show(true) removal for #67 is the right call. the graceful fallback (strip images with a placeholder rather than letting the model 400) is good defensive thinking. and the test suite covering the proxy condition plus the circular-regression guard is solid.

once the blocker and the two formatting issues are sorted, i'll merge with a regular merge commit. no squash, your commits stay intact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Vision Proxy for non vision model [BUG] Annoying output window popup randomly

2 participants