fix(claude): persist image cell outputs to files instead of inlining base64#44
fix(claude): persist image cell outputs to files instead of inlining base64#44herikwebb wants to merge 2 commits into
Conversation
…base64 In Claude Code mode _resolve_supports_vision returns True, so image output bundles were rendered as markdown data-URLs and joined into the CLI query as plain text — an enormous base64 string the model pays tokens for but cannot render as an image. Route image bundles through a new image_saver hook on format_output_context: in Claude Code mode the bundle is decoded and written under NBI's upload dir (same lifecycle and sweeps as pasted images), and the message references the saved path so the agent's Read tool can view the file natively. The formatted message is now small, so Claude-mode output context is budgeted by its real token count instead of the client-side base64 estimate. Non-Claude providers keep the existing data-URL / text rendering.
Automated PR ReviewModel: Changed files: Review result: Automated review findingsHigh —
|
…heck Review follow-ups (PR #44): - format_output_context runs before the token-budget check, so the direct-write saver persisted files for contexts that were then rejected — an authenticated caller could grow the upload dir without bound by spamming over-budget contexts. _DeferredOutputImageSaver only decodes and reserves paths during formatting (bounded by coerce_payload's size caps); nothing touches disk until commit() runs after budget acceptance. - Committed files now trigger the same retention sweep as the upload handler (_sweep_upload_dir with FileUploadHandler's configured retention), so output-context images share the upload lifecycle instead of accumulating until process exit.
Automated PR ReviewModel: Changed files: Review result: No automated findings were found in the changed code. Residual risks to keep in mind:
|
Problem
In Claude Code mode
_resolve_supports_visionreturnsTrue, so image output bundles (matplotlib plots etc.) were rendered as markdown data-URLs and joined into the CLI query as plain text — an enormous base64 string the model pays tokens for but cannot render as an image.Change
image_saverhook onformat_output_context: takes(mime_type, cleaned_base64), persists the image, returns its path (orNoneto fall back to text rendering). It takes precedence oversupports_vision.Testing
Full Python suite passes (1298 tests).