Dashboard: render tool-result images inline#390
Conversation
Implementation PlanProblemThe dashboard silently discards image content blocks returned by tools. Key design decision — do NOT send image bytes over SSEThis is the crux. Chosen approach (robust, reuses existing infra): strip image
(Alternative considered and rejected for the plan: raising Deliverables
Acceptance criteria
Files to create or modify
Testing approach
Risks and open questions
Plan created by mach6 |
Tools that return image content blocks (e.g. read on a PNG/JPG/GIF/WebP) previously showed only the text note in the dashboard; the reducer flattened tool results to text and silently dropped every image block. Surface those images inline so the human viewing the dashboard always sees them, independent of whether the active model supports vision. - reducer: add ToolResultImage type and ToolEntry.images; new contentToParts() helper (mime allowlist excluding SVG, base64 validation) populating images on the live tool_execution_update/end paths and the replay messagesToEntries path - transcript: render entry.images as sanitized data-URI <img>, generic across all tools, with a defense-in-depth mime re-check; bounded via CSS - large images (>1MB) ride the existing oversized_event -> resync -> messages recovery path, so the SSE eventBytes budget is never blown - HTML export already embedded tool-result images; add a regression test - tests: reducer image extraction on all paths, transcript render + mime allowlist, and HTML export image parity
Progress UpdateImplemented inline rendering of tool-result images in the dashboard transcript. Design change vs. the plan (simpler, lower-risk)The posted plan chose to strip image bytes from the SSE frame and lazy-fetch them over HTTP (new endpoint + store fetch logic). While implementing, a closer read of Net result — same acceptance criteria, far less surface area and risk:
Changes
Acceptance criteria
Tests
All green: dashboard vitest + browser suites and coding-agent suite pass; tsgo + biome clean. Commit: Progress tracked by mach6 |
Code ReviewCriticalNone. Important
Suggestions
Strengths
Agents run: code-reviewer, error-auditor, test-reviewer, completeness-checker, simplifier Reviewed by mach6 |
Review AssessmentClassifications
Action Plan
Fix before merge: findings 1, 5, and 7. Assessment by mach6 |
Progress UpdateFixed the three genuine review findings selected for this batch:
Added dispatch-level export regressions for valid Verification completed:
Commit: Progress tracked by mach6 |
|
unblocked since #388 merge (in a moment) Will likely need some reconciliation with the latest master and then confirmation that it still works. Once confirmed good tag me for review and I'll publish :) |
…ashboard-tool-result-images
Progress UpdateReconciled this branch with the latest master (which now includes the SSE-pushed fleet snapshot rework, PR #388) and confirmed the image feature still works end to end. Reconciliation
Compatibility with the SSE-snapshot rework — confirmed
Additional fix
Deferred work
Ready for review. Commit: Progress tracked by mach6 |
Code Review (re-review after master reconciliation)This is a second review pass, run after the branch was reconciled with master (PR #388, the SSE-pushed fleet snapshot rework). The prior review's genuine findings (1, 5, 7) and the bonus CSS comment fix (13) were all confirmed resolved; finding 2 remains legitimately deferred to follow-up issue #398. The reconciliation itself introduced no regressions. CriticalNone. ImportantNone. Suggestions
Strengths
Agents run: code-reviewer, error-auditor, test-reviewer, completeness-checker, simplifier Reviewed by mach6 |
Review AssessmentThis assessment independently verified each finding against the actual source. Notably, the assessor confirmed all three image-filtering layers are genuinely per-item (loop-push / Classifications
Action Plan
Fix before merge: Finding 1 (tests). No new tracking issues required — findings 2–4 are optional and finding 2's deferred hardening work is already tracked in issue #398. Assessment by mach6 |
Add per-item filtering coverage across the reducer, transcript, and HTML export layers: a valid image mixed with disallowed/malformed siblings must still surface the valid one, and multiple valid images must all render. Guards against a future regression that bails on the first bad block.
Progress UpdateAddressed the one genuine finding from the re-review (finding 1 — mixed valid+invalid image arrays untested). The image-filtering code was already correct (per-item in all three layers), so this is coverage that guards against a future first-bad-block regression. Added tests across all three layers:
The three simplifier suggestions (shared Verification:
Commit: Progress tracked by mach6 |
Closes #389
Render tool-result image content blocks (e.g.
readon a PNG/JPG/GIF/WebP) inline in the dashboard transcript, independent of whether the active model supports vision. Currently the dashboard reducer flattens tool results to text only and silently drops image blocks.Implementation plan posted as a comment below.