Bound wiki-agent context use: output caps + distill-then-act for oversized threads - #462
Merged
Merged
Conversation
…sized threads On 2026-07-23 the wiki sweep skipped two long threads with "maximum context length is 163840 tokens ... you requested 65536 output tokens". Diagnosis found three compounding problems, none of them "the model is too small": 1. The headless tool loop never set max_completion_tokens, so the serving backend reserved its own 65536-token default output budget out of the window - for an agent whose real output is tool calls plus a two-sentence summary. runHeadlessAgent now takes maxTokens and the wiki fleet (article agent, record extraction, librarian) caps every round at 8192, reclaiming ~57k tokens of input room. 2. Thread slices were fed to the model unbounded, and the enforced ceiling is not the registry's contextWindow: the same model id (deepseek-v4-flash) rejected at 163840 on Jul 23 and accepted 1M-scale requests a week later. New agents/_accumulator.ts ports fnord's accumulator pattern as distill-then-act: transcripts estimated over a pinned conservative working window (96k tokens) are chunked and folded into an accumulated notes buffer by read-only distill completions, and the normal tool loop then runs once over the notes with the full toolbox, so write dedup discipline is untouched. A context-length 400 on a direct run retries through the same path; a 400 during distillation backs off the chunk budget (0.2 steps, 0.6 floor) before giving up. Context-length errors that survive distillation are flagged deterministic and skip the thread on the first failure with an honest reason instead of burning the transient 3-strike budget. 3. The content-filter fallback model (arcee-trinity-large-thinking) no longer exists in Venice's catalog, so the fallback path could never succeed. Re-pointed to venice-uncensored-1-2 (supports function calling; non-reasoning, so the fallback attempt keeps reasoning_effort off the wire). Threads that fit - 410 of 412 in the live DB - run byte-identical to before; the distill path is a backstop for the tail. Covered by supabase/functions/tests/accumulator.test.ts plus new behavior tests for the mid-run recovery, the deterministic-skip surface, and the per-round output cap. CLAUDE.md gains the "always set maxTokens / never trust contextWindow" lesson so the next sub-call doesn't rediscover it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SYNOPSIS
Fixes the 2026-07-23 wiki-sweep skips ("maximum context length is 163840 tokens"). Caps agent output, distills oversized transcripts, revives the dead content-filter fallback.
PURPOSE
The wiki sweep skipped two long threads with a context-length 400. Diagnosis found the cause was not the model's window size: the headless tool loop never set
max_completion_tokens, so the serving backend reserved 65536 output tokens out of the window for an agent that emits tool calls plus a two-sentence summary. Two aggravators: thread slices go to the model unbounded, against a ceiling that moves between backends serving the same model id (163840 enforced Jul 23; 1M-scale accepted a week later - the registry'scontextWindowis not a contract); and the content-filter fallback model (arcee-trinity-large-thinking) no longer exists in Venice's catalog, so that path could never succeed.DESCRIPTION
Today,
runHeadlessAgentforwards no output cap,runWikiAgentOnThread/runExtractionOnThreadpush the whole thread slice plus the agent prompt into one conversation, every failure burns the 3-strike transient counter (even deterministic ones), and a classifier rejection retries against a vanished model id.This PR changes each point in the same order:
runHeadlessAgentgainsmaxTokens; the wiki fleet (article, records, librarian) caps every round at 8192, reclaiming ~57k tokens of input room.agents/_accumulator.ts(port of fnord's accumulator): transcripts estimated over a pinned 96k working window are rendered to text, chunked, and folded into an accumulated notes buffer by read-only distill completions; the normal tool loop then runs once over the notes, so all writes keep the full toolbox and dedup discipline. A context-length 400 on a direct run retries distilled; a 400 during distillation backs off the chunk budget (0.2 steps, 0.6 floor).deterministicand the sweep skips the thread on the first failure (p_max_failures = 1) with an honest reason.CONTENT_FILTER_FALLBACK_MODELre-pointed tovenice-uncensored-1-2(verified live: supports function calling; non-reasoning, so the fallback attempt keepsreasoning_effortoff the wire).Net effect: both skipped threads would have processed even on the 163840 backend (output cap alone suffices), and the distill path is the backstop for the genuinely huge tail - 410 of 412 threads in the live DB run byte-identical to before.
Notes:
reflection,summary,samskara_evaluation, andthread_topicsstill consume unbounded slices; the accumulator is shared and ready, wiring them is deliberate follow-up scope.Generated by Claude Code