Skip to content

Bound wiki-agent context use: output caps + distill-then-act for oversized threads - #462

Merged
sysread merged 1 commit into
mainfrom
claude/context-window-limits-wps8rl
Jul 29, 2026
Merged

Bound wiki-agent context use: output caps + distill-then-act for oversized threads#462
sysread merged 1 commit into
mainfrom
claude/context-window-limits-wps8rl

Conversation

@sysread

@sysread sysread commented Jul 29, 2026

Copy link
Copy Markdown
Owner

SYNOPSIS

Fixes the 2026-07-23 wiki-sweep skips ("maximum context length is 163840 tokens"). Caps agent output, distills oversized transcripts, revives the dead content-filter fallback.

PURPOSE

The wiki sweep skipped two long threads with a context-length 400. Diagnosis found the cause was not the model's window size: the headless tool loop never set max_completion_tokens, so the serving backend reserved 65536 output tokens out of the window for an agent that emits tool calls plus a two-sentence summary. Two aggravators: thread slices go to the model unbounded, against a ceiling that moves between backends serving the same model id (163840 enforced Jul 23; 1M-scale accepted a week later - the registry's contextWindow is not a contract); and the content-filter fallback model (arcee-trinity-large-thinking) no longer exists in Venice's catalog, so that path could never succeed.

DESCRIPTION

Today, runHeadlessAgent forwards no output cap, runWikiAgentOnThread / runExtractionOnThread push the whole thread slice plus the agent prompt into one conversation, every failure burns the 3-strike transient counter (even deterministic ones), and a classifier rejection retries against a vanished model id.

This PR changes each point in the same order:

  • runHeadlessAgent gains maxTokens; the wiki fleet (article, records, librarian) caps every round at 8192, reclaiming ~57k tokens of input room.
  • New agents/_accumulator.ts (port of fnord's accumulator): transcripts estimated over a pinned 96k working window are rendered to text, chunked, and folded into an accumulated notes buffer by read-only distill completions; the normal tool loop then runs once over the notes, so all writes keep the full toolbox and dedup discipline. A context-length 400 on a direct run retries distilled; a 400 during distillation backs off the chunk budget (0.2 steps, 0.6 floor).
  • Context-length errors that survive distillation are flagged deterministic and the sweep skips the thread on the first failure (p_max_failures = 1) with an honest reason.
  • CONTENT_FILTER_FALLBACK_MODEL re-pointed to venice-uncensored-1-2 (verified live: supports function calling; non-reasoning, so the fallback attempt keeps reasoning_effort off the wire).

Net effect: both skipped threads would have processed even on the 163840 backend (output cap alone suffices), and the distill path is the backstop for the genuinely huge tail - 410 of 412 threads in the live DB run byte-identical to before.

Notes:

  • Intentional behavioral change: deterministic context-length failures now skip on the first strike instead of the third - the retry would fail identically, and the skip reason is surfaced in the Skipped panel either way.
  • The distill-with-attempt-model choice means the rare oversized-AND-classifier-rejected thread re-distills on the fallback attempt; accepted cost, commented at the call site.
  • reflection, summary, samskara_evaluation, and thread_topics still consume unbounded slices; the accumulator is shared and ready, wiring them is deliberate follow-up scope.
  • Not verified end-to-end against a live oversized thread; the completion seam covers the logic (10 new Deno tests). Retrying the two Skipped-panel rows after merge is the cleanest live test.

Generated by Claude Code

…sized threads

On 2026-07-23 the wiki sweep skipped two long threads with "maximum
context length is 163840 tokens ... you requested 65536 output
tokens". Diagnosis found three compounding problems, none of them
"the model is too small":

1. The headless tool loop never set max_completion_tokens, so the
   serving backend reserved its own 65536-token default output
   budget out of the window - for an agent whose real output is tool
   calls plus a two-sentence summary. runHeadlessAgent now takes
   maxTokens and the wiki fleet (article agent, record extraction,
   librarian) caps every round at 8192, reclaiming ~57k tokens of
   input room.

2. Thread slices were fed to the model unbounded, and the enforced
   ceiling is not the registry's contextWindow: the same model id
   (deepseek-v4-flash) rejected at 163840 on Jul 23 and accepted
   1M-scale requests a week later. New agents/_accumulator.ts ports
   fnord's accumulator pattern as distill-then-act: transcripts
   estimated over a pinned conservative working window (96k tokens)
   are chunked and folded into an accumulated notes buffer by
   read-only distill completions, and the normal tool loop then runs
   once over the notes with the full toolbox, so write dedup
   discipline is untouched. A context-length 400 on a direct run
   retries through the same path; a 400 during distillation backs
   off the chunk budget (0.2 steps, 0.6 floor) before giving up.
   Context-length errors that survive distillation are flagged
   deterministic and skip the thread on the first failure with an
   honest reason instead of burning the transient 3-strike budget.

3. The content-filter fallback model (arcee-trinity-large-thinking)
   no longer exists in Venice's catalog, so the fallback path could
   never succeed. Re-pointed to venice-uncensored-1-2 (supports
   function calling; non-reasoning, so the fallback attempt keeps
   reasoning_effort off the wire).

Threads that fit - 410 of 412 in the live DB - run byte-identical to
before; the distill path is a backstop for the tail. Covered by
supabase/functions/tests/accumulator.test.ts plus new behavior tests
for the mid-run recovery, the deterministic-skip surface, and the
per-round output cap. CLAUDE.md gains the "always set maxTokens /
never trust contextWindow" lesson so the next sub-call doesn't
rediscover it.
@sysread
sysread merged commit 0d56f61 into main Jul 29, 2026
1 check passed
@sysread
sysread deleted the claude/context-window-limits-wps8rl branch July 29, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants