Make the hourly sweep reflection's only driver, with a drain loop

docs/dev/architecture.md

@@ -306,10 +306,13 @@ browser tab being open.** The fleet runs server-side in the
 venice edge function on two kinds of trigger:
 
 - **Chat-turn tail** - `getStreamingResponse` registers a
-  sequential curation -> samskara -> reflection chain under
+  sequential curation -> samskara chain under
   `EdgeRuntime.waitUntil` after each completed turn. The tail
   is the low-latency driver for work the user notices
-  in-session: thread titles, samskara mints, fresh memories.
+  in-session: thread titles, samskara mints. Reflection is
+  deliberately NOT in the chain - memory formation is
+  sweep-only so it keeps a fixed, predictable cadence (see
+  `memory.md`).
 - **pg_cron sweeps** - scheduled jobs that pg_net-POST a
   function route, as the catch-up and maintenance drivers.
   The minute ladder: embed backfill `*/5`, bias `:03`, wiki

docs/dev/chat.md

@@ -412,15 +412,13 @@ A chat turn goes:
   `last_summarised_msg_id`. The chat loop creates that
   assistant message; the background workers pick it up on
   their next poll. See `./summaries.md`.
-- **Reflection** — driven directly from the completed-turn
-  tail. `getStreamingResponse` (the streaming orchestrator)
-  fires `reflectOneThread` via `EdgeRuntime.waitUntil` after
-  the chat response ships, draining one day-gate-eligible
-  thread from the reflection queue as background work. The
-  chat loop creates the terminal assistant message that makes
-  a thread eligible; reflection acts on it on the same turn's
-  tail (after at least a calendar day has elapsed). See
-  `./memory.md`.
+- **Reflection** — no direct call. The chat loop creates the
+  terminal assistant message that makes a thread eligible;
+  the hourly reflection sweep (reflection's only driver)
+  picks the thread up once its newest message is at least a
+  calendar day old. Deliberately not tail-driven, so edits
+  and retries any time before that sweep are what reflection
+  sees. See `./memory.md`.
 - **Topics** — `Chat.svelte` owns the `selectedTopics` /
   `topicsVocabulary` state for the drawer's topic-filter
   dropdown and threads `selectedTopics` through the three

docs/dev/memory.md

@@ -41,20 +41,20 @@ ride the system prompt independently.
 ## Role in the app
 
 Every turn the main chat model can call `memory_recall` to pull in
-relevant memories; at the tail of each completed streaming chat turn,
-the venice edge function fires a reflection pass that reads a settled
-thread and decides whether to write, update, or invalidate memories
-based on what it saw. The store lives in each user's own Supabase;
-the writes happen through the same tool harness the main chat uses.
+relevant memories; on an hourly cron cadence, the venice edge
+function's reflection sweep reads settled threads and decides whether
+to write, update, or invalidate memories based on what it saw. The
+store lives in each user's own Supabase; the writes happen through
+the same tool harness the main chat uses.
 
 From the user's perspective this is the Memory feature documented
 in `docs/user/memory.md`. The dev side has five moving parts:
 
 1. **The store** — `memories` table, RLS-scoped, with a pgvector
    `embedding` column populated asynchronously.
 2. **The writer** — the reflection agent runs in the venice edge
-   function, reads settled threads end-to-end, and uses a
-   write-scoped subset of the memory tools.
+   function on the hourly sweep, reads settled threads end-to-end,
+   and uses a write-scoped subset of the memory tools.
 3. **The reader** — the recall agent runs in the venice edge
    function, inline in the tool dispatch when the main model
    invokes the `memory_recall` tool. Read-only.
@@ -124,9 +124,9 @@ in `docs/user/memory.md`. The dev side has five moving parts:
   `memory_conversation` row per surfaced memory - the
   co-occurrence hint queue rem drains.
 - `supabase/functions/venice/agents/reflection.ts` — the reflection
-  agent. Exports `reflectOneThread(adminClient, userId)`; runs
-  write-scoped with no return value (side effects = memory tool
-  calls). Its prompt instructs TIMELESS memories - no "this session",
+  agent. Exports `runReflectionSweepTick(adminClient)`; runs
+  write-scoped (side effects = memory tool calls) and returns a
+  per-tick drain summary. Its prompt instructs TIMELESS memories - no "this session",
   no write-date narration, no first-person AI self-logging - because a
   body that stamps when it was written reads back later as a
   current-chat event (the row's `created_at` already records when it
@@ -327,26 +327,25 @@ in `docs/user/memory.md`. The dev side has five moving parts:
   headless read-only tool loop on the fast tier. Returns a
   structured JSON output the tool encodes as the `role='tool'`
   message payload for the next round.
-- **Reflection (edge function tail)** — at the end of each
-  successfully completed streaming chat turn, `getStreamingResponse`
-  fires `reflectOneThread(adminClient, userId)` via
-  `EdgeRuntime.waitUntil` as background work after the chat response
-  ships. Each invocation opportunistically drains one day-gate-
-  eligible thread from the existing reflection queue - NOT
-  necessarily the thread that just finished. Claim mutual exclusion
-  is the per-thread claim RPC (each call uses a fresh random holder
-  id); no `worker_leases` row is involved.
-- **Reflection catch-up sweep** — pg_cron job
+- **Reflection sweep (the only driver)** — pg_cron job
   `nak-reflection-sweep` (hourly, minute 27) pg_net-POSTs
-  `/reflection-sweep` -> `runReflectionSweepTick`, which claims the
-  most-overdue eligible thread across ALL users
+  `/reflection-sweep` -> `runReflectionSweepTick`, which drains the
+  day-gated queue one thread at a time: claim the most-overdue
+  eligible thread across ALL users
   (`claim_next_thread_for_reflection_sweep`, SECURITY DEFINER,
-  per-owner timezone off the profile) and runs the same shared
-  reflect body. Exists because the tail only fires when its owner
-  converses - without it a dormant account's queue never moves. The
-  per-thread claim makes tail + sweep double-driving safe. The dev
-  shim ticks this route too.
-- **Reflection attempt cap** — both reflection claims count
+  per-owner timezone off the profile), reflect it, claim the next -
+  until the queue is empty, the per-tick cap (5 threads) is hit, or
+  the tick's time budget (180s of new claims) is spent; the next
+  hourly tick resumes. Reflection deliberately does NOT ride the
+  chat turn's waitUntil tail (unlike curation and samskara): the
+  fixed cadence gives the user a predictable window to edit or
+  retry a settled conversation before reflection reads it, and the
+  strictly sequential drain keeps at most one reflection agent
+  writing to the store per tick. Claim mutual exclusion is the
+  per-thread claim RPC (each cycle uses a fresh random holder id);
+  no `worker_leases` row is involved, and overlapping ticks simply
+  claim different threads. The dev shim ticks this route too.
+- **Reflection attempt cap** — the sweep claim counts
   ATTEMPTS at claim time (`threads.reflection_attempt_msg_id` +
   `reflection_attempt_count`): three claims against the same
   terminal message and the thread stops being offered, until a new
@@ -503,7 +502,7 @@ in `docs/user/memory.md`. The dev side has five moving parts:
 - **Reflection claim columns** — `threads.reflection_holder_id`
   and `threads.reflection_claim_expires_at` are the mutual-
   exclusion primitive for the server-side reflection path.
-  The claim-RPC pair (`claim_next_thread_for_reflection` /
+  The claim-RPC pair (`claim_next_thread_for_reflection_sweep` /
   `mark_thread_reflected_if_claimed`) uses a fresh random holder
   id per call; there is no `worker_leases` row for reflection.
 - **`memories.last_librarian_visit_at timestamptz`** — per-row
@@ -651,14 +650,16 @@ from the same ports); the browser carries only the wire schemas.
   `done`); attaching it also injects the `activity` narration
   parameter into the tools' wire schemas, so the sweep paths
   (no listener) stay narration-free.
-- `reflectOneThread(adminClient, userId)` — the edge function
-  entry point. Claims one day-gate-eligible thread (newest message
-  on a prior calendar day in the user's timezone, with >= 2 user
-  messages), runs the reflection agent's headless tool loop, and
-  stamps `last_reflected_msg_id` via a claim-guarded RPC. The
-  agent's "answer" is whatever `memory_*` tool calls it made;
-  the final text is discarded. Returns without error when the
-  queue is empty or the claim is lost.
+- `runReflectionSweepTick(adminClient)` — the edge function
+  entry point. Drains day-gate-eligible threads (newest message
+  on a prior calendar day in the owner's timezone, with >= 2 user
+  messages) one at a time up to the per-tick cap/budget. Each
+  cycle runs the reflection agent's headless tool loop and stamps
+  `last_reflected_msg_id` via a claim-guarded RPC. The agent's
+  "answer" is whatever `memory_*` tool calls it made; the final
+  text is discarded. Non-throwing; returns a per-tick summary
+  (`reflected` / `claimLost` / `emptySlice` counters plus what
+  stopped the drain).
 
 ## The body-length budget (non-growth rule)
 
@@ -826,11 +827,11 @@ renders the delta as a chip (`memorySizeDelta` in
   `./topics.md` under "Memory topics" for the unit shape, the
   schema deltas, and the trigger / claim discipline.
 - **Summaries / conversation recall** — separate store (thread
-  rows), separate agents. Summary and reflection both run in the
-  venice edge function, fired from the completed-chat-turn tail
-  with an hourly sweep as catch-up. Both use the same per-row
-  claim-RPC pattern on `threads` but have independent claim
-  columns and no shared lease.
+  rows), separate agents. Both run in the venice edge function:
+  summary from the completed-chat-turn tail with an hourly sweep
+  as catch-up, reflection from its hourly sweep only. Both use
+  the same per-row claim-RPC pattern on `threads` but have
+  independent claim columns and no shared lease.
 - **Logging** - the reflection agent and both librarian passes
   emit breadcrumbs through `createEdgeLogger` (sources
   `reflection`, `rem`, `deep-sleep`), which both writes to the

docs/dev/samskara.md

@@ -254,7 +254,7 @@ toast is just a glance cue that the bias model is forming.
   LLM work.
 - **`samskaraOnTurnTail(admin, userId)`** - fired from
   `getStreamingResponse`'s `EdgeRuntime.waitUntil` tail on
-  completed turns, sequenced curation -> samskara -> reflection.
+  completed turns, sequenced curation -> samskara.
   Runs the session-responsive phases: a capped assimilate drain,
   then one pair-relate probe, then one mint-tier1 probe. (Reaction
   scoring is no longer a tail phase - it moved to the next-day
@@ -566,8 +566,8 @@ embeddings backfill's claim -> process -> save shape. Two
 drivers run the phases, split by timing sensitivity:
 
 - **Turn tail** (`samskaraOnTurnTail`) - an assimilate drain
-  capped at `TAIL_ASSIMILATE_CAP` (3) so the
-  chain never delays reflection behind it, then one pair-relate
+  capped at `TAIL_ASSIMILATE_CAP` (3) so one tail invocation
+  never monopolises the background budget, then one pair-relate
   probe, then one mint-tier1 probe (the in-session toast
   surface).
 - **Hourly sweep** (`runSamskaraSweepTick`) - a cross-user
@@ -1175,7 +1175,7 @@ summarizer reads samskaras to feed the agent.
   options struct, and samskara is currently the only caller.
   Function side, `getStreamingResponse`'s waitUntil tail drives
   `samskaraOnTurnTail` on every completed turn, sequenced
-  curation -> samskara -> reflection. `Chat.svelte` mounts the
+  curation -> samskara. `Chat.svelte` mounts the
   single `<SamskaraMoodSync />` component and owns the
   `subscribeToSamskaraInserts` realtime subscription that turns
   `samskara-mint` Broadcast events into `SAMSKARA_MINT_EVENT`. See
@@ -1192,9 +1192,9 @@ summarizer reads samskaras to feed the agent.
   predictive bias the model formed on its own. No data flows
   between them. The reflection agent reads thread transcripts
   and writes memories; the samskara assimilator reads
-  individual exchanges and writes substrate. Both ride the same
-  waitUntil tail, samskara first (reflection can span minutes
-  and samskara carries the fleet's only hard timing window).
+  individual exchanges and writes substrate. Samskara rides the
+  waitUntil tail (it carries the fleet's only hard timing
+  window); reflection is sweep-only on an hourly cadence.
   See `./memory.md`.
 - **Bias profile** - sibling server-side pipeline, no data flow.
   Bias profile aggregates cognitive-bias observations across

docs/dev/second-thoughts.md

@@ -63,7 +63,7 @@ user live.
 
 Runs as a turn-tail unit (`secondThoughtsOnTurnTail`), FIRST in the
 `EdgeRuntime.waitUntil` tail of `getStreamingResponse.ts` where
-`curateOnTurnTail` / `samskaraOnTurnTail` / `reflectOneThread` also run,
+`curateOnTurnTail` / `samskaraOnTurnTail` also run,
 and only on `terminalKind === 'completed'`. Detached, so it adds zero
 latency to the user-visible turn. Best-effort and non-throwing: a
 failure leaves the row without a verdict, never breaks the turn.
@@ -331,7 +331,7 @@ composition + wiring.
 ## Interactions
 
 - **Chat ([`chat.md`](./chat.md))** - the reviewer is a turn-tail unit
-  next to curation/samskara/reflection. The refinement reuses the
+  next to curation/samskara. The refinement reuses the
   browser send/regenerate flow for one extra APPEND turn anchored to the
   original user message; it touches the send path,
   `commit_assistant_message`'s anchor handling (append does not conflict

docs/qa/use-cases/context-recall-priming.md

@@ -170,7 +170,7 @@ the recall-quality properties those subsystems now owe.
 8. **Timeless writer (reflection).** In a SEPARATE existing thread (>= 2
    user messages, last activity on a prior calendar day so the drain is
    eligible), hold a short exchange that teaches one clear new fact about
-   the user. Let reflection run (chat-turn tail, or the hourly sweep; see
+   the user. Let reflection run (the hourly sweep; see
    [reflection-drain](./reflection-drain.md) to force it). Inspect the
    newest memory:
 

docs/qa/use-cases/reflection-drain.md

@@ -1,11 +1,14 @@
-# Reflection: tail drain, catch-up sweep, attempt cap
+# Reflection: sweep drain loop, attempt cap
 
 ## Covers
 
-The reflection agent's two drivers - the chat-turn waitUntil tail
-and the hourly `/reflection-sweep` cron route - plus the per-thread
-claim mutual exclusion and the attempt cap
+The reflection agent's single driver - the hourly `/reflection-sweep`
+cron route's per-tick drain loop (claim one thread, reflect it, claim
+the next, up to the cap/time budget) - plus the per-thread claim
+mutual exclusion and the attempt cap
 ([dev: memory](../../dev/memory.md), "Reflection" entries).
+Reflection deliberately does NOT run on the chat-turn tail: a
+completed turn must produce no reflection activity.
 
 ## Preconditions
 
@@ -22,13 +25,17 @@ claim mutual exclusion and the attempt cap
    where id = '<thread>';
   ```
 
+- For the multi-thread drain check (step 2), make two or more
+  threads eligible with the same statement.
 - `SR` = the service-role key from `supabase status -o json`.
 
 ## Steps
 
-1. Tail drain: send a chat message in ANY thread and let the turn
-   complete. Watch the Logs drawer's `reflection` source.
-2. Sweep: tick the route directly and watch the same source:
+1. No tail drive: with an eligible thread queued, send a chat
+   message in ANY thread and let the turn complete. Watch the Logs
+   drawer's `reflection` source.
+2. Sweep drain: tick the route directly with 2+ eligible threads
+   queued and watch the same source:
 
    ```sh
    curl -s -X POST \
@@ -56,15 +63,17 @@ claim mutual exclusion and the attempt cap
 
 ## Expected
 
-- (1) On a turn whose user has an eligible OLDER thread queued, the
-  drawer shows `[reflection] picked up thread ...` then
-  `finished thread ... (N tool calls over M messages)`; new
-  memories appear for content-bearing threads. With an empty queue
-  the tail is silent at default levels (trace line only).
+- (1) The completed turn produces NO `reflection` lines in the
+  drawer and the eligible thread's `last_reflected_msg_id` does not
+  advance - the tail no longer drives reflection.
 - (2) Immediate `{"accepted":true}` (the tick runs detached); the
-  drawer shows the same picked-up/finished pair when a thread was
-  eligible, `last_reflected_msg_id` advances, and
-  `reflection_attempt_count` resets to 0 on the mark.
+  drawer shows a `picked up thread ...` / `finished thread ... (N
+  tool calls over M messages)` pair PER eligible thread, one after
+  another (sequential, not interleaved), until the queue empties or
+  the cap (5 threads) / time budget (180s of new claims) stops the
+  loop. `last_reflected_msg_id` advances and
+  `reflection_attempt_count` resets to 0 on each mark; new memories
+  appear for content-bearing threads.
 - (3) Gateway 401 without a JWT; route-level
   `{"error":"forbidden"}` 403 with a non-service JWT.
 - (4) The count reaches 3 and the fourth claim returns no row for
@@ -73,7 +82,8 @@ claim mutual exclusion and the attempt cap
   hosted invocation wall clock, OR caps out at 3 attempts and stops
   burning Venice calls. Local measurement: ~9 minutes end-to-end on
   a 14-message/69KB thread - likely over the hosted window; the cap
-  is the backstop.
+  is the backstop, and the 180s claim cutoff keeps one slow thread
+  from dragging later claims past the wall clock with it.
 
 ## Cleanup
 
@@ -90,7 +100,7 @@ update threads set reflection_attempt_count = 0,
 
 | Date | Env | Commit | Result | Notes |
 | ---- | --- | ------ | ------ | ----- |
-| 2026-06-09 | local | 4e33cc3 | pass (1) | four queued reflections drained in order across turns, drawer lines live |
-| 2026-06-10 | local | d37dbcd | pass (2,3) | detached tick accepted in 215ms; sweep claimed cross-user; 401/403 posture held |
+| 2026-06-09 | local | 4e33cc3 | pass (1) | pre-rework baseline: four queued reflections drained in order across turns via the then-extant tail driver, drawer lines live |
+| 2026-06-10 | local | d37dbcd | pass (2,3) | pre-rework baseline: detached tick accepted in 215ms; sweep claimed cross-user; 401/403 posture held |
 | 2026-06-10 | local | 2e37c8b | pass (4) | counter hit 3, fourth claim skipped the thread |
 | 2026-06-10 | local | d37dbcd | note | full reflect+mark on the 69KB thread took ~9 min detached; completed only after the TTL fix (600s) |

docs/user/memory.md

@@ -34,9 +34,12 @@ or overly behavioral. That's intentional.
 
 ## How it grows
 
-When a conversation settles (no new messages for a while), a
-background agent reads the whole thread and decides whether anything
-it saw is worth remembering. For each candidate memory, the agent:
+When a conversation settles (its newest message is at least a
+calendar day old), an hourly background pass reads the whole thread
+and decides whether anything it saw is worth remembering. The lag is
+deliberate: until that pass runs, you can still edit or retry any
+part of the conversation and memory formation only ever sees the
+corrected version. For each candidate memory, the agent:
 
 1. **Searches existing memories** for anything close. Duplicates are
    worse than nothing — they dilute search.

scripts/dev-backfill-cron.mjs

@@ -18,8 +18,8 @@
 //     (the two memory librarians; same most-overdue-user claim shape
 //     with their own 12h cadences);
 //   - `nak_trigger_reflection_sweep()` hourly -> POST /reflection-sweep
-//     (reflection's catch-up drain - the chat-turn tail is the primary
-//     driver, this reaches queues whose owners stopped conversing);
+//     (reflection's only driver - each tick drains eligible threads
+//     one at a time until its cap or time budget stops it);
 //   - `nak_trigger_curation_sweep()` hourly -> POST /curation-sweep,
 //     `nak_trigger_bias_sweep()` hourly -> POST /bias-sweep, and
 //     `nak_trigger_samskara_sweep()` hourly -> POST /samskara-sweep