Specify RFC-026 B2 activation contracts

[aaltshuler]

What & why

Specify the RFC-026 Phase B2-0 contracts required before public MemWAL streaming can activate: compare-and-chain submission identity, trusted attribution, revision-fenced lifecycle management, bounded strict correction, Lance-owned reclamation, and a graph-global history budget.

Add two RC.1 substrate guards proving the current no-go boundaries: generic Lance cleanup does not reclaim _mem_wal, and deleting the successor epoch sentinel can let a stale writer report durable success. The surrounding architecture and state-of-affairs docs now reflect those decisions.

This PR activates no public streaming surface or new storage format. RFC-026 remains Draft; schema v9, stream-config v3, lifecycle-state v2, recovery-v12, SDK/HTTP/CLI routes, and production reclamation remain inactive.

Backing issue / RFC

• Maintainer design-series revision: RFC-026 — MemWAL streaming ingest

Per the maintainer design-series lifecycle, merging publishes the reviewed Draft design state; it does not by itself mark RFC-026 accepted or activate B2.

Checklist

• Change is focused (one logical change)
• Tests added/updated for behavior changes
• Public docs updated if user-facing surface changed (N/A — no public surface activates)
• Reviewed against docs/dev/invariants.md — no Hard Invariant weakened and no deny-list side door added

Validation

• cargo test -p omnigraph-engine --test lance_surface_guards --locked — 26 passed
• scripts/check-agents-md.sh — 69 links / 65 docs OK
• git diff --check

Notes for reviewers

The two new tests are deliberately negative substrate evidence, not a reclamation implementation. They establish why OmniGraph must not delete raw MemWAL objects and why B2 needs a Lance-owned opaque inspect/plan/execute primitive with post-success epoch fencing. The RFC commits us to authoring and pinning that upstream patch rather than waiting for a release, while keeping public activation closed until the positive crash, storage, authorization, cross-version, and product-parity gates pass.

Greptile Summary

This PR specifies the RFC-026 Phase B2-0 activation contracts for public MemWAL streaming — compare-and-chain submission identity, trusted attribution, revision-fenced lifecycle, bounded correction, Lance-owned reclamation, and graph-global history budget — and adds two runtime substrate guards proving the current RC.1 no-go boundaries. No public streaming surface, schema version, or SDK/HTTP/CLI route activates; RFC-026 remains Draft.

• Two new runtime guards in lance_surface_guards.rs: cleanup_old_versions_does_not_reclaim_mem_wal_objects proves Lance's generic version cleanup leaves _mem_wal objects untouched, and mem_wal_deleted_fence_slot_allows_stale_writer_success_on_pinned_lance proves that deleting the successor's empty epoch-2 fence sentinel lets a stale writer report durable success while a fresh check_fenced() call returns PeerClaimedEpoch — together establishing why OmniGraph must wait for a Lance-owned reclamation primitive with post-success epoch fencing.
• Broad documentation updates across AGENTS.md, WAL-thinking.md, canon.md, invariants.md, lance.md, testing.md, writes.md, writing-path-state-of-affairs.md, and the RFC itself to reflect B2-0 contract closure and the Phase B1 private-core green evidence.

Confidence Score: 5/5

Safe to merge — no public surface activates, no schema or wire format changes ship, and the two new runtime guards correctly document RC.1 ownership boundaries without mutating production state.

All substantive changes are either documentation updates or deliberately negative substrate tests that prove no-go boundaries on Lance RC.1. The new tests operate on isolated temp-dir datasets, make no changes to production manifest state, and the PR description's validation output (26 tests passing) is consistent with the code reviewed. The only concern found is cleanup .unwrap() calls on intentionally broken writer handles at the end of the fence-slot guard, which is a minor maintenance point that cannot affect the guard's correctness.

crates/omnigraph/tests/lance_surface_guards.rs — specifically the cleanup tail of mem_wal_deleted_fence_slot_allows_stale_writer_success_on_pinned_lance

Important Files Changed

Filename	Overview
crates/omnigraph/tests/lance_surface_guards.rs	Adds append_pk_guard_row helper and two new runtime guards; minor concern in fence-slot guard cleanup (see comment), but overall well-structured negative evidence tests
docs/rfcs/0026-memwal-streaming-ingest.md	Extends RFC with B2-0 contracts for token identity, attribution, lifecycle, correction, reclamation, and history budget; no activation, status remains Draft
AGENTS.md	Updates RFC-026 bullet to reflect B2-0 contract closure and the two new RC.1 guards; no structural changes
docs/dev/testing.md	Adds entries for the two new B2-0 guards and their test commands; accurately reflects what each guard proves and does not prove
docs/dev/invariants.md	Updates the governing principle to include acknowledged MemWAL data as pending physical state requiring durability/recoverability; consistent with B2-0 contracts
WAL-thinking.md	Documentation-only update capturing MemWAL architecture thinking; no code changes
docs/dev/writes.md	Documentation update reflecting B2-0 contract additions; no structural changes
docs/dev/writing-path-state-of-affairs.md	State-of-affairs update to reflect Phase B1 private-core green and B2-0 contracts specified; no activation

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant T as Test
    participant DS as Lance Dataset
    participant OS as Object Store
    participant MW as MemWAL Writer

    Note over T,MW: Guard 1 - cleanup does not reclaim _mem_wal
    T->>DS: fresh_pk_dataset + initialize_mem_wal
    T->>MW: mem_wal_writer - put - seal - drain - abort
    T->>OS: list(_mem_wal) snapshot before
    T->>DS: append rows x2 then cleanup_old_versions
    DS-->>T: "removed.old_versions > 0"
    T->>OS: list(_mem_wal) snapshot after
    T-->>T: "assert before == after"

    Note over T,MW: Guard 2 - deleted fence slot allows stale success
    T->>DS: fresh_pk_dataset + initialize_mem_wal
    T->>MW: mem_wal_writer stale epoch 1
    T->>MW: mem_wal_writer successor epoch 2
    T->>OS: list wal dir find empty epoch-2 sentinel
    T->>OS: delete sentinel
    T->>MW: stale.put_no_wait watcher succeeds gap demonstrated
    T->>MW: stale.check_fenced
    MW-->>T: Err PeerClaimedEpoch
    T->>MW: stale.abort + successor.abort

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant T as Test
    participant DS as Lance Dataset
    participant OS as Object Store
    participant MW as MemWAL Writer

    Note over T,MW: Guard 1 - cleanup does not reclaim _mem_wal
    T->>DS: fresh_pk_dataset + initialize_mem_wal
    T->>MW: mem_wal_writer - put - seal - drain - abort
    T->>OS: list(_mem_wal) snapshot before
    T->>DS: append rows x2 then cleanup_old_versions
    DS-->>T: "removed.old_versions > 0"
    T->>OS: list(_mem_wal) snapshot after
    T-->>T: "assert before == after"

    Note over T,MW: Guard 2 - deleted fence slot allows stale success
    T->>DS: fresh_pk_dataset + initialize_mem_wal
    T->>MW: mem_wal_writer stale epoch 1
    T->>MW: mem_wal_writer successor epoch 2
    T->>OS: list wal dir find empty epoch-2 sentinel
    T->>OS: delete sentinel
    T->>MW: stale.put_no_wait watcher succeeds gap demonstrated
    T->>MW: stale.check_fenced
    MW-->>T: Err PeerClaimedEpoch
    T->>MW: stale.abort + successor.abort

Loading

_{Reviews (2): Last reviewed commit: "Clarify MemWAL cleanup guard intent" | Re-trigger Greptile}