feat(state): give memory an expiry and multi-source provenance

[ion-alpha-dev]

What

Three changes to the state.MemoryStore contract, all of them about what a memory item is allowed to say about itself:

• MemoryItem.ExpiresAt. An item can now carry the instant it stops being recallable. The zero value never expires. Recall omits an expired item on every backend.
• MemoryItem.Sources []string replaces the single-valued Source. An item distilled from several inputs can record all of them, in the writer's order.
• Dedup and retention are documented as host policy on the MemoryStore interface, along with what a backend may not do about them.

Why

The first two are things the contract could not express, and both are cheap now and breaking later: once other implementations exist, changing the shape of MemoryItem breaks all of them at once.

Expiry. A fact with a known death date, a credential that rotates on Friday, a plan void at the end of the month, had nowhere to record it. That date is known at write time and nowhere else, so a host sweeping the store afterwards can only guess at it.

Provenance. A single source string makes a purge approximate exactly where it needs to be exact. Retiring everything one compromised source contributed to means finding the items where it was one contributor among several, and a single field silently under-reports precisely those. It also lets the write gate be graded correctly: a multi-source item now takes the trust of its weakest source, so mixing one trusted input into a poisoned item no longer buys it through.

Dedup and retention. Memory is append-only, so contradictory facts accumulate and nothing links a correction to what it corrects. Resolving that stays the host's job, because the rule is domain knowledge a store cannot infer (one preference per key, but every observation kept). The defect was that the contract never said so, leaving each backend to invent its own answer. It now states the ownership, and states that a backend may not silently drop a write it judges duplicate or garbage-collect old items on its own.

How to verify

go test ./...

The behaviour is pinned in the shared conformance suite, so both backends are held to it:

• memorytest.RunSuite gains Provenance and Expiry. The expiry case runs every recall shape a backend implements: scoped and query-less, full-text, unfiltered, kind-filtered, widened over a scope chain, relevance-ordered, and capped. That matters because SQLite answers the scoped query-less read from a prepared statement and everything else from a query built per call; a backend that put the expiry predicate in only one of them would serve expired memory from the other and still pass a suite that tested one shape.
• TestProviderMemoryExpiryIsExactInBothQueryShapes drives a manual clock across the expiry instant and asserts both SQLite query shapes agree on it, which is the part a suite running on the wall clock cannot reach.
• TestMemoryItemExpiredAtBoundary and TestRecallQuerySelectsRejectsExpired pin the half-open boundary and that expiry outranks every other selector.
• TestMemoryItemUnmarshalAcceptsLegacySource and TestRecallReadsLegacySingleSourceResources cover reading data written under the old shape.

Patch coverage is 96.8%; the uncovered lines are the failure branches of the conformance suite's shared assertion helpers, which by construction never run on a green build.

Notes for reviewers

Compatibility. Items written under the old single-source shape are still on the spine and in the resource store, and both read paths accept them: MemoryItem.UnmarshalJSON folds a legacy Source into a one-element Sources, and the resource spec does the same for a stored source. Without that, a rebuild would decode those events into items with no provenance and quietly rewrite the history the replay exists to reproduce. Nothing writes the old field any more.

Migration 0021 adds sources (a JSON array) and expires_at, backfills existing rows into one-element arrays so no row loses provenance, and drops source.

Expiry is stored as unix nanoseconds, not RFC3339 text. created_at is RFC3339Nano, which drops trailing zeros from the fractional second and so is not fixed-width and does not compare correctly as text. Expiry is evaluated on every recall including the prepared statement, so it has to be a predicate SQLite can answer exactly rather than one the Go side re-checks afterwards. (The created_at ordering defect itself is separate and untouched here.)

RecallQuery.Selects takes a now argument now. Expiry rides in the shared selector rather than in a separate pass because a backend that forgot the separate pass would serve expired memory and pass every other test. Each backend reads its clock once per recall, so one result set is judged against one instant.

Deliberately not included: a Supersedes field. Linking a correction to what it corrects would need a reverse lookup in every backend and makes a recall's result set depend on writes it did not read; that is worth doing on its own evidence, not folded in here. The contract now names the gap instead of being silent about it.

[codecov]

Codecov Report

❌ Patch coverage is 93.92265% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
memory/memorytest/memorytest.go	83.07%	5 Missing and 6 partials ⚠️

📢 Thoughts on this report? Let us know!