Skip to content

fix(backend): detect ledger gaps via a contiguous-cursor checkpoint table - #1572

Merged
ogazboiz merged 1 commit into
LabsCrypt:mainfrom
tosin-zoffun:fix/issue-1376-cursor-invariant
Jul 30, 2026
Merged

fix(backend): detect ledger gaps via a contiguous-cursor checkpoint table#1572
ogazboiz merged 1 commit into
LabsCrypt:mainfrom
tosin-zoffun:fix/issue-1376-cursor-invariant

Conversation

@tosin-zoffun

Copy link
Copy Markdown
Contributor

Summary

eventIndexer.ts advanced its cursor (indexer_state.last_ledger) whenever it was monotonic, with no record of which ledger ranges had actually been confirmed scanned by the RPC. A restart that resumes from a stale or clamped starting point could silently skip a range of ledgers with no error — empty ledgers legitimately produce no events, so the old code had no way to distinguish "empty" from "skipped."

This adds a checkpoint table so the indexer can detect that kind of gap and flag it, and gates the one consumer (defaultChecker.ts) that draws conclusions from indexed events on there being no unresolved gap for the loan manager contract.

  • New ledger_checkpoints table (contract, range_start, range_end, status). After every poll iteration, EventIndexer.recordCheckpoint records the range that was just requested from the RPC:
    • 'verified' when range_start immediately follows the previously-recorded range's range_end.
    • 'suspect' for the gap in between when it doesn't — logged and persisted for follow-up.
  • EventIndexer.getSuspectRanges() and a standalone hasUnresolvedLedgerGaps(contract) (in the new ledgerCheckpoints.ts, kept separate from eventIndexer.ts so consumers don't pull in the indexer's full dependency graph) expose this to callers.
  • defaultChecker.checkOverdueLoans() now checks hasUnresolvedLedgerGaps for LOAN_MANAGER_CONTRACT_ID before evaluating overdue loans, since fetchOverdueLoanIds/fetchOverdueStats derive loan state entirely from indexed contract_events — a missed LoanRepaid or LoanDefaulted in a gap risks a false default. The run is skipped (not failed) when a gap is unresolved, returning { skipped: true, skippedReason: 'unresolved_ledger_gap' }. The gate itself fails open on a query error so a checkpoint-table hiccup alone doesn't block default checking.
  • Fixed migrations/1801..._create-pause-state-table.js: it used CommonJS exports.up = async (db) => ... in an ESM package ("type": "module"), which throws ReferenceError: exports is not defined when node-pg-migrate loads it. This migration runs immediately before the new 1802..._create-ledger-checkpoints.js in file order, so it had to be fixed to verify migrate:up/migrate:down at all. Converted to the same ESM pgm-style used by every other migration in the directory — no behavior change.

Out of scope

Issue #1376's full spec covers five phases across contracts, backend, frontend, and CI. This PR is the backend detection primitive plus the one known consumer that needed gating. Left out, each being its own multi-hour, multi-file change:

  • Reorg detection via a content digest (range_digest) — requires the contract-level event_seq identifier added to loan_manager/lending_pool/multisig_governance/remittance_nft first.
  • Automatic backfill of a detected gap (backfillRange).
  • The frontend SSE resync frame and TanStack Query cache invalidation.
  • The docker-compose.yml fixture RPC and the corresponding CI job wiring.

Also unrelated and pre-existing on main (verified against a clean checkout before making any changes, not introduced by this PR):

  • npm run build / npm run typecheck in backend/ already fail on transactionController.ts, pagination.ts, pauseGuard.ts, sorobanService.ts, and others.
  • The migration-check job's "roll back every migration one at a time" loop breaks on migration 1800..._keyset-pagination-seq-columns.js: its down() drops an index name that doesn't match what its own up() created.

Test plan

  • npm test -- --testPathPatterns="eventIndexer|defaultChecker" — 32/32 passing (7 new: gap/contiguous/first-checkpoint scenarios, getSuspectRanges, hasUnresolvedLedgerGaps happy/empty/error paths; 2 new: defaultChecker skips when a gap exists, proceeds when it doesn't).
  • Full npm test — same 30 pre-existing failing suites / 15 pre-existing failing tests as a clean checkout of main (confirmed via git stash); no new failures, 9 more passing tests than baseline.
  • npx eslint on every changed/added file — clean.
  • Migration verified against a local Postgres 16: migrate:upmigrate:down (removes ledger_checkpoints and pause_state with no residual objects) → migrate:up again succeeds.

Closes #1376

…able

The event indexer advanced its cursor (indexer_state.last_ledger) as long
as it was monotonic, with no record of which ledger ranges had actually
been confirmed scanned. A restart that resumes from a stale or clamped
starting point — or any other source of a skipped range — could silently
drop events (e.g. a LoanDefaulted) with no error raised, since empty
ledgers legitimately produce no events and the old code couldn't tell an
empty range from a skipped one.

- Add ledger_checkpoints (contract, range_start, range_end, status) and
  record a checkpoint after every poll iteration. A range is 'verified'
  when it immediately follows the previously-recorded range; otherwise the
  gap between them is flagged 'suspect' and logged.
- Expose EventIndexer.getSuspectRanges() and a standalone
  hasUnresolvedLedgerGaps(contract) so consumers can check before trusting
  indexed events.
- Gate defaultChecker.checkOverdueLoans on the loan manager contract having
  no unresolved gap, since fetchOverdueLoanIds/fetchOverdueStats derive
  loan state entirely from indexed contract_events — a missed
  LoanRepaid/LoanDefaulted there risks a false default. The gate fails open
  on a query error so a checkpoint-table hiccup alone doesn't block runs.
- Fix migrations/1801..._create-pause-state-table.js: it used CommonJS
  `exports.up = async (db) => ...` in an ESM package ("type": "module"),
  which throws `ReferenceError: exports is not defined` when node-pg-migrate
  loads it — this runs before the new 1802 migration in file order, so it
  had to be fixed to verify migrate:up/down at all. Converted to the same
  ESM pgm-style used by every other migration in this directory.

Out of scope (see issue LabsCrypt#1376 for the full spec): reorg detection via a
content digest, automatic backfill of a detected gap, the contract-level
event_seq identifier, the frontend SSE resync flow, and the CI/docker-compose
fixture RPC wiring — each is its own multi-hour, multi-file change; this PR
covers the backend detection primitive and the one known consumer that
needed gating.

Verified backend/migrations/1802..._create-ledger-checkpoints.js up/down/
up-again against a local Postgres 16 with no residual objects. Pre-existing
main already fails `npm run build`/`npm run typecheck` (transactionController.ts,
pagination.ts, pauseGuard.ts, etc.) and the migration-check job's full
down-everything loop breaks on migration 1800's down() referencing an index
name that doesn't match what its own up() created — both unrelated to this
change and left as-is.

Closes LabsCrypt#1376
@ogazboiz
ogazboiz merged commit 47a5022 into LabsCrypt:main Jul 30, 2026
7 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Cross-Layer] Contiguous-Cursor Invariant and Reorg/Gap Recovery for the Soroban Event Indexer

2 participants