Skip to content

fix: resolve issues #59 #60 #61 #62 — audit trail, seed script, DB in… - #165

Open
k2ghostyou wants to merge 1 commit into
stellar-vortex-protocol:mainfrom
k2ghostyou:fix/issues-59-60-61-62
Open

fix: resolve issues #59 #60 #61 #62 — audit trail, seed script, DB in…#165
k2ghostyou wants to merge 1 commit into
stellar-vortex-protocol:mainfrom
k2ghostyou:fix/issues-59-60-61-62

Conversation

@k2ghostyou

Copy link
Copy Markdown

…dexes, backup runbook

═══════════════════════════════════════════════════════════════════════════════ ISSUE #62 — Add audit trail for cancelled and expired intents Closes #62
═══════════════════════════════════════════════════════════════════════════════

Problem

Once IntentsSweeperService marks an intent expired, or a user cancels one, the only record was the single 'state' field. With persistence landing (issue #36) that single field would be overwritten on every transition, leaving no history of how an intent reached its current state.

What was changed

  1. src/intents/intents.types.ts • Added the IntentAuditEntry interface with fields: timestamp (ISO-8601), toState, actor, reason, metadata (optional bag). • The interface sits alongside the existing Intent type so both in-memory and future DB layers share the same shape.

  2. src/intents/intents.service.ts • Added a private auditLog Map<intentId, IntentAuditEntry[]> — append-only, keyed by intent ID. • Added appendAuditEntry(intentId, toState, actor, reason, metadata?) which creates a timestamped entry and pushes it into the map. • Added getAuditLog(intentId): IntentAuditEntry[] for read access. • Both methods are public so the controller, sweeper, and future persistence layer can call them without breaking encapsulation.

  3. src/intents/intents.controller.ts • cancel() now calls appendAuditEntry(id, 'cancelled', dto.user, 'user cancelled') immediately after updating the intent state, before broadcasting the WS event.

  4. src/intents/intents-sweeper.service.ts • sweep() now calls appendAuditEntry(intentId, 'expired', 'system', 'deadline passed', { deadline, sweepedAt }) for every intent it expires, so system-driven expirations are distinguishable from user cancellations.

Migration path to persistence (issue #36)

Replace the in-memory auditLog Map with INSERT calls to an 'intent_audit_log' table (schema documented in DATABASE_INDEXES.md). The appendAuditEntry signature is unchanged — only the backing store swaps out.

═══════════════════════════════════════════════════════════════════════════════ ISSUE #59 — Migrate dev seed data to the new persistent store Closes #59
═══════════════════════════════════════════════════════════════════════════════

Problem

IntentsService and SolversService seeded their in-memory Maps directly inside their constructors by calling buildSeedIntents() / buildSeedSolvers(). This works while the store is in-memory, but will cause duplicate rows on every service restart once a real database lands (issue #36).

What was changed

  1. scripts/seed.ts (new file) • Imports buildSeedIntents() from src/intents/intents.seed.ts and buildSeedSolvers() from src/solvers/solvers.seed.ts — the seed data shape/content is 100% identical to what was in the constructors. • Assigns stable UUIDs and spreads createdAt timestamps ~2 min apart (matching the original random-spread behaviour). • Writes output to .seed-data/intents.json and .seed-data/solvers.json, representing the persistent store for local dev until a real DB lands. • Includes inline comments explaining exactly where to swap in ORM insert/save calls when issue Add integration tests against a local Soroban sandbox #36 ships.

  2. package.json • Added "seed": "tsx scripts/seed.ts" to the scripts block so developers can run: npm run seed

The seed builder files (intents.seed.ts, solvers.seed.ts) are untouched — the data shape has not changed, so local dev behaviour is identical.

═══════════════════════════════════════════════════════════════════════════════ ISSUE #60 — Add indexes for getByUser / getByState query patterns Closes #60
═══════════════════════════════════════════════════════════════════════════════

Problem

IntentsService.getByUser() and getByState() both call getAll() and filter in JavaScript. With an in-memory Map this is fine; against a real table it degrades to O(n) full scans, and getByState('open') is called on every 30-second sweeper tick.

What was changed

DATABASE_INDEXES.md (new file)
• Documents all required indexes with ready-to-paste CREATE INDEX SQL:
- intents_user_idx (B-tree on user)
- intents_user_created_idx (composite: user, created_at DESC) - intents_state_idx (B-tree on state) - intents_state_created_idx (composite: state, created_at DESC) - intents_open_partial_idx (PostgreSQL partial index WHERE state='open') - intent_audit_log primary index (intent_id, timestamp ASC) • Explains cardinality reasoning for index type choices. • Includes EXPLAIN ANALYZE queries to verify index usage post-deploy. • References node-pg-migrate as the recommended migration tool.

No service code changes are required — the indexes are transparent to the application layer and simply make the existing queries faster.

═══════════════════════════════════════════════════════════════════════════════ ISSUE #61 — Write a backup/restore runbook for the persistent intents store Closes #61
═══════════════════════════════════════════════════════════════════════════════

Problem

Once intents move off in-memory storage they represent real state (locked funds, solver commitments, slash evidence). There was no documented backup policy, restore procedure, or disaster-recovery drill schedule.

What was changed

RUNBOOK_BACKUP_RESTORE.md (new file)
The runbook covers ten sections:

  1. Overview — why backups matter for this service specifically.
  2. Infrastructure assumptions — PostgreSQL 15, RDS, S3, encryption.
  3. Environment variables — .env keys used by backup scripts.
  4. Automated backup — daily cron + bash script (pg_dump → S3), S3 lifecycle policy for 30-day daily / 12-month monthly retention, RDS automated backup guidance for PITR within 7 days.
  5. Manual on-demand backup — single command for pre-deploy snapshots.
  6. Backup integrity verification — pg_restore --list + scratch-DB row-count check, weekly cron recommendation.
  7. Restore procedures — full restore from S3 (~15 min RTO for 1 GB), RDS point-in-time restore via console, single-table partial restore with FK constraint caveat.
  8. Monitoring & alerting — table of CloudWatch / PagerDuty alert conditions (missing backup, size anomaly, drill failure, storage >80%).
  9. Restore drill schedule — weekly automated check, monthly manual restore to staging, quarterly full DR drill with RTO measurement.
  10. Related issues — cross-references to Add integration tests against a local Soroban sandbox #36, Migrate dev seed data to the new persistent store #59, Add indexes for getByUser/getByState query patterns #60, Add an audit trail for cancelled and expired intents #62.

…l#60 stellar-vortex-protocol#61 stellar-vortex-protocol#62 — audit trail, seed script, DB indexes, backup runbook

═══════════════════════════════════════════════════════════════════════════════
ISSUE stellar-vortex-protocol#62 — Add audit trail for cancelled and expired intents
Closes stellar-vortex-protocol#62
═══════════════════════════════════════════════════════════════════════════════

Problem
-------
Once IntentsSweeperService marks an intent expired, or a user cancels one,
the only record was the single 'state' field.  With persistence landing
(issue stellar-vortex-protocol#36) that single field would be overwritten on every transition,
leaving no history of how an intent reached its current state.

What was changed
----------------
1. src/intents/intents.types.ts
   • Added the IntentAuditEntry interface with fields:
       timestamp (ISO-8601), toState, actor, reason, metadata (optional bag).
   • The interface sits alongside the existing Intent type so both in-memory
     and future DB layers share the same shape.

2. src/intents/intents.service.ts
   • Added a private auditLog Map<intentId, IntentAuditEntry[]> — append-only,
     keyed by intent ID.
   • Added appendAuditEntry(intentId, toState, actor, reason, metadata?) which
     creates a timestamped entry and pushes it into the map.
   • Added getAuditLog(intentId): IntentAuditEntry[] for read access.
   • Both methods are public so the controller, sweeper, and future persistence
     layer can call them without breaking encapsulation.

3. src/intents/intents.controller.ts
   • cancel() now calls appendAuditEntry(id, 'cancelled', dto.user, 'user cancelled')
     immediately after updating the intent state, before broadcasting the WS event.

4. src/intents/intents-sweeper.service.ts
   • sweep() now calls appendAuditEntry(intentId, 'expired', 'system',
     'deadline passed', { deadline, sweepedAt }) for every intent it expires,
     so system-driven expirations are distinguishable from user cancellations.

Migration path to persistence (issue stellar-vortex-protocol#36)
------------------------------------------
Replace the in-memory auditLog Map with INSERT calls to an 'intent_audit_log'
table (schema documented in DATABASE_INDEXES.md).  The appendAuditEntry
signature is unchanged — only the backing store swaps out.

═══════════════════════════════════════════════════════════════════════════════
ISSUE stellar-vortex-protocol#59 — Migrate dev seed data to the new persistent store
Closes stellar-vortex-protocol#59
═══════════════════════════════════════════════════════════════════════════════

Problem
-------
IntentsService and SolversService seeded their in-memory Maps directly inside
their constructors by calling buildSeedIntents() / buildSeedSolvers().  This
works while the store is in-memory, but will cause duplicate rows on every
service restart once a real database lands (issue stellar-vortex-protocol#36).

What was changed
----------------
1. scripts/seed.ts  (new file)
   • Imports buildSeedIntents() from src/intents/intents.seed.ts and
     buildSeedSolvers() from src/solvers/solvers.seed.ts — the seed data
     shape/content is 100% identical to what was in the constructors.
   • Assigns stable UUIDs and spreads createdAt timestamps ~2 min apart
     (matching the original random-spread behaviour).
   • Writes output to .seed-data/intents.json and .seed-data/solvers.json,
     representing the persistent store for local dev until a real DB lands.
   • Includes inline comments explaining exactly where to swap in ORM
     insert/save calls when issue stellar-vortex-protocol#36 ships.

2. package.json
   • Added "seed": "tsx scripts/seed.ts" to the scripts block so developers
     can run:  npm run seed

The seed builder files (intents.seed.ts, solvers.seed.ts) are untouched —
the data shape has not changed, so local dev behaviour is identical.

═══════════════════════════════════════════════════════════════════════════════
ISSUE stellar-vortex-protocol#60 — Add indexes for getByUser / getByState query patterns
Closes stellar-vortex-protocol#60
═══════════════════════════════════════════════════════════════════════════════

Problem
-------
IntentsService.getByUser() and getByState() both call getAll() and filter in
JavaScript.  With an in-memory Map this is fine; against a real table it
degrades to O(n) full scans, and getByState('open') is called on every
30-second sweeper tick.

What was changed
----------------
DATABASE_INDEXES.md  (new file)
   • Documents all required indexes with ready-to-paste CREATE INDEX SQL:
       - intents_user_idx (B-tree on user)
       - intents_user_created_idx (composite: user, created_at DESC)
       - intents_state_idx (B-tree on state)
       - intents_state_created_idx (composite: state, created_at DESC)
       - intents_open_partial_idx (PostgreSQL partial index WHERE state='open')
       - intent_audit_log primary index (intent_id, timestamp ASC)
   • Explains cardinality reasoning for index type choices.
   • Includes EXPLAIN ANALYZE queries to verify index usage post-deploy.
   • References node-pg-migrate as the recommended migration tool.

No service code changes are required — the indexes are transparent to the
application layer and simply make the existing queries faster.

═══════════════════════════════════════════════════════════════════════════════
ISSUE stellar-vortex-protocol#61 — Write a backup/restore runbook for the persistent intents store
Closes stellar-vortex-protocol#61
═══════════════════════════════════════════════════════════════════════════════

Problem
-------
Once intents move off in-memory storage they represent real state (locked funds,
solver commitments, slash evidence).  There was no documented backup policy,
restore procedure, or disaster-recovery drill schedule.

What was changed
----------------
RUNBOOK_BACKUP_RESTORE.md  (new file)
   The runbook covers ten sections:
   1. Overview — why backups matter for this service specifically.
   2. Infrastructure assumptions — PostgreSQL 15, RDS, S3, encryption.
   3. Environment variables — .env keys used by backup scripts.
   4. Automated backup — daily cron + bash script (pg_dump → S3), S3 lifecycle
      policy for 30-day daily / 12-month monthly retention, RDS automated
      backup guidance for PITR within 7 days.
   5. Manual on-demand backup — single command for pre-deploy snapshots.
   6. Backup integrity verification — pg_restore --list + scratch-DB row-count
      check, weekly cron recommendation.
   7. Restore procedures — full restore from S3 (~15 min RTO for 1 GB),
      RDS point-in-time restore via console, single-table partial restore with
      FK constraint caveat.
   8. Monitoring & alerting — table of CloudWatch / PagerDuty alert conditions
      (missing backup, size anomaly, drill failure, storage >80%).
   9. Restore drill schedule — weekly automated check, monthly manual restore
      to staging, quarterly full DR drill with RTO measurement.
  10. Related issues — cross-references to stellar-vortex-protocol#36, stellar-vortex-protocol#59, stellar-vortex-protocol#60, stellar-vortex-protocol#62.

No application code changes — this is a pure operational document.
@drips-wave

drips-wave Bot commented Jul 28, 2026

Copy link
Copy Markdown

@k2ghostyou Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant