Description
Full-year ledger exports can exceed 50 MB and time out under standard JSON serialization. Add a streaming GET /ledger/export.jsonl that writes JSON-Lines chunk-by-chunk with a keepalive comment every N rows so proxies do not drop the socket. Enforce back-pressure through the DB cursor.
Requirements and context
- Must be secure, tested, and documented
- Should be efficient and easy to review
- Relevant code:
src/routes/, src/db/pool.ts
- Use
pg-cursor (or equivalent) instead of buffering all rows
Suggested execution
- Fork the repo and create a branch
git checkout -b feat/ledger-export-stream-jsonl- Implement changes
- Add streaming handler with node stream pipeline
- Include first-line manifest with row count estimate
- Emit
export.stream.rows counter and export.stream.duration histogram
- Validate security and correctness assumptions
Test and commit
- Run tests
- Cover edge cases
- Client disconnect mid-stream releases the cursor and pool client
- Include test output and notes
Example commit message
feat: streaming JSON-Lines ledger export
Guidelines
- Minimum 95 percent test coverage
- Clear documentation
- Timeframe: 96 hours
Description
Full-year ledger exports can exceed 50 MB and time out under standard JSON serialization. Add a streaming
GET /ledger/export.jsonlthat writes JSON-Lines chunk-by-chunk with a keepalive comment every N rows so proxies do not drop the socket. Enforce back-pressure through the DB cursor.Requirements and context
src/routes/,src/db/pool.tspg-cursor(or equivalent) instead of buffering all rowsSuggested execution
git checkout -b feat/ledger-export-stream-jsonlexport.stream.rowscounter andexport.stream.durationhistogramTest and commit
npm testExample commit message
feat: streaming JSON-Lines ledger exportGuidelines