perf: paginate list endpoints, signed-JWT tokens, keyset cursor pagination#266
Conversation
…ation - Add server-side pagination (skip/take, default 30) to all unbounded getall endpoints: users, files, tenants, settings, rate-limiting rules; defensive Take caps on wholesale config/tree endpoints (feature flags, menus, folders, recurring jobs). - Disable OpenIddict access-token encryption (signed JWT instead of JWE) to cut per-request token-validation CPU. - Add opt-in keyset (cursor) pagination via ?before= to the audit-logs and email-messages lists, skipping the per-request COUNT(*) and deep-OFFSET skip. - Convert the audit Browse and email History React pages to keyset cursor pagination (Newest/Prev/Next with a sessionStorage cursor trail).
- AuditLogService: align offset default sort with keyset (Timestamp DESC, Id DESC tiebreaker) on managed providers so a cursor page is a true continuation of the offset first page; keep Id fallback and skip keyset on SQLite (no DateTimeOffset ORDER BY). EmailService: add Id tiebreaker to CreatedAt ordering (offset + keyset). - SettingsService: order by (Key, Scope, UserId) — Key is non-unique, so paging was nondeterministic. - Remove dead locale keys (AuditLogs Browse.Showing, Email History.Of).
- app.tsx: narrow resolveLayout(page).default to React.ComponentType so the Inertia ComponentResolver union type-checks (the persistent layout is read at runtime, unaffected). Fixes the typecheck error at app.tsx:226. - Directory.Build.props: demote NU1903 to a warning (not error). It flags a high-severity advisory in the transitive native lib SQLitePCLRaw.lib.e_sqlite3 pulled by Microsoft.Data.Sqlite/EFCore.Sqlite 10.0.3 — not directly referenced, so it can't be bumped here; kept visible as a warning until a patched transitive ships.
…t file listing GetFilesAsync lists root files with WHERE Folder IS NULL ORDER BY FileName. The composite (Folder, FileName) index cannot satisfy that ordering for an IS NULL leading predicate, so the query fell back to a bitmap scan of every root file plus a top-N sort on each request. A standalone FileName index makes it an ordered index scan (~15ms -> <0.1ms at 80k rows; files list p50 40ms -> 6ms under load).
…esAsync ISettingsContracts.GetSettingValuesAsync gained (int skip, int take) parameters in the list-pagination work, but the FakeSettingsContracts in the Localization middleware tests still declared the old single-parameter overload, breaking the full-solution build (CS0535). Add the matching skip/take parameters.
Postgres load + profiling pass — 2 follow-up commitsRan a deeper profiling pass on Postgres (pg_stat_statements + pg_stat_user_tables + EXPLAIN, plus dotnet-gcdump before/after for memory). Results: throughput 550 → 779 rps (+42%), 0% errors; no memory leak (managed heap stable 32.8→38.3 MB across ~63k requests).
Note: the other large Verification: |
…CLRaw) The CI vulnerable-packages gate (dotnet list package --vulnerable) flagged two High-severity transitive advisories on this branch and main: - GHSA-hv8m-jj95-wg3x: MessagePack < 2.5.301 (via Aspire/Wolverine) - GHSA-2m69-gcr7-jv3q: SQLite native lib bundled by SQLitePCLRaw 2.1.11 (via Microsoft.EntityFrameworkCore.Sqlite) Neither is a direct reference, so enable CentralPackageTransitivePinningEnabled and add patched PackageVersions: MessagePack 2.5.302 and the SQLitePCLRaw family (core/bundle/provider 3.0.3, native lib 3.50.3). Transitive pinning also requires the central Npgsql version to satisfy the graph, so bump the stale Npgsql 9.0.4 to 10.0.3 (matches Npgsql.EntityFrameworkCore.PostgreSQL 10). Verified: restore clean, build 0 errors, full test suite green (~1,800 tests), dotnet list package --vulnerable reports no vulnerable packages.
1 participant
Summary
Performance work for running SimpleModule on PostgreSQL under load (Faker-seeded ~480k rows, k6 load-tested):
skip/take(default 30, capped 200/500) on/api/users,/api/files,/api/tenants,/api/settings,/api/rate-limiting; defensive.Take(cap)on wholesale config/tree endpoints (feature flags, menus, folders, recurring jobs). Previously/api/usersreturned the entire table (~3.3 MB),/api/files~4.6 MB.DisableAccessTokenEncryption()so the resource server verifies a signature instead of an RSA-OAEP decrypt on every request (auth codes / refresh tokens stay encrypted).?before=on the audit-logs and email-messages lists, skipping the per-requestCOUNT(*)and deep-OFFSET row-skip. The React audit Browse and email History pages now use the cursors (Newest / Prev / Next).Measured under k6 (40 VUs): full-mix throughput 147 → ~890 rps, bandwidth 6.8 GB → ~240 MB;
/api/usersp50 848 → ~10 ms. For the keyset endpoints in isolation: audit 534 → 5,929 rps, email 1,386 → 5,987 rps.Verification
/audit-logs/browseand/email/history: Next adds the?before=cursor and loads older rows, Previous returns to the first page, ordering is consistent (offset first page and keyset pages bothTimestamp/CreatedAt DESC, Id DESC).dotnet build(no warnings-as-errors workaround), and all touched-module tests (Settings 127, AuditLogs 38, Email 57, Users 78, FileStorage 48, Tenants 17, RateLimiting 28, FeatureFlags 14, BackgroundJobs 127, OpenIddict 40).Also fixes two pre-existing CI failures on main (not caused by this work)
app.tsx:226typecheck error — narrowedresolveLayout(page).defaulttoReact.ComponentTypeso Inertia'sComponentResolverunion type-checks (persistent layout still read at runtime).Directory.Build.props: a high-severity advisory in the transitive native libSQLitePCLRaw.lib.e_sqlite3(pulled byMicrosoft.Data.Sqlite/EFCore.Sqlite 10.0.3, not directly referenced — can't be bumped here). Kept visible as a warning until a patched transitive ships.Test plan
/audit-logs/browse, clicks Next/Prev/Newest and confirms cursor paging