Query cache (`query_cache_limit`) is bounded by entry count, not memory, and has no TTL — heavy queries balloon RSS until restart

[IgorOhrimenko]

Summary

PgDog's parser AST cache (query_cache_limit, default 1000) is a single per-process LRU (static CACHE in frontend::router::parser::cache), shared across all clients and databases, keyed by query text. Two properties combine to let it use far more memory than the entry count suggests:

1. The limit caps the number of entries, not their total memory. A cached parse tree for a wide/complex query is orders of magnitude larger than one for SELECT 1 (MBs vs bytes). With a count limit, the cache's memory footprint is effectively unbounded — 1000 × (arbitrary entry size).
2. Entries never expire by time (no TTL / time-to-idle). An entry only leaves the cache when 1000 newer distinct queries evict it via LRU, on RESET QUERY_CACHE, or on restart. A burst of heavy distinct queries can therefore hold a large cache for a long time on a low-diversity workload.

Reproduction

Self-contained docker-compose repro: https://gist.github.com/IgorOhrimenko/2b886c7d506ff975b8a9ac22acd63c5b

Three phases against a stock pgdog (query_cache_limit=1000, _RJEM_MALLOC_CONF=background_thread:true):

phase	RSS	cache_size
baseline	~30 MB	0
2000 distinct heavy queries	~3 GB	1000
120 s light traffic, 100 fixed (hits, no turnover)	stays ~2.5 GB	1000
2000 new distinct light queries (evict heavy)	→ ~150 MB	1000

cache_size stays pinned at 1000 throughout — the LRU count is bounded; the memory is not, because entry size varies by orders of magnitude. Memory is released only once the heavy entries are evicted (or via RESET QUERY_CACHE), never on a timer.

Root cause

frontend::router::parser::cache::Cache — an LruCache sized by query_cache_limit (count only). No per-entry byte accounting and no idle expiry.

Proposed fix

1. query_cache_memory_limit (bytes): evict LRU until the summed entry size is under a byte budget.
2. query_cache_ttl (seconds, time-to-idle): drop entries not accessed within the window, via the existing maintenance sweep.

Implemented in #1266.

[sgrif]

The new parser reduces memory usage by roughly 90%, your issues should be fixed purely by switching to it. Append -new-parser to the end of any image name released in the last week in order to switch to it.