fix(security): rate limiter fail-closed on Redis error, add Prometheus counter (#895) - #3
Open
Vincent6581 wants to merge 1 commit into
Open
fix(security): rate limiter fail-closed on Redis error, add Prometheus counter (#895)#3Vincent6581 wants to merge 1 commit into
Vincent6581 wants to merge 1 commit into
Conversation
… Prometheus counter (solutions-plug#895) The IpRateLimiter in src/newsletter.rs returned Ok(true) (allow) whenever Redis was unavailable. A Redis outage silently became an open door for newsletter spam or brute-force abuse. Changes: src/newsletter.rs - IpRateLimiter.allow() now returns false (deny → 429) on Redis error instead of true (allow) — fail-closed behaviour for all security-critical limiters - Add optional Metrics field; increment rate_limiter_redis_errors_total on every Redis failure so operators are alerted before an outage becomes abuse - Add IpRateLimiter::with_metrics(cache, metrics, name) constructor for production use; IpRateLimiter::new(cache) still works without metrics - Add 'name' field used as the 'limiter' label in the Prometheus counter - Replace tracing::warn log message to make fail-closed semantics explicit - Add module-level doc comment explaining fail-closed vs fail-open tradeoff, when fail-open is acceptable, and how to alert on the error counter - Add limiter_fails_closed_when_redis_unavailable test: creates a limiter pointed at a non-listening port, calls allow(), asserts false is returned src/metrics.rs - Add rate_limiter_redis_errors: IntCounterVec field with label 'limiter' - Register rate_limiter_redis_errors_total with the Prometheus registry - Add observe_rate_limiter_redis_error(limiter: &str) helper method Closes solutions-plug#895
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves solutions-plug#895.
The newsletter subscribe rate limiter returned
true(allow) whenever Redis was unavailable. A Redis outage silently became an open door for newsletter spam or brute-force abuse — the opposite of the intended safety policy.This PR inverts the error path to fail-closed (deny → 429), adds a Prometheus counter so operators are alerted during Redis degradation, and documents the fail-closed vs fail-open tradeoff inline.
Changes
services/api/src/newsletter.rsFail-closed behaviour
IpRateLimiter::allow()now returnsfalse(deny → HTTP 429) on any Redis error instead oftrue(allow)Prometheus counter
metrics: Option<Metrics>andname: Stringfields toIpRateLimiterIpRateLimiter::with_metrics(cache, metrics, name)constructor for production usemetrics.observe_rate_limiter_redis_error(&self.name)incrementsrate_limiter_redis_errors_total{limiter="newsletter_subscribe"}IpRateLimiter::new(cache)constructor still works (metrics = None)Documentation
rate_limiter_redis_errors_totalTest
limiter_fails_closed_when_redis_unavailable: creates a limiter pointed at port 16399 (nothing listening), callsallow(), assertsfalseis returned — confirms fail-closed path returns 429services/api/src/metrics.rsrate_limiter_redis_errors: IntCounterVecfield with labellimiterrate_limiter_redis_errors_totalwith the Prometheus registryobserve_rate_limiter_redis_error(limiter: &str)public methodAcceptance criteria
rate_limiter_redis_errors_totalPrometheus counter added, incremented on every Redis errorAlert recommendation
Add a CloudWatch metric filter + alarm on:
This will page on-call as soon as Redis degradation starts — before it escalates.