fix(tts): rate-limit bypass, silent breaker DoS, dead tracing SDK, and no shared state under scaling - #1250
Merged
Merged
Conversation
… header The Express rate-limit keyGenerator used the raw Authorization header as the bucket key before auth middleware validated anything, letting a caller rotate a fake Bearer token per request to get a fresh bucket every time and bypass the per-IP throttle on /tts/generate and /tts/enqueue. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…unting opossumOptions never set errorFilter, so non-retriable 4xx TTSProviderErrors (bad voice, invalid credential) counted toward tripping the breaker exactly like 5xx/network failures. A handful of bad requests from one caller could open the breaker for every user of that provider. errorFilter now excludes 4xx TTSProviderErrors from failure stats while still rejecting the call. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
initTracing() set up the NodeSDK, OTLP exporter, and HTTP instrumentation but was never invoked anywhere, so every tracer.startActiveSpan(...) call in TTSService/HealthCheck ran against the no-op global tracer provider and spans were silently discarded. Now invoked once at server module load, before any request handling. app.listen() is guarded behind a require.main check so importing server.ts (e.g. from tests) no longer binds a real port. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…hared store jobStore, RateLimiter, and AudioCache were process-local Maps with no shared backing, so under horizontal scaling GET /tts/job/:id 404s when polling hits a different replica than the one that processed the job, rate limits are only enforced per-replica (multiplying the effective global limit by replica count), and cache hit rate collapses across replicas. Adds a SharedStore abstraction (SharedJobStore/SharedRateLimitStore/ SharedCacheStore) with a Redis-backed implementation, wired in via config.sharedStore / REDIS_URL. New enqueueAsync/getJobAsync/listJobsAsync methods use it when configured, falling back to the existing in-memory, single-instance behavior otherwise — enqueue/getJob/listJobs are unchanged for backward compatibility. server.ts routes now use the async methods. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…scoped job store with async/shared-store paths
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conflict-resolved re-submission of #1247 (fork PR, no push access to update it directly). Merges main in and reconciles the tenant-scoped job store (from #1246) with the async/shared-store job paths added here, so cross-tenant isolation is preserved on the scaled-out code paths too. Closes #1247 in favor of this.