Skip to content

fix(ocm): serialise OCM requests via a global concurrency gate (stop the 429 storm)#89

Merged
GeiserX merged 1 commit into
mainfrom
fix/ocm-concurrency-gate
Jul 18, 2026
Merged

fix(ocm): serialise OCM requests via a global concurrency gate (stop the 429 storm)#89
GeiserX merged 1 commit into
mainfrom
fix/ocm-concurrency-gate

Conversation

@GeiserX

@GeiserX GeiserX commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Problem

The v1.10.1 tiling fix works (downtown Portland now covered — corridor 4 → 236), but it made every large country tile, and at startup ~6 of them (US, DE, FR, GB, IT, NL) tile concurrently and flood OCM's shared rate limit. Observed live on the v1.10.1 deploy:

  • An escalating 429 stormRetry-After climbing to 10 s across DE/GB/IT/US.
  • The US scrape finished partial (hit its 15-min budget at 518 requests, 0 give-ups — the 429 backoff held, but throttling ate the time).
  • Real risk: OCM throttling/banning the shared free API key, which would break all EV coverage.

Fix

  • Global concurrency gate (PUMPERLY_OCM_MAX_CONCURRENCY, default 1) shared across every OCM scraper via module-level state, so requests serialise instead of competing for the rate limit. The slot is held across a request's 429 backoff, so backpressure is global — a rate-limit pauses all OCM traffic and lets the shared limit recover, rather than firing the next country straight into another 429.
  • Per-country time budget now measures ACTIVE fetch time (slot-held duration), not wall-clock. Under serialisation a country spends most of its wall-clock waiting its turn; charging that against its 15-min budget would starve later countries into partial coverage with almost no requests. Active-time bounds each country by its own work.

Validation

  • New unit test asserts the gate never lets two OCM requests overlap at concurrency 1 (two countries scraping at once → maxInFlight === 1).
  • Existing tiling/budget/429/malformed tests still pass (the budget test terminates on the request budget; active-time stays ~0 with instant mocks).
  • The coverage win from v1.10.1 is already verified live (Portland corridor 4 → 236, Broadway Tower Garage + Fox Tower Supercharger present); this change makes big countries complete fully and stops the storm.

Trade-off

Serial OCM requests make the full daily OCM sweep take ~15–20 min (background, staggered) — deliberately polite to a free community API. Tunable via PUMPERLY_OCM_MAX_CONCURRENCY if more speed is ever needed.

Refs #85, #87.

v1.10.1 made every large country tile, and at startup ~6 of them (US, DE,
FR, GB, IT, NL) tiled concurrently and flooded OCM's shared rate limit — an
escalating 429 storm (Retry-After climbing to 10s) that left the US scrape
partial (hit its time budget at 518 requests) and risks OCM throttling the
shared API key.

Add a module-level concurrency gate shared across all OCM scrapers
(PUMPERLY_OCM_MAX_CONCURRENCY, default 1) so requests serialise instead of
competing. The slot is held across a request's 429 backoff so backpressure is
global (pauses all OCM traffic to let the limit recover). Because a country
now spends much of its wall-clock waiting its turn, the per-country time
budget switches from wall-clock to ACTIVE fetch time (slot-held duration) so a
country is bounded by its own work, not by waiting — otherwise a country that
waited would starve and go partial with almost no requests.

Verified live: the v1.10.1 fix already captured downtown Portland (corridor
4 -> 236, Broadway Tower Garage + Fox Tower Supercharger now present); this
change makes big countries complete fully and stops the 429 storm.

Refs #85, #87.
@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@GeiserX, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 54 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b5dcb99a-2d37-417d-bd7d-7f5e8c478b08

📥 Commits

Reviewing files that changed from the base of the PR and between f890041 and 4b55b7c.

📒 Files selected for processing (2)
  • src/scrapers/ocm.test.ts
  • src/scrapers/ocm.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ocm-concurrency-gate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@GeiserX
GeiserX merged commit 8044bed into main Jul 18, 2026
8 checks passed
@GeiserX
GeiserX deleted the fix/ocm-concurrency-gate branch July 18, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant