Source: Full code review 2026-07-17 (docs/reviews/2026-07-17-full-code-review.md), finding F6.
Category: robustness / performance · Severity: low · Effort: medium
Problem
deliver (internal/alert/notify.go:214-232) treats every non-2xx alike, so a permanent 400/404/410 still burns the full retry/backoff budget (default ~7 s; much longer with user-configured timeouts).- Combined with the single serial worker (
Start, notify.go:128-141), one dead webhook head-of-line-blocks every other webhook and every queued event for up to (retries+1)×timeout + backoffs per event.
The queue-drop safety valve already protects the collector, so this is degradation (delayed/dropped notifications), not loss of metric collection.
Suggested fix
Skip retries for 4xx responses except 408/429, and/or deliver per-webhook concurrently (one lightweight worker or semaphore per destination) so a slow endpoint can't delay the others.
Source: Full code review 2026-07-17 (
docs/reviews/2026-07-17-full-code-review.md), finding F6.Category: robustness / performance · Severity: low · Effort: medium
Problem
deliver(internal/alert/notify.go:214-232) treats every non-2xx alike, so a permanent400/404/410still burns the full retry/backoff budget (default ~7 s; much longer with user-configured timeouts).Start,notify.go:128-141), one dead webhook head-of-line-blocks every other webhook and every queued event for up to(retries+1)×timeout + backoffsper event.The queue-drop safety valve already protects the collector, so this is degradation (delayed/dropped notifications), not loss of metric collection.
Suggested fix
Skip retries for 4xx responses except 408/429, and/or deliver per-webhook concurrently (one lightweight worker or semaphore per destination) so a slow endpoint can't delay the others.