Skip to content

fix(websocket): tighten upstream WS dead-peer window under ~30s head silence#9

Closed
snowkide wants to merge 1 commit into
feat/websocket-supportfrom
fix/ws-upstream-liveness-window
Closed

fix(websocket): tighten upstream WS dead-peer window under ~30s head silence#9
snowkide wants to merge 1 commit into
feat/websocket-supportfrom
fix/ws-upstream-liveness-window

Conversation

@snowkide

Copy link
Copy Markdown

Summary

  • Shorten upstream WS liveness from 30s ping / 75s pongWait to 10s / 25s so a black-holed upstream is torn down and re-dialed before typical downstream head-silence thresholds (~30s).
  • Keeps pongWait > 2× pingInterval (two ping chances before declare-dead).

Why (single Chainlink RPC = eRPC)

Chainlink pins one sticky client WS to one eRPC pod. eRPC’s internal HTTP/upstream failover on other pods does not move that socket.

If the head-feeding upstream on that pod blackholes, eRPC previously waited up to 75s before re-dial. Consumers (e.g. Polygon NoNewHeadsThreshold = 30s) already treat silence as unhealthy by then — so the node looks “eRPC down” while the rest of the eRPC fleet can still tip.

This does not refuse client eth_subscribe or push failure into Chainlink. It only makes upstream self-heal start sooner so the sticky client keeps getting heads.

Out of scope

Test plan

  • go test ./clients/ -run 'WsJsonRpc|Pong|Reconnect|Liveness|Dead'
  • go test ./indexer/adapters/wsupstream/ -run 'Reconnect|Healthy|Resub'
  • go test ./erpc/ -run UpstreamDiesUngracefully_SelfHeals
  • Canary: under upstream blackhole, erpc_upstream_websocket_connected drops and re-subscribes within ~25s; Chainlink head sub on that pod does not sit silent past ~30s

Made with Cursor

…silence

wsPingInterval/wsPongWait were 30s/75s, so a black-holed upstream could
silence newHeads on a sticky client WS for longer than typical downstream
NoNewHeads thresholds (~30s) before eRPC re-dialed — even while other pods
or HTTP failover still tipped. Use 10s/25s (still ≥2 pings per pongWait).

Co-authored-by: Cursor <cursoragent@cursor.com>
@snowkide snowkide closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant