fix(echo-sql): probe TCP in postgres healthcheck (fixes macOS record flake at source)#237
Merged
Merged
Conversation
… race init `pg_isready -U postgres` probes the unix socket, which is up during the postgres first-boot init (a temporary socket-only server running init.sql) while TCP is not yet listening. `depends_on: service_healthy` then releases go-app, whose TCP connect (host=postgresDb:5432, single attempt, no retry) is refused -> the app exits 1 and keploy record fails. The race is near-permanent on emulated amd64 (postgres:10.5 on Apple-silicon/colima CI), where the init window is wide. Probe TCP (127.0.0.1:5432) instead, matching how the app connects, and give the gate a generous retry/start_period budget for the slow emulated init. Fixes the intermittent echo-sql failures on the macOS docker lane at the source, for both CI lanes and anyone running the sample. Ref: keploy/keploy#4335 Signed-off-by: slayerjain <shubhamkjain@outlook.com>
slayerjain
force-pushed
the
fix/echo-sql-postgres-tcp-healthcheck
branch
from
087cde1 to
b996c3a
Compare
slayerjain
added a commit
to keploy/keploy
that referenced
this pull request
Jul 8, 2026
…ess (#4337) The macOS echo-sql lane flakes because the sample compose healthcheck (`pg_isready -U postgres`) probes the unix socket, which is up during postgres first-boot init while TCP is not — so `depends_on: service_healthy` releases the go-app before TCP accepts and its single-attempt, no-retry connect is refused -> exit 1 -> record fails. Near-permanent on the emulated amd64 postgres:10.5 image on Apple-silicon runners. The durable fix is upstream in keploy/samples-go#237 (TCP healthcheck). This lane change hardens the CI orchestration so the lane is robust regardless: - pre-start postgres and block on a TCP-ready barrier before keploy launches the app (also keeps the slow init out of the --record-timer window), - idempotently normalize the compose healthcheck to a TCP probe as defense-in-depth (no-op once #237 is in the pinned samples-go; also covers the keploy-recreates-postgres path), - a Docker-daemon readiness barrier for cold colima/Docker Desktop, - replace bare `sleep` barriers with bounded condition-polls that dump app + DB diagnostics and fail loudly on timeout instead of hanging. No retries/skips/sleeps-as-barriers in the product; root-cause only. Fixes #4335 Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The
echo-sqlsample's postgres healthcheck probes the unix socket (pg_isready -U postgres). During the postgres image's first-boot init, a temporary socket-only server runsinit.sqlwhile TCP is not yet listening — so the socket probe reportshealthybefore127.0.0.1:5432accepts.depends_on: service_healthythen releasesgo-app, whose TCP connect (host=postgresDb:5432, single attempt, no retry) is refused → the app exits 1.On emulated amd64 (
postgres:10.5on Apple-silicon / colima CI runners) the init window is wide, so this is a near-permanent race — it's the root cause of the flakyecho-sqlmacOS docker lane in keploy/keploy (see keploy/keploy#4335). Linux is latently exposed too; it just usually wins the race.Fix
Probe TCP (
pg_isready -h 127.0.0.1 -p 5432), matching how the app actually connects, soservice_healthyonly fires once the real server accepts TCP. Generousretries/start_periodkeep the gate patient through slow emulated init; a fast native boot still flips healthy on the first probe.Fixing it in the sample (not just seddng it in one CI lane) fixes both CI lanes and every downstream user of this sample in one place.
Ref: keploy/keploy#4335