fix(serve): self-heal orphaned git index.lock that wedges checkouts

[tinder-maxwellelliott]

Problem

A production bazel-diff serve instance wedged on git and had to be restarted (a run of identical checkout failures over several minutes, then a fresh clone). The fatal error:

git checkout --force <sha> failed (exit 128)
fatal: Unable to create '/data/workspace/.git/index.lock': File exists.
... a git process may have crashed in this repository earlier: remove the file manually to continue.

An orphaned .git/index.lock permanently wedges the service. Every cache-missing request runs git checkout --force via process(..., destroyForcibly = true). When the compute thread is force-cancelled — a --requestTimeout firing future.cancel(true), JVM shutdown via computeExecutor.shutdownNow(), or an OOM kill — the git subprocess is SIGKILL'd mid-checkout. Git can't clean up index.lock on SIGKILL, so it's orphaned, and since nothing ever clears it, every subsequent checkout fails until the workspace is rebuilt.

The "Auto packing the repository in background" lines in the same window are a correlated symptom, not the cause — gc never touches index.lock.

Fix (GitClient.kt)

1. Self-heal the orphaned lock. checkout catches the failure and — because HashService serializes all index-mutating git ops, so a lock present at checkout time is provably orphaned, not live — clears a stale <git-dir>/index.lock and retries once. This is exactly the recovery git's own message advises. A genuine failure (bad revision, no stale lock) still propagates unchanged.
2. Disable auto-gc on checkouts (-c gc.auto=0). The service checks out constantly; letting each one fork a background repack contends with the next checkout, slowing it and making a mid-checkout kill more likely (and it's the source of the log noise). Fetches still trigger gc, so loose objects stay packed.

Tests (GitClientTest.kt)

• checkoutClearsOrphanedIndexLockAndSucceeds — plants a stale index.lock (the wedged state) and asserts checkout self-heals.
• checkoutStillThrowsOnGenuinelyBadRevision — ensures the recovery path doesn't swallow real failures.

Reproduced the exact exit 128 failure from a planted stale lock and confirmed heal-and-retry recovers. Server test suite (GitClientTest, HashServiceTest, ImpactedTargetsServiceTest, BazelDiffServerTest) passes; ktfmt clean.

Reviewer note

This stops the permanent wedge — each request now self-heals the lock. If --requestTimeout is set below the real checkout time on a very large workspace, checkouts can still be killed and re-orphan the lock per-request (now recovered each time rather than wedging); the underlying remedy there is a timeout sized to the workspace.

🤖 Generated with Claude Code