Skip to content

fix(hypervisors): treat zombie VMM as dead in killProcess to avoid 2s deadlock#812

Open
Atishyy27 wants to merge 1 commit into
urunc-dev:mainfrom
Atishyy27:fix/790-killprocess-zombie-deadlock
Open

fix(hypervisors): treat zombie VMM as dead in killProcess to avoid 2s deadlock#812
Atishyy27 wants to merge 1 commit into
urunc-dev:mainfrom
Atishyy27:fix/790-killprocess-zombie-deadlock

Conversation

@Atishyy27

Copy link
Copy Markdown

Problem

Closes #790.

When a VMM (Firecracker/QEMU/HVT/SPT/Cloud Hypervisor) terminates it enters the zombie (defunct) state until its parent — the containerd shim — reaps it with waitpid. killProcess sends SIGKILL and then polls unix.Kill(pid, 0) waiting for ESRCH:

for {
    if err := unix.Kill(pid, 0); err != nil {
        if errors.Is(err, unix.ESRCH) { break } // never reached for a zombie
        ...
    }
    if time.Now().After(deadline) {
        return fmt.Errorf("timeout waiting for pid %d to die", pid)
    }
    time.Sleep(100 * time.Millisecond)
}

unix.Kill(pid, 0) returns nil for a zombie (it still exists in the process table), so the loop never sees ESRCH. Every forced-cleanup path (e.g. urunc delete --force) therefore blocks for the full 2-second timeout and then returns a bogus "timeout waiting for pid to die" error, even though the VMM is already dead.

Fix

Detect the zombie state via /proc/<pid>/stat and treat it as dead, breaking out of the loop immediately:

  • isZombie(pid) reads the stat file (missing file → already reaped → treated as gone).
  • statIsZombie([]byte) is a pure helper that reads the process state field. It parses the state as the first byte after the final ), so a comm field containing spaces or ) cannot corrupt parsing.

Result: forced cleanup returns as soon as the VMM is dead instead of waiting ~2000 ms, and no longer reports a false timeout error.

Tests

TestStatIsZombie covers zombie/running/sleeping states, a comm with embedded spaces and ), and malformed/empty input.

Verification

go build, go vet and test compilation pass (cross-compiled to linux/amd64). The pure statIsZombie logic was also run directly against all test cases. CI will run the full suite on Linux.

… deadlock

killProcess sends SIGKILL and then polls unix.Kill(pid, 0) until it
returns ESRCH. However, once a VMM (Firecracker/QEMU/HVT/...) terminates
it becomes a zombie until its parent (the containerd shim) reaps it, and
unix.Kill(pid, 0) keeps returning nil for a zombie. The loop therefore
never observed the process dying and burned the full 2s timeout on every
forced cleanup path (e.g. urunc delete --force), then returned a bogus
"timeout waiting for pid to die" error.

Detect the zombie (Z) state via /proc/<pid>/stat and treat it as dead,
so cleanup returns immediately. The stat parsing is split into a pure
statIsZombie() helper (handles a comm field containing spaces or ')')
and unit tested.

Fixes urunc-dev#790
@netlify

netlify Bot commented Jul 8, 2026

Copy link
Copy Markdown

Deploy Preview for urunc canceled.

Name Link
🔨 Latest commit 763c6a8
🔍 Latest deploy log https://app.netlify.com/projects/urunc/deploys/6a4ebffcb7dea600087c1495

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: killProcess deadlocks for 2 seconds waiting for zombie VMM process to be reaped

1 participant