Skip to content

Add supervised pause and human takeover #21

Description

@ElbertePlinio

Parent: #17
Depends on: #15, #19, #20, and #22

Goal

Let a user safely pause PickLab-managed agent input, control the isolated desktop through a temporary writable viewer, and return control with fresh visual state and an evidence record.

Authority model

Do not store control state in the last-writer-wins session JSON.

Use the session directory for:

  • O_EXCL human.lease.json
  • short-lived unique agent permit files

Human lease fields include random lease ID, owner PID/process identity, creation/expiry timestamps, 30-second TTL, 5-second heartbeat, and writable VNC PID metadata.

Every desktop input call and every DevTools relay request:

  1. Acquires an agent permit.
  2. Rechecks for a human lease.
  3. Executes only if allowed.
  4. Removes the permit in finally.

Human acquisition creates the lease first, waits for existing permits to drain, then grants control. Timeout aborts cleanly. This closes the check-then-act race.

UX

picklab watch --control
  • Normal picklab watch remains read-only.
  • --control acquires the human lease.
  • It starts a second temporary loopback VNC server without -viewonly.
  • Writable VNC exists only while a valid lease is held.
  • During human control, the DevTools relay rejects all forwarding with a stable busy error.
  • On viewer exit/cancellation: stop writable VNC, capture a fresh screenshot, append the control transition, then remove the lease.
  • request_user_input handles conversational handoff where supported.
  • Expired leases recover only after the owner process identity is dead; recovery also kills recorded writable VNC.

The guarantee covers PickLab-managed desktop tools and bridge traffic, not arbitrary same-user local processes. Android viewing/takeover is out of scope for v1.

Checklist

  • Atomic human lease and agent permits
  • Process identity, TTL, heartbeat, and recovery
  • Input/relay permit integration
  • Temporary writable VNC lifecycle
  • picklab watch --control
  • Stable bridge busy response
  • Fresh-state/evidence handoff
  • Cancellation, crash, session-death, and destroy recovery
  • Race/concurrency tests
  • Live supervised proof
  • README and docs/releases/UNRELEASED.md
  • PR/review

Validation

  • bun run typecheck
  • focused lease/viewer/bridge tests
  • bun run test
  • bun run test:coverage
  • bun run build
  • two human acquisitions yield exactly one owner
  • no agent operation reaches the fake sink after human acquisition completes
  • relay fails closed during control
  • writable VNC never outlives the lease
  • heartbeat expiry, dead owner, cancellation, viewer crash, session death, and destroy recover safely
  • one recorded pause → control → interact → close viewer → resume cycle

Current status: Planned
Next: implement after viewer, relay, evidence, and input-parity foundations are stable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions