Record computer-use actions and diagnostic evidence

[ElbertePlinio]

Parent: #17
Depends on: #19

Goal

Record a coherent, privacy-safe timeline of what the agent did and what the computer returned, producing both machine-readable evidence and a replayable human HTML filmstrip.

Storage contract

Do not let multiple CLI/MCP/bridge processes race on manifest.json.

Each computer-use run has:

• manifest fields evidenceVersion: 1 and actionLog: "actions.jsonl"
• append-only actions.jsonl as the authoritative timeline
• an atomically-created session-scoped active-run pointer
• static report.html filmstrip plus the existing text summary

Existing manifests without evidence fields remain valid. Rendering sorts by timestamp plus action ID and assigns display step numbers.

Each sanitized action contains:

• actionId, source, tool, sessionId
• startedAt, durationMs, status
• optional allowlisted target metadata
• artifact paths
• redacted error summary

Privacy contract

• Typed/fill values persist only as length and input type.
• Network failures persist only method, origin/path without query, status, resource type, timing, and sanitized error text.
• Persist no request/response headers or bodies.
• Strip Cookie, Set-Cookie, Authorization, session/JWT/CSRF/OTP values, bare JWTs, query tokens, and CDP websocket capabilities.
• Do not auto-capture screenshots for typing/fill/password/OTP/payment actions.
• Screenshot pixels cannot be redacted; document this and allow evidence capture to be disabled.

Bounds

• Maximum auto-generated evidence: 100 MiB per run.
• After the cap, continue metadata and mark evidence truncated.
• Retain the latest 20 finalized computer-use runs per project.
• Never prune active/running runs.

Checklist

• Active-run pointer and append-only action journal
• Structured evidence redaction
• MCP tool instrumentation
• DevTools relay instrumentation
• Screenshot capture policy
• Console/network failure metadata
• Backward-compatible manifest/resource rendering
• HTML filmstrip report
• 100 MiB cap and 20-run pruning
• Concurrent append and planted-secret tests
• README and docs/releases/UNRELEASED.md
• PR/review

Validation

• bun run typecheck
• focused core/run/MCP/relay tests
• bun run test
• bun run test:coverage
• bun run build
• concurrent writers lose zero actions
• old manifests still list and render
• planted cookies, headers, query tokens, JWTs, OTPs, typed passwords, and CDP GUIDs are absent from the entire run directory
• sensitive actions create no automatic screenshots
• cap and pruning are deterministic
• a failed browser flow produces a useful sanitized report.html

Current status: Completed
Next: complete — delivered across PRs #33, #35, #37, #38, #40, and #42.

[ElbertePlinio]

Agent plan

Goal: add a privacy-safe, append-only computer-use timeline and static HTML filmstrip without allowing MCP and relay processes to race on manifest.json.

No feature flag. Every slice keeps main releasable. evidence.enabled: false is the product off switch, defaulting to enabled; it is not rollout gating.

Decisions

• actions.jsonl is authoritative; manifest.json is single-owner summary data written only at create/finalize.
• Manifest additions are optional: evidenceVersion: 1, actionLog: "actions.jsonl", and truncation metadata. Old runs remain valid.
• Active run resolution uses a session-scoped, confined pointer claimed atomically with wx; racing processes adopt the winner and do not create orphan runs.
• Every action is sanitized before persistence. Unknown target fields are dropped, not copied.
• Typed/fill values always persist as length + input type only. Sensitivity classification additionally suppresses automatic screenshots.
• Network failures keep only method, origin/path without query, status, resource type, timing, and sanitized error. No headers or bodies.
• Journal records are bounded and appended as one newline-terminated write. Cross-process stress tests must prove zero lost or corrupt actions.
• The 100 MiB cap applies to auto-generated evidence; metadata continues with a deterministic truncation marker.
• Retention removes only old finalized evidence runs. Active/running and legacy runs are never pruned.
• Rendering sorts by startedAt then actionId; display step numbers are assigned at render time.
• report.html consumes already-sanitized records, escapes all page-controlled text, uses a restrictive CSP, and makes no external requests.

PR plan

• PR 1 — Structured privacy sanitizers

  • Depends on: none
  • Strengthen secret patterns: cookies, headers, bare JWTs, query tokens, OTP/CSRF, CDP capability URLs
  • Add allowlisted target, typed-value, URL, network-failure, and error sanitizers
  • Validation: planted-secret table + recursive security sweep

• PR 2 — Evidence storage foundation

  • Depends on: none; may proceed in parallel with PR 1
  • Optional manifest v1 fields, append-only journal, atomic active-run pointer, evidence.enabled, 100 MiB accounting/truncation, finalized-run retention primitives
  • No producers yet; current behavior remains unchanged
  • Validation: separate-process concurrent writers, pointer races/stale recovery, cap, pruning, and legacy manifests

• PR 3 — Evidence rendering and confined resources

  • Depends on: PR 1 + PR 2
  • Backward-compatible text summary, static escaped/CSP report.html, actions/report MCP resources, deterministic ordering
  • Validation: legacy fixtures, XSS cases, symlink confinement, planted-secret HTML sweep

• PR 4 — MCP tool instrumentation

  • Depends on: PR 1 + PR 2
  • Record sanitized desktop/Android/session actions into the active journal; associate screenshot artifacts; finalize/clear pointer on destroy/reap
  • Preserve current one-shot screenshot behavior when no active evidence run exists
  • Validation: exact action records, no typed-value leakage, no sensitive auto-screenshots, disabled-config behavior

• PR 5 — DevTools relay instrumentation

  • Depends on: PR 1 + PR 2; may proceed in parallel with PR 4
  • Wire existing before-forward/after-response hooks, correlate IDs, record sanitized status/console/network metadata without mutating or delaying protocol frames
  • Hook failures drop evidence and emit only redacted stderr; stdout remains protocol-only
  • Validation: concurrent MCP+relay writers, overlapping IDs, cancellation/errors, planted headers/bodies/query tokens/JWTs/CDP GUIDs, real failed-request smoke

• PR 6 — Final integration, docs, and proof

  • Depends on: PRs 3–5
  • README, SECURITY.md screenshot-pixel caveat, release notes, complete failed-browser-flow filmstrip
  • Validate cap/retention end to end and recreate-session relay reconnect
  • Final PR closes Record computer-use actions and diagnostic evidence #20

Full validation

• bun run typecheck
• focused core/run/MCP/relay tests per slice
• bun run test
• bun run test:coverage
• bun run build
• recursive grep of the entire run directory finds zero planted cookies, headers, query tokens, JWTs, OTPs, typed passwords, or CDP GUIDs
• failed browser flow produces a useful sanitized report.html

Current status: Planned
Next action: start PR 1 and PR 2 as independent foundation worktrees.

[ElbertePlinio]

Implementation started in two independent foundation worktrees:

• PR 1 branch: feat/evidence-sanitizers
• PR 2 branch: feat/evidence-core

Both are additive-only and may proceed in parallel. No producer will persist evidence until both privacy and storage foundations are merged.

Current status: In progress
Next: implement and independently validate both foundation slices.

[ElbertePlinio]

PR 1 opened: #33 — structured privacy sanitizers

• Cookie/header/JWT/query/session/OTP/CSRF/CDP redaction
• URL/error/typed-target/network sanitizers
• Adversarial privacy review and fixes
• Full local validation: 71 files, 820 passed / 2 skipped
• Required CI
• Merge

PR 2 evidence storage remains local and will be rebased after #33 merges because both foundations touch core exports/release notes.

Current status: PR 1 in review
Next: ship #33, then rebase and ship PR 2.

[ElbertePlinio]

PR 1 merged: #33 — structured privacy sanitizers.

PR 2 (feat/evidence-core) is rebased onto main and locally validated:

• Atomic active-run pointer and append-only journal
• 100 MiB cap with recoverable exactly-once truncation marker
• Finalized-run retention and legacy manifest compatibility
• Focused core suite: 237 passed
• Full suite + coverage: 893 passed / 2 skipped
• Typecheck and build
• Multi-model review
• PR / required CI / merge

Current status: PR 2 review in progress
Next: resolve any review findings, then open and ship PR 2.

[ElbertePlinio]

PR 2 opened: #35 — evidence storage foundation

• Optional manifest fields and evidence.enabled
• Atomic active-run pointer
• Append-only journal with recoverable cross-process lock and torn-tail repair
• 100 MiB truncation and 20-finalized-run retention primitives
• Multi-model review; both high findings fixed and independently verified
• Local validation: core 239/239; full + coverage 895 passed / 2 skipped; typecheck/build pass
• Required CI
• Merge

Known unrelated flake: #32 failed once under parallel full-suite load, then passed in isolation and on the full rerun.

Current status: PR 2 in CI
Next: merge #35 when required checks are green, then start dependent rendering/producer slices.

[ElbertePlinio]

PR 2 #35 is review-clean and locally green, but merge is blocked by the existing Xvfb race in #32. CI failed three attempts in the same unrelated browser startup test: once during test, then twice during test:coverage; the other phase passed on reruns. Isolated browser tests and local full test/coverage pass.

Current status: PR 2 blocked on required CI
Next: resolve #32 or obtain a genuinely green required run; do not merge while the check is failing.

[ElbertePlinio]

Progress update:

• PR 1 — privacy sanitizers: feat(core): add evidence sanitizers #33
• PR 2 — evidence storage foundation: feat(core): add evidence journal foundation #35
• Delivery blocker Stabilize browser Xvfb lifecycle tests under full-suite load #32 fixed first in test(browser): stabilize Xvfb startup cancellation #36
• Current PR 2 HEAD independently reviewed and required CI green

PR 2 merged as b608900.

Foundation now on main:

• session-scoped active-run pointer
• append-only, crash-recoverable action journal
• 100 MiB cap with exactly-once truncation marker
• 20-finalized-run retention
• backward-compatible manifest fields
• structured privacy sanitizers

Current status: In progress
Next: PR 3 — rendering and confined resources. Producer/instrumentation slices remain deferred until that slice is ready.

[ElbertePlinio]

PR 3 started in feat/evidence-rendering.

Scope:

• shared deterministic action ordering + backward-compatible Markdown timeline
• escaped, no-script report.html with restrictive CSP and no external requests
• confined MCP resources for actions.jsonl and report.html
• legacy-run compatibility, XSS/planted-secret, malformed journal, and symlink tests

No producers or instrumentation in this slice. PRs 4–5 remain deferred.

Current status: PR 3 implementation
Next: add core rendering primitives and focused tests, then wire resource/report consumers.