feat: cross-harness observability hooks — the dashboard now mirrors terminal activity on every harness (#251)

[richard-devbot]

Closes #251. Root cause of the reported terminal↔dashboard mismatch: the Business Hub derives everything from events.jsonl, but only Pi's native extension wrote tool events — the guard on Claude Code/Tau/Operator is read-only, so the dashboard was blind to ordinary terminal work on 3 of 4 harnesses.

Six bisected commits:

1. rstack-agents observe (src/commands/observe.js): framework-neutral observability writer. Reads PostToolUse/Stop/SessionEnd hook JSON on stdin (or flags), resolves the active run (silent no-op when none — nothing written, exit 0), appends Pi-shaped events (tool_call/tool_result/session_shutdown + source label) to events.jsonl under lock.
2. init: Claude Code hook block now wires PostToolUse + Stop + SessionEnd → observe --source claude-code, alongside the existing guard; pinned-shape test + claude-code.md updated.
3. Tau: registered @tau.on("tool_result") (real post-execution event, verified against Tau 0.6.1) emitting source:"tau" events + a tool_call intent on the pre-exec hook so blocked calls still appear. Guard enforcement unchanged.
4. doctor: "observability wired" check per framework.
5. dashboard: harness source surfaced on tool-call activity.
6. Tests (+14): end-to-end hook payload → event → dashboard state; no-run no-op; malformed stdin never throws; secret safety.

Session-safety invariants (this runs inside users' live sessions):

• ALWAYS exit 0 — internal errors are swallowed and logged, never surfaced as hook failures; a post-tool hook must never disrupt the user.
• No active run = silent no-op; .rstack/runs is never created by observation.
• Secret redaction: secret-looking keys dropped, content/body fields collapsed to length notes, inline credentials (AWS keys, GitHub/Slack tokens, JWTs, password=/token: patterns) replaced with [redacted], ~1200-char truncation matching Pi. Tested with real credential patterns + a 50KB payload.

Gates: typecheck, 847/847 tests, lint, validate (196), security-audit, diff-check, Tau AST — all green.

Adversarial review of the session-safety + redaction surface follows on this PR before merge.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added a new observe command to record tool activity and session events.
  • Expanded Claude Code and Tau integrations to emit post-call observability data.
  • Dashboard activity feeds now show source-aware tool bursts.

• Bug Fixes

  • Improved redaction and size limiting for sensitive or oversized event data.
  • observe is best-effort and will not interrupt a session, even on errors or missing run state.

• Documentation

  • Updated integration docs and setup guidance to cover the new observability flow.

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

[coderabbitai]

Warning

Review limit reached

@richard-devbot, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 49 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d27d8815-667a-460c-a6e9-8453cd59c3cc

📥 Commits

Reviewing files that changed from the base of the PR and between b7c0e30 and d0ec358.

📒 Files selected for processing (1)

• tests/observe-cli.test.js

📝 Walkthrough

Walkthrough

Adds a new rstack-agents observe CLI command that normalizes hook payloads (Claude Code, Tau, generic) into events.jsonl records, wires PostToolUse/Stop/SessionEnd hooks for Claude Code and equivalent Tau adapter emission, adds doctor checks for the new wiring, updates dashboard feed source labeling, and documents/tests the behavior.

Changes

Cross-harness observability

Layer / File(s)	Summary
observe.js core event writer src/commands/observe.js	New module normalizes/sanitizes hook payloads, redacts secrets, appends locked JSONL events, and exposes non-throwing runObserve/runObserveCommand and stdin reading.
CLI entrypoint bin/rstack-agents.js	Adds observe subcommand with flags for event type, tool, summary, error, source, project/run selection, and verbose mode, always exiting 0.
Claude Code hook wiring and docs src/integrations/init.js, docs/integrations/claude-code.md, tests/integrations-init.test.js	Extends CLAUDE_CODE_HOOKS with PostToolUse/Stop/SessionEnd hooks invoking observe, updates init report guidance, documentation, and init tests.
Doctor observability checks src/commands/doctor.js, tests/doctor.test.js	Adds OBSERVE_HOOK_SNIPPET, extends checkClaudeCodeWiring and checkAdapterWiring/checkTauObservability to WARN/PASS on observability wiring, with matching tests.
Tau adapter emission src/integrations/tau/rstack_sdlc.py	Adds _emit_observation helper and wires PreToolUse/PostToolUse observability emission into tool_call/tool_result hooks.
Dashboard feed source labeling src/observability/dashboard/state/feed.js	buildActivityFeed tracks per-source burst counts and labels summaries/data with dominant non-pi source.
Integration and unit tests tests/observe-cli.test.js	Adds end-to-end CLI subprocess tests for event writing, redaction, no-op behavior, run targeting, and dashboard propagation, plus unit tests for normalization/sanitization.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Harness as Claude Code / Tau
  participant Hook as PostToolUse Hook
  participant ObserveCLI as rstack-agents observe
  participant Normalize as normalizeObservation
  participant EventsFile as events.jsonl
  participant Dashboard

  Harness->>Hook: tool call/result payload
  Hook->>ObserveCLI: pipe JSON via stdin
  ObserveCLI->>Normalize: parse & sanitize payload
  Normalize-->>ObserveCLI: normalized event or null
  alt active run exists
    ObserveCLI->>EventsFile: append event with source label
  else no active run
    ObserveCLI-->>Hook: silent no-op
  end
  ObserveCLI-->>Hook: exit code 0
  Dashboard->>EventsFile: poll events.jsonl
  EventsFile-->>Dashboard: normalized events

Loading

Poem

A rabbit hops through terminal grass,
watching every tool call pass 🐇
"observe!" I cry, and events take flight,
from Tau and Claude into dashboard light.
No secrets leak, no session breaks —
just quiet logs where the hare awakes.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR covers the observe flow, Claude Code wiring, Tau emission, dashboard labeling, and tests, but it doesn't show the requested Operator wiring or docs.	Add Operator observability wiring or document how it is handled, then verify the same no-op, source labeling, and event-append behavior for that harness.
Docstring Coverage	⚠️ Warning	Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: cross-harness observability hooks that mirror terminal activity in the dashboard.
Out of Scope Changes check	✅ Passed	The changed files all support observability, wiring, dashboard attribution, or tests, with no unrelated feature work visible.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/observability-hooks-251

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[richard-devbot]

Adversarial security review — PASSED, no blockers

Independent attack-first review focused on the two launch-killer classes for a hook that runs in every user session. Zero BLOCKING/HIGH findings.

Session disruption (highest priority) — HELD: every path in observe.js is wrapped; the CLI action forces exit 0 on all paths incl. catch — no input, missing/locked/corrupt .rstack, or malformed stdin produces a nonzero exit, throw, or hang. withFileLock serializes appends (25ms retry, 10s stale-break → bounded, always progresses, no infinite hang). No network, one O(1) append on the hot path. Tau observe is best-effort and can't block a tool call. No active run = silent no-op, never creates run state.

Secret leakage — HELD: redaction runs before truncation; secret-looking keys dropped, content/body/new_string collapsed to length notes, inline credential patterns (AWS keys, GitHub/Slack tokens, JWTs, bearer, password=/token:) replaced with [redacted]. File content never echoed. Verified with AWS key, password field, PEM, JWT, and a 50KB payload.

Integrity — HELD: events are JSON-encoded (crafted tool names/--source can't inject a second JSONL line), run-id resolution is safe, no run-state mutation / metric double-count.

Follow-through: the one reviewer recommendation worth acting on — a concurrency test for parallel bursts (PostToolBatch) — was added (d0ec358): 12 parallel observe writes to one run, all exit 0, all 12 lines land as valid JSON, none lost. 848 tests green.

Reviewer verdict: solid, strictly additive, no risk to enforcement or user sessions. Ready to merge.

🤖 Generated with Claude Code

[coderabbitai]

🧹 Nitpick comments (3)

src/observability/dashboard/state/feed.js (1)

10-38: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Per-source burst tracking and dominant-source labeling logic is correct.

The dual-map approach (toolBursts + burstSources) is sound — both are populated in the same loop so burstSources[min] is guaranteed non-empty when count >= 3. Edge cases (missing ts, non-string/empty source) are properly guarded.

One minor inconsistency: data on line 38 is set to { source: dominant } even when dominant === 'pi', but the via label on line 29 is only added for non-pi sources. This means pi-dominant bursts get data: { source: 'pi' } without a corresponding human-readable label. Consider aligning the data condition with the via condition for consistency.

♻️ Optional alignment for data field condition

-          data: dominant ? { source: dominant } : undefined,
+          data: via ? { source: dominant } : undefined,

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/observability/dashboard/state/feed.js` around lines 10 - 38, The burst
feed labeling in feed.js is inconsistent because the `data` payload on the
`activityFeed.push` entry still includes `source: dominant` even when the
dominant source is `pi`, while the human-readable `via` suffix is only shown for
non-pi sources. Update the `activityFeed.push` logic in the tool-burst
aggregation block so the `data` field follows the same condition as `via`, using
the `dominant` source only when it is a real harness source and omitting it for
`pi`.

src/commands/observe.js (1)

119-119: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Content-key matcher is substring-based and over-redacts common keys.

/(content|contents|body|data|...)/i matches any key containing the substring, so benign fields like metadata, database, or formData get their values collapsed to [N chars omitted]. Since the point of this writer is dashboard fidelity, consider anchoring to whole-key names to avoid silently dropping useful, non-secret input.

♻️ Anchor to exact key names

-    if (/(content|contents|body|data|new_string|old_string)/i.test(key)) {
+    if (/^(content|contents|body|data|new_string|old_string)$/i.test(key)) {
       out[key] = typeof value === 'string' ? `[${value.length} chars omitted]` : REDACTED;
       continue;
     }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/observe.js` at line 119, The key matcher in the observe writer
is too broad because the regex in the redaction check matches substrings,
causing non-sensitive fields like metadata or database to be collapsed. Update
the condition in observe.js to match only exact key names in the content-key
path, using the existing key variable in the same redaction block, so only
intended fields are redacted and dashboard fidelity is preserved.

src/integrations/tau/rstack_sdlc.py (1)

366-389: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Consider making _emit_observation non-blocking to avoid adding latency to every tool call.

await proc.communicate(...) blocks the caller until the observe subprocess finishes. In the tool_call hook this adds a full subprocess round-trip to the critical path before the guard even runs, and in tool_result it delays delivering the result back to the user. With no timeout, a slow npx invocation can hang a Tau session indefinitely.

Since the observe result is never needed, fire-and-forget via asyncio.create_task() would remove this from the critical path while preserving the best-effort contract. Adding a timeout as a safety net would also help.

♻️ Suggested non-blocking emission

 async def _emit_observation(payload: dict, cwd: str) -> None:
     """Feed one observability event to `rstack-agents observe` (`#251`).
     ...
     """
     npx = shutil.which("npx")
     if npx is None:
         return
     try:
+        data = json.dumps(payload).encode("utf-8")
         proc = await asyncio.create_subprocess_exec(
             npx, "--yes", "rstack-agents", "observe", "--source", "tau", "--project", cwd,
             stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL,
             env=os.environ, cwd=cwd,
         )
-        await proc.communicate(json.dumps(payload).encode("utf-8"))
+        proc.stdin.write(data)
+        await proc.stdin.drain()
+        proc.stdin.close()
+        # Reap the process in the background — prevents zombies without blocking the caller.
+        asyncio.create_task(proc.wait())
     except Exception:
         pass  # observability is additive — never let it break a session

Then in the tool_call and tool_result hooks, replace await _emit_observation(...) with asyncio.create_task(_emit_observation(...)) to avoid blocking the hook entirely. Since _emit_observation already swallows all exceptions, the fire-and-forget task is safe.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/integrations/tau/rstack_sdlc.py` around lines 366 - 389, The
_emit_observation path is still blocking the caller because it awaits
proc.communicate, which can add latency or hang the Tau session. Update
_emit_observation in rstack_sdlc.py to be truly best-effort by making the hook
fire-and-forget with asyncio.create_task when called from the tool_call and
tool_result hooks, and add a timeout or other safety guard around the subprocess
wait so slow npx observe invocations cannot block the critical path. Keep the
existing exception swallowing behavior so observability remains non-disruptive.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/commands/observe.js`:
- Line 119: The key matcher in the observe writer is too broad because the regex
in the redaction check matches substrings, causing non-sensitive fields like
metadata or database to be collapsed. Update the condition in observe.js to
match only exact key names in the content-key path, using the existing key
variable in the same redaction block, so only intended fields are redacted and
dashboard fidelity is preserved.

In `@src/integrations/tau/rstack_sdlc.py`:
- Around line 366-389: The _emit_observation path is still blocking the caller
because it awaits proc.communicate, which can add latency or hang the Tau
session. Update _emit_observation in rstack_sdlc.py to be truly best-effort by
making the hook fire-and-forget with asyncio.create_task when called from the
tool_call and tool_result hooks, and add a timeout or other safety guard around
the subprocess wait so slow npx observe invocations cannot block the critical
path. Keep the existing exception swallowing behavior so observability remains
non-disruptive.

In `@src/observability/dashboard/state/feed.js`:
- Around line 10-38: The burst feed labeling in feed.js is inconsistent because
the `data` payload on the `activityFeed.push` entry still includes `source:
dominant` even when the dominant source is `pi`, while the human-readable `via`
suffix is only shown for non-pi sources. Update the `activityFeed.push` logic in
the tool-burst aggregation block so the `data` field follows the same condition
as `via`, using the `dominant` source only when it is a real harness source and
omitting it for `pi`.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 55b498a5-0013-4277-bd21-d12775252f9d

📥 Commits

Reviewing files that changed from the base of the PR and between 3ff0c92 and b7c0e30.

📒 Files selected for processing (10)

• bin/rstack-agents.js
• docs/integrations/claude-code.md
• src/commands/doctor.js
• src/commands/observe.js
• src/integrations/init.js
• src/integrations/tau/rstack_sdlc.py
• src/observability/dashboard/state/feed.js
• tests/doctor.test.js
• tests/integrations-init.test.js
• tests/observe-cli.test.js