feat(cli): flowproof doctor, a wiring check before you spend a key - #182
Merged
Conversation
Adoption splits in two. The AUDIT half is judgement - is this tool
destructive, does the grandchild inherit env - and cannot be mechanised.
The VERIFICATION half is mechanical, and its dominant failure is a client
that never reaches the proxy. Today that is discovered only after a spec has
been written and a key spent on `record`.
flowproof doctor --agent "./start-agent"
Starts the proxy, runs the command once against a synthetic one-turn
cassette, and reports how many model requests ARRIVED. No spec, no
assertions, no key: the canned reply comes from the proxy. A request that
DIVERGES from that cassette still arrived, which is the fact being measured.
It never prints a green verdict, and that is deliberate. An agent with more
than one client can reach the proxy with one and the real provider with
another, so an arriving request proves A client found it, not that all of
them did. It reports the observation and says `record` is what settles it -
the same posture as the zero-capture guard, which names causes as
possibilities rather than facts.
Zero arrivals prints the injected variables, the two `${flowproof.*}`
handles to map, and the reminder that a model call in a CHILD process needs
that child to inherit the environment.
`init` was considered and rejected in the design pass: the hard part of
adoption is judgement, and a scaffold that guesses the command or the
tools/mcp boundary wrong sends an adopter down a false path with false
confidence. The YAML it would save is six lines.
FOUND WHILE TESTING THIS, and reported rather than hidden: the agent
deadline stops the process flowproof started, not the tree. An agent that
spawns a child which outlives it keeps the inherited stdout pipe open, so
the wall clock can exceed --timeout by a lot. Verified: `exec sleep` (no
grandchild) honours the timeout exactly; a spawned `sleep` does not. That
affects `record` and `run` too, so it is a behaviour change to shared code
and wants its own design pass. Both the command output and the docs say so
rather than letting it look like an unexplained hang.
Tests: three e2e - a wired agent passes, an agent reading its own variable
FAILS (the case the command exists for; a green doctor there would send
someone off to waste a key), and a command that cannot start is an error.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Do not merge without review - this is new CLI surface, which the loop's mandate puts in the open-PR-do-not-merge class. The design pass is recorded in
LOOP-STATE.mdand summarised below.The design call:
doctor, notinitAdoption splits in two. The audit half is judgement (is this tool destructive? does the grandchild inherit env?) and cannot be mechanised. The verification half is mechanical, and its dominant failure is a client that never reaches the proxy - discovered today only after a spec has been written and a key spent on
record.initwas considered and rejected. The hard part is judgement, and detecting "the agent entry point" in an arbitrary repo is unreliable. A scaffold that guesses the command or thetools:/mcp:boundary wrong sends an adopter down a false path with false confidence, which is the one artifact this product must not ship. The YAML it would save is six lines.What it does
flowproof doctor --agent "./start-agent"Starts the proxy, runs the command once against a synthetic one-turn cassette, and reports how many model requests arrived. No spec, no assertions, no key - the canned reply comes from the proxy. A request that diverges from that cassette still arrived, which is the fact being measured.
Zero arrivals prints the injected variables, the two
${flowproof.*}handles to map, and the reminder that a model call in a child process needs that child to inherit the environment.It never prints a green verdict
Deliberate, and the sharpest point in the design audit. An agent with more than one client can reach the proxy with one and the real provider with another - so an arriving request proves a client found it, not that all of them did. It reports the observation and says
recordis what settles it. Same posture as the zero-capture guard, which names causes as possibilities rather than facts.A defect found while testing this, reported rather than hidden
The agent deadline stops the process flowproof started, not the tree. An agent that spawns a child which outlives it keeps the inherited stdout pipe open, so wall clock can exceed
--timeoutsubstantially.Verified both directions:
exec sleep 300(no grandchild) honours--timeout 5exactly; a spawnedsleep 300ran past three minutes.This affects
recordandruntoo, not justdoctor- and it is precisely the shape of a real adopter whose model call happens in a grandchild. Fixing it is a behaviour change to shared process handling, so it wants its own design pass rather than being slipped in here. Both the command output and the docs state the limit so it does not read as an unexplained hang.Tests
Three e2e: a wired agent passes; an agent reading its own variable fails (the case the command exists for - a green doctor there would send someone off to waste a key); a command that cannot start is an error. They run under the normal workspace test, so no browser-job listing is needed.
41 suites green, clippy clean under
--all-features.Also in here
A doc-comment placement bug I introduced and caught: the new variant absorbed
heal's description, leavingheal --helpblank. Fixed, and both are verified in the help output.🤖 Generated with Claude Code