docs(skills): tell agents to file reproducible render bugs, not paraphrases#2301
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. |
vanceingalls
force-pushed
the
07-12-feedback-skill-bug-repro-guidance
branch
from
1b4be0c to
5e4ad0e
Compare
vanceingalls
changed the base branch from
graphite-base/2301
to
07-12-fix-software-video-beyond-viewport
miga-heygen
left a comment
miga-heygen
left a comment
There was a problem hiding this comment.
Review — docs(skills): tell agents to file reproducible render bugs, not paraphrases
Docs/skill-only change. Good improvements to the feedback guidance:
- Lead with
--file-issue— makes it the default for reproducible bugs instead of a footnote. The published composition is what lets a maintainer reproduce; a comment alone almost never does. - Concrete bug checklist — exact error string verbatim, failure mode (output produced / fell back / hard-exited), isolated trigger, exact command + env, frame/timestamp + visual defect. Each item answers a specific question a maintainer would ask.
- Dropped "repeat env" — correct, the CLI already attaches os/arch/mem/ffmpeg/version via doctorSummary.
Consistent between SKILL.md and the reference file. No code change.
LGTM — no issues found.
— Miga
james-russo-rames-d-jusso
left a comment
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
Reviewed at 5e4ad0e2.
Verified against actual CLI (packages/cli/src/commands/feedback.ts): all flags in the guidance exist (--file-issue, --dir, --yes), the CLI auto-attaches doctorSummary (os/arch/mem/ffmpeg/CLI-version) so "don't repeat env" is accurate, and the error-string examples all appear in the codebase (re-rendering via screenshot at renderOrchestrator.ts:2611, falling back at captureStreamingStage.ts:699, Killed: 9 / Target closed are real Node/Puppeteer signals). The trigger examples (blank CJK, 4K, <video>) are all real reports the team hit this arc — grounded, not made up.
Content parity between SKILL.md (structured sub-bullets) and references/preview-render.md (compressed prose) is consistent — same information at two levels of detail, which matches the file-role split.
The --file-issue framing correctly emphasizes "the published composition is what lets a maintainer reproduce; a comment alone almost never does" — that IS the load-bearing message. Nice.
Nits
- 🟡 One agent-safety subtlety worth noting:
preview-render.mdmentions--yes"for non-interactive shells" but doesn't call out that--yesbypasses the consent-gate (seeconfirmFileIssueatfeedback.ts:56). The CLI still prints the consent message before publishing, but a non-interactive agent using--yespublishes without a review step — so if the composition contains proprietary customer content (voice-over scripts, unreleased brand assets), it goes public. Not a blocker on a docs PR, but a one-line "consider whether the composition contains sensitive content before adding--yesin agent flows" note would be defensive against future misuse. Alternatively, guide agents to prefer the interactive path when running against non---dirworkspaces.
LGTM. Magi already approved; layering this on top per the peer-review protocol.
jrusso1020
left a comment
jrusso1020
left a comment
There was a problem hiding this comment.
Approving on Rames-D's go. Rames-D reviewed + Magi APPROVED at head 5e4ad0e, docs PR with CLI flags + error-strings verified real, no changes-requested. (--yes consent-gate note is non-blocking.)
miguel-heygen
left a comment
miguel-heygen
left a comment
There was a problem hiding this comment.
Independent current-head review.
Strengths
skills/hyperframes-cli/SKILL.md:42-49makes reproducible reports concrete while preserving the existing telemetry opt-out.- The guidance matches the actual CLI:
--file-issue,--dir, and--yesexist; the command publishes only after consent and opens a non-submitted issue draft. - Skills lint passed, and the full skill suite passed 225/225 locally. The stack edge is exact/fresh; CI has no failures, with optional render-regression shards still running.
Important (non-blocking)
- Rule-9 follow-up: this changes LLM-facing instruction text but adds no positive-pin test for the new reproducibility concepts. A future focused skill test should assert that bug guidance retains
--file-issue, exact errors, the command/env, and frame/timestamp context.
No material blockers found.
Verdict: APPROVE
Reasoning: The instructions accurately describe the implemented consent-gated CLI flow and materially improve report reproducibility; the missing prompt pin is test-hardening, not a correctness blocker.
— deepwork
…stalls (#2298) ## Stack 1. **#2298 — DE router stall watchdog** ← you are here 2. #2300 — video bottom-edge clip 3. #2301 — feedback bug-report guidance ## Problem The DE parallel router auto-enables the interleaved parallel-**streaming** capture for the ≥24 GB macOS trial cohort. If a worker wedges mid-capture (a hung seek/screenshot at an early frame), the render makes **zero frame progress** yet sits until the per-frame CDP `protocolTimeout` (~5 min) fires before the pinned self-verify fallback can run — a silent multi-minute hang shipped to real users. Reported: stuck at frame 2/2031 for 6+ min, no fallback, until the user manually set `HF_DE_PARALLEL_ROUTER=false` (71 s clean). ## Fix Add a no-frame-progress watchdog to the parallel branch of `runCaptureStreamingStage`: - Ticks off `executeParallelCapture`'s progress callback. If no **new** frame lands within `HF_DE_PARALLEL_STALL_MS` (default **60 s** — well under the 5-min protocol timeout, ≫ the 15–32 ms/frame budget), it fires. - On trip: aborts the **reorder buffer** (so peer workers parked in `waitForFrame` reject instead of deadlocking the `Promise.all` pool) and aborts the pool via a **separate** `AbortController` linked to the parent abort. - The parent `abortSignal` stays un-aborted, so the orchestrator reads the failure as a generic `capture_error` (not a cancellation) and re-renders on the pinned screenshot path — the same fallback a verify failure already uses. ## Test - Watchdog trips on no progress → rethrows a stall error (routes to fallback). - A genuine parent-abort is **not** relabeled as a stall (stays a cancellation).
…hrases The `hyperframes feedback` convention only prompted for a free-text `--comment` "with the failing composition pattern and what you tried". Agents dutifully filed vague reports (blank CJK text, mid-run exit, 4K timeout) with no error string, no failure-mode, and — critically — no published composition, so none could be reproduced or root-caused. Two additions to the CLI skill: - Lead bug reports with `--file-issue` (+ `--dir`), which publishes a minimal repro of the project to a public URL. A comment alone almost never lets a maintainer reproduce; the composition is what does. - Give the `--comment` a concrete bug checklist: exact error string verbatim + whether output was produced / fell back / hard-exited; the isolated trigger; exact command + HF_*/PRODUCER_* env; frame/timestamp + visual defect. Drop the "repeat env" ask (the CLI already attaches it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vanceingalls
force-pushed
the
07-12-fix-software-video-beyond-viewport
branch
from
18a1d1f to
de95daa
Compare
vanceingalls
force-pushed
the
07-12-feedback-skill-bug-repro-guidance
branch
from
5e4ad0e to
6a3da1f
Compare
vanceingalls
dismissed stale reviews from miguel-heygen and jrusso1020
The base branch was changed.
5 participants
Stack
Problem
The
hyperframes feedbackconvention only prompted agents for a free-text--comment"with the failing composition pattern and what you tried". Agents dutifully filed vague reports (blank CJK text, DE mid-run exit, 4K timeout) with no error string, no failure-mode, and no published composition — so none could be reproduced or root-caused. Three of the six reports in the last batch were un-actionable for exactly this reason.Fix
Two additions to the
hyperframes-cliskill (SKILL.md+references/preview-render.md):--file-issue(+--dir), which publishes a minimal repro of the project to a public URL and opens a pre-filled GitHub issue. A comment alone almost never lets a maintainer reproduce; the published composition is what does.--commentbug checklist: exact error string verbatim + whether output was produced / silently fell back / hard-exited; the isolated trigger; exact command +HF_*/PRODUCER_*env; frame/timestamp + visual defect. Dropped the "repeat env" ask (the CLI already attaches os/arch/mem/ffmpeg/version).No behavior change — docs/skill only.
skills-manifest.jsonhash bump is the registry tracking the skill edit.