fix(opencode): raise review cadence so the direct-OpenAI slot can finish substantive reviews#517
fix(opencode): raise review cadence so the direct-OpenAI slot can finish substantive reviews#517seonghobae wants to merge 4 commits into
Conversation
…ish substantive reviews Since the GitHub Models org budget cap, gpt-5.6-luna is the only model-pool candidate producing verdicts, and the dynamic review cadence tightened in #495 (small 300s/600s, medium 420s/900s, large 420s/1080s) no longer fits it: on 2026-07-13 (~04:10Z onward) every substantive PR across the org failed opencode-review with MODEL_OUTPUT_UNAVAILABLE because luna timed out mid-review (exit 124 after producing hundreds of KB of events) while only one-line dependabot diffs passed. The org merge queue stalled. - raise per-attempt/total budgets: small 900s/1500s, medium 1500s/2400s, large 1800s/2700s, unknown 1500s/2400s - raise the pool step shell watchdog 1080s -> 2880s, the pool step cap 20 -> 50 minutes, and the review job cap 45 -> 75 minutes so the shell watchdog still fires before the runner kills the step - keep the queue-protection structure of #495 intact: one attempt per model, single catalog cycle, outer watchdog, continue-on-error - update the contract pins in tests/test_opencode_agent_contract.py and scripts/ci/test_strix_quick_gate.sh in lockstep Verified: pytest tests/ -> 377 passed, 9 skipped on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OpenCode Review Overview
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> Conflict["Merge conflict blocks this path"]
Conflict --> V3["targeted test run"]
Merge Conflict Guidance
gh pr checkout 517 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:fix/opencode-review-cadence-luna
# rebase path only: git push --force-with-lease origin HEAD:fix/opencode-review-cadence-luna |
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
Adversarial validation
{"status":"passed","probes":[{"path":".github/workflows/opencode-review.yml","line":1520,"hypothesis":"Increased timeouts might lead to resource exhaustion.","attack_or_counterexample":"Simulate high-load scenarios.","evidence":"No test failures observed; workflow remains bounded by outer watchdog.","outcome":"falsified"},{"path":"scripts/ci/test_strix_quick_gate.sh","line":560,"hypothesis":"Adjusted timeout assertions might miss edge cases.","attack_or_counterexample":"Verify assertions in test script.","evidence":"Tests pass with updated timeout values.","outcome":"falsified"}],"residual_risk":"Low; workflow remains bounded by outer watchdog."}- Result: APPROVE
- Reason: Adjusted timeouts and cadence settings improve workflow reliability under budget constraints.
- Head SHA:
4f52825932ddccb3f9a5db39659f4a3645a6c7b5 - Workflow run: 29228574116
- Workflow attempt: 1
There was a problem hiding this comment.
Pull request overview
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
Findings
No blocking findings.
Adversarial validation
{"status":"passed","probes":[{"path":".github/workflows/opencode-review.yml","line":1520,"hypothesis":"Increased timeout settings may lead to resource exhaustion.","attack_or_counterexample":"Simulated high-load review scenario.","evidence":"Verified workflow handles increased timeouts without resource leaks.","outcome":"falsified"},{"path":"scripts/ci/test_strix_quick_gate.sh","line":560,"hypothesis":"Updated timeout assertions may not match new settings.","attack_or_counterexample":"Checked assertions against new timeout values.","evidence":"Assertions correctly reflect updated timeout settings.","outcome":"falsified"}],"residual_risk":"Low; changes are well-tested and bounded by existing safeguards."}Evidence
- Result: APPROVE
- Reason: No blocking issues found in the current-head evidence.
- Scope:
central OpenCode/Strix review-process - Changed files:
3 - Head SHA:
b831984a116b57d21f3ba4ca92c3f77b8db9f327 - Workflow run: 29231891208
- Workflow attempt: 1
This approval path is limited to ContextualWisdomLab/.github central review-process self-repair.
|
Closing as superseded by main's subsequent evolution. This PR raised the review cadence so gpt-5.6-luna (then the only live candidate) could finish substantive reviews during the 2026-07-13 GitHub Models budget outage. Since it was opened, main has adopted a better structural fix for the same failure mode: #518 allows deterministic clean-evidence approval on model outage (so the queue no longer stalls when the pool dies), the GitHub Models org budget was restored (deepseek verdicts are back in ~1-3 min), a long-context github-models/openai/gpt-4.1 candidate was added after luna, and the cadence was deliberately tightened further (90-360s) to fail fast into the fallback instead of holding runners. Merging this raise on top would undo that intentional tightening. The outage diagnosis this PR documented (luna exit-124 mid-review at 420s, exhausted pool, org-wide stall) remains recorded in the PR body for future reference. 🤖 Generated with Claude Code |
Root cause (org merge-queue stall, 2026-07-13 ~04:10Z onward)
With the GitHub Models org budget capped,
openai/gpt-5.6-lunais the only model-pool candidate that produces verdicts — everygithub-models/*candidate dies instantly on "budget limit". The dynamic review cadence tightened in #495 gives a medium change only 420 s per attempt / 900 s total. Luna demonstrably works (fast-mlsirm PR #152 run: ~400 KB of review events) but cannot finish a substantive multi-file review in that window →exit 124→ poolexhausted→MODEL_OUTPUT_UNAVAILABLE→ required check fails.Observed effect: since ~04:10Z, every substantive PR org-wide (fast-mlsirm #152/#153, bolt, etc.) failed
opencode-reviewidentically across four separate attempts, while one-line dependabot diffs (which fit in 300 s) kept passing. Reruns cannot help — the window is structurally too small.Change
timeout-minutestimeout-minutesThe queue-protection structure of #495 is intact: one attempt per model, single catalog cycle (
OPENCODE_POOL_MAX_CYCLES: "1"), outer shell watchdog that marks the pool exhausted before the runner kills the step,continue-on-erroron the pool step. Only the per-attempt room grows. Worst-case runner hold rises from 18 min to 48 min for the pool step — bounded, and far cheaper than the current state where every substantive PR burns 15+ min to a guaranteed failure and blocks the queue indefinitely. When the GitHub Models budget is restored (or #511's early-kill watchdog lands), fast candidates soak up most reviews again and luna rarely needs the full window.Contract pins in
tests/test_opencode_agent_contract.pyandscripts/ci/test_strix_quick_gate.shupdated in lockstep.Verification
pytest tests/on this branch: 377 passed, 9 skippedtest_strix_quick_gate.sh(e.g."180","540", pre-luna candidate order) fail on pristinemaintoo and are untouched — out of scope.Note on this PR's own review
This diff touches the ~6k-line
opencode-review.yml, which is known to overflow luna's context — the model pool cannot review it directly. All three changed files are on the central review-process scope allowlist, so the deterministic current-head evidence fallback (working since #510) is the intended approval path.🤖 Generated with Claude Code