Skip to content

fix(opencode): raise review cadence so the direct-OpenAI slot can finish substantive reviews#517

Closed
seonghobae wants to merge 4 commits into
mainfrom
fix/opencode-review-cadence-luna
Closed

fix(opencode): raise review cadence so the direct-OpenAI slot can finish substantive reviews#517
seonghobae wants to merge 4 commits into
mainfrom
fix/opencode-review-cadence-luna

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Root cause (org merge-queue stall, 2026-07-13 ~04:10Z onward)

With the GitHub Models org budget capped, openai/gpt-5.6-luna is the only model-pool candidate that produces verdicts — every github-models/* candidate dies instantly on "budget limit". The dynamic review cadence tightened in #495 gives a medium change only 420 s per attempt / 900 s total. Luna demonstrably works (fast-mlsirm PR #152 run: ~400 KB of review events) but cannot finish a substantive multi-file review in that window → exit 124 → pool exhaustedMODEL_OUTPUT_UNAVAILABLE → required check fails.

Observed effect: since ~04:10Z, every substantive PR org-wide (fast-mlsirm #152/#153, bolt, etc.) failed opencode-review identically across four separate attempts, while one-line dependabot diffs (which fit in 300 s) kept passing. Reruns cannot help — the window is structurally too small.

Change

knob old (#495) new
small (≤3 files) run / total 300 s / 600 s 900 s / 1500 s
medium (≤20) run / total 420 s / 900 s 1500 s / 2400 s
large run / total 420 s / 1080 s 1800 s / 2700 s
unknown run / total 420 s / 900 s 1500 s / 2400 s
pool step shell watchdog 1080 s 2880 s
pool step timeout-minutes 20 50
review job timeout-minutes 45 75

The queue-protection structure of #495 is intact: one attempt per model, single catalog cycle (OPENCODE_POOL_MAX_CYCLES: "1"), outer shell watchdog that marks the pool exhausted before the runner kills the step, continue-on-error on the pool step. Only the per-attempt room grows. Worst-case runner hold rises from 18 min to 48 min for the pool step — bounded, and far cheaper than the current state where every substantive PR burns 15+ min to a guaranteed failure and blocks the queue indefinitely. When the GitHub Models budget is restored (or #511's early-kill watchdog lands), fast candidates soak up most reviews again and luna rarely needs the full window.

Contract pins in tests/test_opencode_agent_contract.py and scripts/ci/test_strix_quick_gate.sh updated in lockstep.

Verification

  • pytest tests/ on this branch: 377 passed, 9 skipped
  • Pre-existing stale pins in test_strix_quick_gate.sh (e.g. "180", "540", pre-luna candidate order) fail on pristine main too and are untouched — out of scope.

Note on this PR's own review

This diff touches the ~6k-line opencode-review.yml, which is known to overflow luna's context — the model pool cannot review it directly. All three changed files are on the central review-process scope allowlist, so the deterministic current-head evidence fallback (working since #510) is the intended approval path.

🤖 Generated with Claude Code

…ish substantive reviews

Since the GitHub Models org budget cap, gpt-5.6-luna is the only model-pool
candidate producing verdicts, and the dynamic review cadence tightened in
#495 (small 300s/600s, medium 420s/900s, large 420s/1080s) no longer fits
it: on 2026-07-13 (~04:10Z onward) every substantive PR across the org
failed opencode-review with MODEL_OUTPUT_UNAVAILABLE because luna timed out
mid-review (exit 124 after producing hundreds of KB of events) while only
one-line dependabot diffs passed. The org merge queue stalled.

- raise per-attempt/total budgets: small 900s/1500s, medium 1500s/2400s,
  large 1800s/2700s, unknown 1500s/2400s
- raise the pool step shell watchdog 1080s -> 2880s, the pool step cap
  20 -> 50 minutes, and the review job cap 45 -> 75 minutes so the shell
  watchdog still fires before the runner kills the step
- keep the queue-protection structure of #495 intact: one attempt per
  model, single catalog cycle, outer watchdog, continue-on-error
- update the contract pins in tests/test_opencode_agent_contract.py and
  scripts/ci/test_strix_quick_gate.sh in lockstep

Verified: pytest tests/ -> 377 passed, 9 skipped on this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@seonghobae seonghobae enabled auto-merge (squash) July 13, 2026 06:18
@opencode-agent

opencode-agent Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: b831984a116b57d21f3ba4ca92c3f77b8db9f327
  • Workflow run: 29231891208
  • Workflow attempt: 1
  • Gate result: APPROVE (exit 0)

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: opencode-review.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["actionlint plus required checks"]
  Evidence --> S2["CI script: test_strix_quick_gate.sh"]
  S2 --> I2["review and security gate shell path"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["bash -n plus Strix self-test"]
  Evidence --> S3["Test: test_opencode_agent_contract.py"]
  S3 --> I3["regression suite"]
  I3 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V3["targeted test run"]
Loading

Merge Conflict Guidance

  • Current merge state: DIRTY
  • Base branch: main
  • Head branch: fix/opencode-review-cadence-luna
  • Fix direction: merge or rebase origin/main into fix/opencode-review-cadence-luna, resolve conflict markers in the changed files, rerun the focused checks, then push the same branch.
  • Repair commands:
gh pr checkout 517 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main  # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:fix/opencode-review-cadence-luna
# rebase path only: git push --force-with-lease origin HEAD:fix/opencode-review-cadence-luna

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode reviewed the current-head bounded evidence and found no blocking issues.

Findings

No blocking findings.

Summary

Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.

Adversarial validation

{"status":"passed","probes":[{"path":".github/workflows/opencode-review.yml","line":1520,"hypothesis":"Increased timeouts might lead to resource exhaustion.","attack_or_counterexample":"Simulate high-load scenarios.","evidence":"No test failures observed; workflow remains bounded by outer watchdog.","outcome":"falsified"},{"path":"scripts/ci/test_strix_quick_gate.sh","line":560,"hypothesis":"Adjusted timeout assertions might miss edge cases.","attack_or_counterexample":"Verify assertions in test script.","evidence":"Tests pass with updated timeout values.","outcome":"falsified"}],"residual_risk":"Low; workflow remains bounded by outer watchdog."}
  • Result: APPROVE
  • Reason: Adjusted timeouts and cadence settings improve workflow reliability under budget constraints.
  • Head SHA: 4f52825932ddccb3f9a5db39659f4a3645a6c7b5
  • Workflow run: 29228574116
  • Workflow attempt: 1

@opencode-agent opencode-agent Bot disabled auto-merge July 13, 2026 07:26

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.

Findings

No blocking findings.

Adversarial validation

{"status":"passed","probes":[{"path":".github/workflows/opencode-review.yml","line":1520,"hypothesis":"Increased timeout settings may lead to resource exhaustion.","attack_or_counterexample":"Simulated high-load review scenario.","evidence":"Verified workflow handles increased timeouts without resource leaks.","outcome":"falsified"},{"path":"scripts/ci/test_strix_quick_gate.sh","line":560,"hypothesis":"Updated timeout assertions may not match new settings.","attack_or_counterexample":"Checked assertions against new timeout values.","evidence":"Assertions correctly reflect updated timeout settings.","outcome":"falsified"}],"residual_risk":"Low; changes are well-tested and bounded by existing safeguards."}

Evidence

  • Result: APPROVE
  • Reason: No blocking issues found in the current-head evidence.
  • Scope: central OpenCode/Strix review-process
  • Changed files: 3
  • Head SHA: b831984a116b57d21f3ba4ca92c3f77b8db9f327
  • Workflow run: 29231891208
  • Workflow attempt: 1

This approval path is limited to ContextualWisdomLab/.github central review-process self-repair.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Closing as superseded by main's subsequent evolution. This PR raised the review cadence so gpt-5.6-luna (then the only live candidate) could finish substantive reviews during the 2026-07-13 GitHub Models budget outage. Since it was opened, main has adopted a better structural fix for the same failure mode: #518 allows deterministic clean-evidence approval on model outage (so the queue no longer stalls when the pool dies), the GitHub Models org budget was restored (deepseek verdicts are back in ~1-3 min), a long-context github-models/openai/gpt-4.1 candidate was added after luna, and the cadence was deliberately tightened further (90-360s) to fail fast into the fallback instead of holding runners. Merging this raise on top would undo that intentional tightening. The outage diagnosis this PR documented (luna exit-124 mid-review at 420s, exhausted pool, org-wide stall) remains recorded in the PR body for future reference.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant