Skip to content

test: make real Durable Object CI reliable and required#1648

Open
simple-agent-manager[bot] wants to merge 4 commits into
mainfrom
sam/current-main-real-durable-mdea0b
Open

test: make real Durable Object CI reliable and required#1648
simple-agent-manager[bot] wants to merge 4 commits into
mainfrom
sam/current-main-real-durable-mdea0b

Conversation

@simple-agent-manager

@simple-agent-manager simple-agent-manager Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes the real Cloudflare Workers test tier deterministic, visible, and merge-blocking through one stable Durable Object Workers result. Every discovered apps/api/tests/workers/**/*.test.ts file is assigned exactly once to a real-runtime shard; no files are mocked, skipped, or converted to success.

This is a fresh PR from current main. It does not reopen or reuse #1619, and it must not be merged without separate explicit authorization.

Reconciliation and root cause

PR #1619 closed unmerged at 7219669d, while SAM task 01KXQZXGB2VPVPET0JN1Y2F036 had been marked complete without an output summary. Final run 29721018453 / job 88283931687 used Node 22.23.1 and selected 36 files because final-head config replaced the seven-file DURABLE_OBJECT_GATE_FILES list with tests/workers/**/*.test.ts. It also changed main from tests/workers/worker-entrypoint.ts to src/index.ts and upgraded pool 0.14.0 to 0.16.18/workerd 1.20260617.1. The earlier 7-file/228-test result therefore did not exercise the final merge head.

Scientific minimization found:

  • Final-head reproduction: 36 files, zero collected tests, 36 worker-pool errors, repeated SIGSEGV.
  • maxWorkers=1 still crashed, ruling out parallel-worker pressure as the primary cause.
  • Repository pin: pool 0.14.0, Vitest 4.1.5, Miniflare 4.20260329.0, workerd 1.20260329.1 crashed when the full production graph was main.
  • Current upstream trial: pool 0.18.6 / workerd 1.20260714.1 also crashed, ruling out a defect limited to workerd 1.20260617.1.
  • Loading the production graph initializes unrelated Container/Sandbox/full-route modules; focused entrypoints using the same production DO classes and route modules run reliably. The causal condition is full-graph link/initialization in the pool, not the 36-file inventory, Node 22 alone, runner resources, or a single workerd release.

The solution uses Cloudflare's supported cloudflareTest({ main, miniflare }) configuration with focused binding/route entrypoints and one worker at a time. Official references: Vitest integration, configuration, isolation and concurrency, and test APIs.

Coverage and resource evidence

Shard Files Tests Local duration Real capabilities
durable-objects 27 449 158.58s DO RPC, SQLite SqlStorage, D1, alarms, hibernatable WebSockets, snapshots/recovery, node lifecycle
http 9 102 64.32s Real Hono routes, D1, ProjectData persistence, callback/node JWTs, deployment callbacks, TrialEventBus SSE, MCP, CORS
Total 36 551 222.90s sequential local Zero skipped tests; unexpected unhandled errors remain fatal

The restored vertical slices include task-scoped ACP stuck-task reconciliation, ProjectData listing with ~1,500 sessions plus substantial message history, origin/attention behavior, node lifecycle, and session snapshot/recovery.

The inventory guard recursively discovers test files from the repository and derives ownership from whether a file uses SELF; it fails if the inventory is empty or any file is unowned/duplicated. CI runs both shards with fail-fast: false, then aggregates their result without masking a failed shard. For unrelated changes the stable aggregate job remains present and green; change-detection failures fail closed.

Red/green and final-head evidence

  • Controlled red commit c2b6ffbe6: run 29831509892 recorded HTTP shard failure and aggregate failure. Revert a5e0391ab removed the assertion.
  • Final green head: a2a630b7c39340ec4bb1b4f1cd6a8d906a4fb3b8.
  • Three consecutive final-head worker passes: attempt 1 (88639985103, 88639985173, aggregate 88640976675); attempt 2 (88641497291, 88641497278, aggregate 88642585822); attempt 3 (88642926738, 88642926724, aggregate 88643941294). Attempts 1 and 2 were intentionally stopped only after all worker jobs and the aggregate passed; attempt 3 was left to complete all repository gates.

Required-check enforcement

The workflow now exposes one stable Durable Object Workers context and keeps it green for unrelated changes. Repository inspection found the only Main ruleset (ID 13102493) is disabled and contains no required checks. The attempted API update to activate it and require this context returned HTTP 403 (Resource not accessible by integration). A repository administrator must activate/add the context, or grant the SAM GitHub App Administration write permission; until then GitHub cannot literally block merge despite the check behavior being ready.

Verification

  • pnpm --filter @simple-agent-manager/api test:workers:inventory — PASS, 27 + 9 = 36 files, each owned once.
  • pnpm --filter @simple-agent-manager/api test:workers:durable-objects — PASS, 27 files / 449 tests / 158.58s.
  • pnpm --filter @simple-agent-manager/api test:workers:http — PASS, 9 files / 102 tests / 64.32s.
  • Worker rejection policy unit tests — PASS, 1 file / 4 tests.
  • API TypeScript check — PASS.
  • API lint — PASS with pre-existing warnings and zero errors.
  • Staging — not required: this PR changes CI/test infrastructure and focused test entrypoints only; production runtime code is unchanged.

Specialist Review Evidence

Reviewer Status Outcome
cloudflare-specialist PASS Real workerd bindings and focused-entrypoint configuration reviewed; no mocks or hidden shard failure.
test-engineer PASS 36-file ownership, vertical slices, fixture relationships, and zero-skip result reviewed.
security-auditor PASS Test-only keys, callback/CORS/WebSocket assertions, workflow permissions, and fail-closed aggregation reviewed.
constitution-validator PASS New timeouts/concurrency have environment overrides; test-only URLs/IDs are not business configuration.
task-completion-validator PASS Acceptance criteria and local evidence reconciled; GitHub red/green/repeat items remain visibly pending until recorded.
  • Preflight completed before code changes
  • external-api-change
  • cross-component-change
  • infra-change

External References

Consulted Cloudflare official documentation before finalizing the configuration: https://developers.cloudflare.com/workers/testing/vitest-integration/, https://developers.cloudflare.com/workers/testing/vitest-integration/configuration/, and https://developers.cloudflare.com/workers/testing/vitest-integration/isolation-and-concurrency/.

Codebase Impact Analysis

Changes are limited to .github/workflows/ci.yml, apps/api/vitest.workers.config.ts, apps/api/tests/workers/, the API package scripts, and the repository task record. Production apps/api/src/ runtime code is unchanged.

Documentation & Specs

The active task record documents prior-PR reconciliation, root cause, acceptance criteria, counts, durations, and the no-staging decision. The PR body carries final external run evidence.

Constitution & Risk Check

Reviewed Principle XI and CI security: runtime test timeout and worker concurrency use environment-overridable defaults; test-only domains, keys, and identifiers are isolated to Miniflare. Shards fail on crashes and unexpected unhandled errors, and aggregation fails closed.

Co-Authored-By: Claude noreply@anthropic.com

@codspeed-hq

codspeed-hq Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 6 untouched benchmarks


Comparing sam/current-main-real-durable-mdea0b (a2a630b) with main (2921cbd)

Open in CodSpeed

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

💡 Need a hand with PR review? Try Gitar by Sonar!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant