Document the test setup, and the mise-in-sandbox investigation - #463
Merged
Conversation
The vitest environment split had no home in the docs tree. `jsdom` appeared once in all of docs/ (incidentally, in offline-cache.md) and `environmentMatchGlobs` appeared nowhere - the only record was an inline comment in vite.config.ts plus a commit message. Same for the other test infrastructure: the Deno island's two-task split, the postcss and markdownlint guardrail tests, integration-test gating on VENICE_INFERENCE_KEY, the Playwright specs, and the __test namespace convention were scattered or unwritten. The new doc collects all of it, following the feature-doc shape (Files / Contracts / Interactions / Gotchas). CLAUDE.md gets a pointer rather than a copy: it is read at session start but attention to it decays over a long session, while the dev docs are what a session consults when it is unsure. The one thing CLAUDE.md states outright is the jsdom trap, because it is actionable at the moment of writing a test rather than at the moment of reading a doc. The existing inline comment in vite.config.ts is reactive - it tells you to add your file to the list once you see `document is not defined`. That is the right instruction for a fresh branch and useless for one that forked before the split, where the old jsdom default makes the test pass locally, the rebase stays clean, the files stay disjoint, and only CI fails. Two stale figures fixed while in the neighbourhood: the manual pnpm fallback omitted knip, which now gates, and the Rollup chunk warning was quoted at 500 kB where vite.config.ts sets chunkSizeWarningLimit to 750.
The gotcha as written pointed at "no versions found matching date filter" as though a version cutoff were configured somewhere. It is not, and `mise settings` has no such setting - the next session to read that would go hunting for a knob that does not exist. The real cause is authorization. aqua reads its version lists from the GitHub releases API, and a session whose GitHub access is scoped to this repository alone gets denied for charmbracelet/gum and cli/cli; mise renders the resulting empty list with that misleading phrasing. Also records the part that makes the blast radius wider than the two tools involved: mise resolves the entire [tools] set before running any task, so gum and gh block the gate even though no gate task uses either, and both already degrade gracefully when missing. Removing one would not unblock anything while the other remains. And notes the limit of the raw pnpm fallback - it skips the Deno island, so it does not fully cover a change under supabase/functions/.
Cloud sessions cannot run the gate through mise, and the error names the wrong cause: "no versions found matching date filter" when no date filter exists anywhere. Two further plausible readings are also wrong - that the aqua backend is broken in the sandbox, and that the session cannot reach GitHub. Written down so the next session does not re-walk any of the three. The confirmed mechanism: mise resolves the whole [tools] set before dispatching any task, two entries float on `latest`, and `latest` forces a GitHub releases-list call that the sandbox proxy refuses with 403 for repositories outside the session's scope. The pinned aqua:supabase/cli is the control - same backend, same host, exact version, no list call, installs fine. Which also proves release-asset downloads are not blocked, only the api.github.com version query. Also records a probe that lies: checking resolution by the exit code of `mise ls-remote` reports success on the failing tools, because it exits 0 with an empty list. Deliberately stops at research. The open questions - whether pinning would unblock it, whether mise can scope tools per task, and whether the denial is intentional egress policy - are listed unanswered, and no remediation is proposed. The ccr proxy README's guidance on 403s is to report a blocked host rather than route around it, which may make "make aqua work" the wrong frame entirely.
Rebasing onto the wiki-agent context-bounding work added Deno tests over venice/agents/_accumulator.ts, which made the new testing doc wrong on its first day: it described functions-test as covering "the pure logic in _shared", inherited from the mise task's own description. The suite is 48 files spanning _shared, the agent modules, and the GC entrypoints. Says so, and notes that the task description undersells itself so the next reader does not trust it over the directory listing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SYNOPSIS
Adds
docs/dev/testing.md, a CLAUDE.md pointer to it, and an open investigation into why cloud sessions cannot runmise run check. Docs only - no code changes.PURPOSE
The vitest environment split had no home in the docs tree.
jsdomappeared once in all ofdocs/(incidentally, inoffline-cache.md) andenvironmentMatchGlobsappeared nowhere - the only record was an inline comment plus a commit message. The same held for the rest of the test infrastructure: the Deno island's two-task split, the postcss and markdownlint guardrails, integration gating, and the__testconvention were scattered or unwritten.This surfaced concretely: a branch that forked before the env split landed added a component test, passed locally, rebased clean, and would have failed only in CI.
DESCRIPTION
How it was. Test infra knowledge lived in
vite.config.tscomments,.mise.tomltask descriptions, and CLAUDE.md's "Running the checks locally".What this adds.
docs/dev/testing.md- the three test islands, the env split,tests/setup.tsand its three shims, the two guardrail tests, integration gating, the Deno split, Playwright,__testexports. Feature-doc shape w/ Contracts / Interactions / Gotchas; linked from the dev index and reciprocated frombuild-deploy.md.docs/dev/in-progress/mise-in-cloud-sandbox.md- research on the mise failure, marked open, no plan by request.The mise finding, since the error lies.
mise run checkdies withno versions found matching date filter. There is no date filter -mise settingshas no such key. mise resolves the entire[tools]set before dispatching any task; two entries float onlatest;latestforces aGET api.github.com/repos/<owner>/<repo>/releasesthat the sandbox proxy 403s for repos outside the session's scope.aqua:supabase/cliis the control - pinned, no list call, installs fine - which also proves release-asset downloads are not blocked. Two dead ends are recorded so nobody re-walks them, along with a probe that gives a false pass (mise ls-remoteexits 0 on an empty list).Notes:
knip(now gates); the Rollup chunk warning was quoted at 500 kB wherevite.config.tssetschunkSizeWarningLimit: 750; andfunctions-testwas described as covering_sharedwhen the suite is 48 files spanning agents and GC entrypoints too.in-progress/placement is deliberate - the work is genuinely open, and the doc leads withStatus: open. It is not a shipped-milestone doc left behind.Gate green: 2147 vitest, svelte-check, lint, build, knip.
The Deno island was not run -
denois only reachable via mise, which is what this PR documents as broken here. No Deno code is touched, and the base commit's Deno tests were CI-verified on their own merge.Generated by Claude Code