Add test location audit and Windows-safe lane runner preflight to prevent misplaced tests and wasted Playwright reruns - PR_26146_025-test-location-audit-and-runner-preflight

Author: ToolboxAid

docs/dev/reports/test_location_audit_report.md

@@ -0,0 +1,46 @@
+# Test Location Audit Report
+
+Generated: 2026-05-26T17:59:33.363Z
+Status: PASS
+
+## Scope
+
+- Tool Playwright specs: tests/playwright/tools
+- Game Playwright specs: tests/playwright/games
+- Shared helpers: tests/helpers
+
+## Blocking Findings
+
+No blocking test location findings.
+
+## Relocated Game-Owned Specs
+
+| Previous Path | Current Path | Status | Reason |
+| --- | --- | --- | --- |
+| tests/playwright/tools/AsteroidsBackgroundAssetResolution.spec.mjs | tests/playwright/games/AsteroidsBackgroundAssetResolution.spec.mjs | PASS | Game-owned Asteroids Playwright spec is under tests/playwright/games. |
+| tests/playwright/tools/AsteroidsBeatTiming.spec.mjs | tests/playwright/games/AsteroidsBeatTiming.spec.mjs | PASS | Game-owned Asteroids Playwright spec is under tests/playwright/games. |
+| tests/playwright/tools/AsteroidsGameSceneCleanup.spec.mjs | tests/playwright/games/AsteroidsGameSceneCleanup.spec.mjs | PASS | Game-owned Asteroids Playwright spec is under tests/playwright/games. |
+| tests/playwright/tools/AsteroidsShipStateVisuals.spec.mjs | tests/playwright/games/AsteroidsShipStateVisuals.spec.mjs | PASS | Game-owned Asteroids Playwright spec is under tests/playwright/games. |
+
+## Documented Tool Tests With Game Fixtures
+
+| File | Referenced Game Fixture(s) | Reason |
+| --- | --- | --- |
+| tests/playwright/tools/AssetManagerV2.spec.mjs | Asteroids, GravityWell, Pong | Tool runtime validation uses repo/game manifests as explicit asset payload fixtures. |
+| tests/playwright/tools/CollisionInspectorV2.spec.mjs | Asteroids | Tool runtime validation uses game manifests as explicit collision payload fixtures. |
+| tests/playwright/tools/WorkspaceManagerV2.spec.mjs | AITargetDummy, Asteroids, Bouncing-ball, Breakout, GravityWell, Pacman, Pong, SolarSystem, SpaceDuel, SpaceInvaders, vector-arcade-sample | Workspace contract validation uses game manifests as explicit launch/toolState fixtures. |
+
+## Intentionally Shared Helpers
+
+| File | Reason |
+| --- | --- |
+| tests/helpers/playwrightRepoServer.mjs | Shared HTTP repo fixture used by tool, game, and integration Playwright suites. |
+| tests/helpers/playwrightStorageIsolation.mjs | Shared localStorage/sessionStorage cleanup helper used before targeted Playwright tests. |
+| tests/helpers/workspaceV2CoverageReporter.mjs | Shared V8 coverage collection for Workspace V2 and impacted browser runtime suites. |
+
+## Ownership Rules Checked
+
+- Game-specific Playwright tests must live under tests/playwright/games.
+- Tool Playwright tests may use game manifests only as explicit, documented fixtures for tool behavior.
+- Shared helper filenames must not include game-specific names.
+- Lane execution should stop before expensive Playwright runs when this audit reports blocking findings.

docs/dev/reports/testing_lane_execution_report.md

@@ -1,59 +1,58 @@
 # Testing Lane Execution Report
 
 Generated: 2026-05-26
-PR: PR_26146_024-targeted-test-lanes-and-isolation-execution
+PR: PR_26146_025-test-location-audit-and-runner-preflight
 
 ## Summary
 
 PASS: 3
 WARN: 1
 FAIL: 0
-SKIP: 1
+SKIP: 3
 
 ## Full Samples Smoke
 
 Status: SKIP
-Reason: Changed files only add targeted lane execution and Playwright/test isolation infrastructure. No sample JSON, shared sample loader, or sample framework files were changed.
+Reason: This PR changes Playwright test placement, audit/preflight scripts, and targeted lane runner behavior only. No sample JSON, shared sample loader, or sample framework files were changed.
 
-## Lane Results
+## Executed and Skipped Lanes
 
-| Lane | Status | Executed/Skipped Reason | Affected Surface | Evidence |
+| Lane / Check | Status | Executed/Skipped Reason | Affected Surface | Evidence |
 | --- | --- | --- | --- | --- |
-| workspace-contract | WARN | Executed because Workspace V2 test wiring was impacted by Playwright storage isolation. | Workspace Manager V2 contract/lifecycle behavior | `PLAYWRIGHT_BROWSERS_PATH=0 npm run test:lane:workspace-contract` invoked existing `npm run test:workspace-v2`; 64 passed, 8 out-of-scope Workspace/Input Mapping/manifest/Preview Generator expectations failed. |
-| tool-runtime | PASS | Executed because tool-runtime lane wiring and storage isolation were added for focused tool suites. | Asset Manager V2, Preview Generator V2, Collision Inspector V2 runtime behavior | `PLAYWRIGHT_BROWSERS_PATH=0 npm run test:lane:tool-runtime`; Asset Manager V2 focused launch/runtime subset 5 passed, Preview Generator V2 7 passed, Collision Inspector V2 4 passed. |
-| integration | PASS | Executed because integration lane wiring was added for explicit cross-surface handoff validation. | Game index preview manifest handoff | `PLAYWRIGHT_BROWSERS_PATH=0 npm run test:lane:integration`; focused Pong handoff subset 3 passed. |
-| engine-src | PASS | Executed because engine/src lane wiring and node storage isolation were added. | src/ engine and shared runtime capability behavior | `npm run test:lane:engine-src`; 11/11 targeted node test files passed. |
-| samples | SKIP | Skipped because samples are on-request or affected-sample only and this PR did not impact samples. | On-request samples lane | Not run. Full samples smoke also skipped. |
+| test-location-preflight | PASS | Executed first because this PR adds the audit/preflight gate. | Playwright file ownership and lane placement | `npm run test:lanes:preflight` passed and wrote `docs/dev/reports/test_location_audit_report.md`. |
+| tool-runtime | PASS | Executed because tool test routing changed and the runner preflight/Node CLI path needed validation. | Asset Manager V2, Preview Generator V2, Collision Inspector V2 runtime tests | `PLAYWRIGHT_BROWSERS_PATH=0 npm run test:lane:tool-runtime`; preflight passed, Asset Manager V2 focused subset 5 passed, Preview Generator V2 7 passed, Collision Inspector V2 4 passed. |
+| integration | PASS | Executed because the runner preflight also guards game/integration Playwright placement. | Game index preview manifest handoff | `PLAYWRIGHT_BROWSERS_PATH=0 npm run test:lane:integration`; preflight passed, focused Pong handoff subset 3 passed. |
+| moved-game-specs | WARN | Executed once to validate relocated Asteroids game specs without changing game runtime behavior. | Asteroids game-specific Playwright tests moved from tools to games | 5 passed, 1 existing Asteroids ship-state expectation failed; exact failing test listed below. |
+| workspace-contract | SKIP | Skipped because `npm run test:workspace-v2` behavior was preserved and Workspace V2 runner wiring was not changed for this PR. | Workspace Manager V2 contract/lifecycle behavior | Not run. |
+| engine-src | SKIP | Skipped because no `src/`, engine, or node runtime test lane files were changed. | src/ engine/shared runtime behavior | Not run. |
+| samples | SKIP | Skipped because samples are on-request or affected-sample only and this PR did not impact samples. | On-request samples lane | Not run. |
 
-## Workspace V2 WARN Details
+## Moved Game Spec WARN Detail
 
-`npm run test:workspace-v2` was run once through the workspace-contract lane and exited non-zero. The exact failing tests were:
+Focused command:
 
-- `Workspace Manager V2 bootstrap > sizes Input Mapping V2 columns and live-highlights mapped non-keyboard inputs`
-- `Workspace Manager V2 bootstrap > resolves game manifest schema refs from the game schema during repo discovery`
-- `Workspace Manager V2 bootstrap > enables object vector and collision tools only from manifest geometry without fallback defaults`
-- `Workspace Manager V2 bootstrap > uses header lifecycle controls and launches tools from fixed Workspace Manager V2 tiles`
-- `Workspace Manager V2 bootstrap > keeps Preview Generator V2 repo writer after Asset Manager V2 deletes the preview asset entry`
-- `Workspace Manager V2 bootstrap > fails Preview Generator V2 without OK WRITE when live handle read-back verification fails`
-- `Workspace Manager V2 bootstrap > loads Gravity Well and Pong manifests as current Workspace Manager V2 manifests`
-- `Workspace Manager V2 bootstrap > owns temporary UAT manifest seeding and launches Asset Manager V2 through session context`
+`PLAYWRIGHT_BROWSERS_PATH=0 node ./node_modules/@playwright/test/cli.js test tests/playwright/games/AsteroidsBackgroundAssetResolution.spec.mjs tests/playwright/games/AsteroidsBeatTiming.spec.mjs tests/playwright/games/AsteroidsGameSceneCleanup.spec.mjs tests/playwright/games/AsteroidsShipStateVisuals.spec.mjs --project=playwright --workers=1 --reporter=list`
 
-These were reported as lane WARN because they are broad Workspace/Input Mapping/manifest/Preview Generator assertions outside this testing-lane infrastructure PR. The full suite was not rerun.
+Result: 5 passed, 1 failed.
+
+Failing test:
+
+- `AsteroidsShipStateVisuals.spec.mjs > validates Asteroids ship visual states from manifest runtime rendering`
+
+Observed failure: expected ship visual states to include `destroyed`, but the runtime returned `["idle", "move"]`. This was recorded as WARN because the PR only moves test ownership and runner preflight behavior; it does not alter Asteroids runtime or sample JSON.
 
 ## Static Validation
 
+- PASS `node --check scripts/audit-playwright-test-locations.mjs`
 - PASS `node --check scripts/run-targeted-test-lanes.mjs`
-- PASS `node --check scripts/run-node-test-files.mjs`
-- PASS `node --check tests/helpers/playwrightStorageIsolation.mjs`
-- PASS `node --check tests/playwright/tools/WorkspaceManagerV2.spec.mjs`
-- PASS `node --check tests/playwright/tools/AssetManagerV2.spec.mjs`
-- PASS `node --check tests/playwright/tools/PreviewGeneratorV2Baseline.spec.mjs`
-- PASS `node --check tests/playwright/tools/CollisionInspectorV2.spec.mjs`
-- PASS `node --check tests/playwright/games/GameIndexPreviewManifestResolution.spec.mjs`
-
-## Lane Routing Notes
-
-- Targeted execution is now available through `npm run test:lanes` and per-lane scripts.
-- `npm run test:workspace-v2` remains unchanged and callable directly.
-- The integration lane intentionally validates an explicit Pong handoff subset; broad all-game thumbnail coverage is not part of the default targeted lane.
-- The samples lane is available on request through `npm run test:lane:samples`, but the full samples smoke remains manual/on-request unless sample scope is active.
+- PASS `node --check tests/playwright/games/AsteroidsBackgroundAssetResolution.spec.mjs`
+- PASS `node --check tests/playwright/games/AsteroidsBeatTiming.spec.mjs`
+- PASS `node --check tests/playwright/games/AsteroidsGameSceneCleanup.spec.mjs`
+- PASS `node --check tests/playwright/games/AsteroidsShipStateVisuals.spec.mjs`
+
+## Runner Safeguards
+
+- Playwright lane commands run through `node node_modules/@playwright/test/cli.js`, so grep patterns such as `launch guard|temporary UAT context|rejects non-Workspace` remain literal on Windows.
+- Playwright-bearing lanes run the location audit before expensive execution.
+- If the audit reports blocking placement findings, the lane runner writes the lane report and exits before running long Playwright suites.
+- No full lanes were rerun after known out-of-scope failures.

package.json

@@ -3,7 +3,9 @@
   "scripts": {
     "pretest": "node tools/dev/checkSharedExtractionGuard.mjs",
     "test": "node ./scripts/run-node-tests.mjs",
+    "test:audit:locations": "node ./scripts/audit-playwright-test-locations.mjs",
     "test:lanes": "node ./scripts/run-targeted-test-lanes.mjs",
+    "test:lanes:preflight": "node ./scripts/audit-playwright-test-locations.mjs",
     "test:lane:workspace-contract": "node ./scripts/run-targeted-test-lanes.mjs --lane workspace-contract",
     "test:lane:tool-runtime": "node ./scripts/run-targeted-test-lanes.mjs --lane tool-runtime",
     "test:lane:integration": "node ./scripts/run-targeted-test-lanes.mjs --lane integration",