Skip to content

[codex] Modernize gaze runtime path#28

Draft
meekmachine wants to merge 11 commits into
mainfrom
codex/modern-gaze-agency-clean
Draft

[codex] Modernize gaze runtime path#28
meekmachine wants to merge 11 commits into
mainfrom
codex/modern-gaze-agency-clean

Conversation

@meekmachine

@meekmachine meekmachine commented May 14, 2026

Copy link
Copy Markdown
Owner

What This Does

This PR makes GazeService the canonical eye/head gaze path in Latticework. Gaze state and events now flow through Most streams with Effect-managed refs for mutable service state, and eye/head tracking routes through a GazeRuntime adapter instead of owning direct engine side effects everywhere.

When an animation agency is available, gaze is projected into Loom3 as persistent scheduler-backed snippets for each eye/head axis. Those snippets are retargeted with time, reverse, playback-rate, and intensity controls instead of creating replacement clips for every gaze sample. Direct transitionContinuum output remains only as the fallback for runtimes without an animation agency.

Why This Matters

The benefit is smoother, more composable gaze behavior. Eye/head tracking can blend with mouth, blink, prosody, and other Loom3 snippets instead of fighting them through direct AU/head writes. It also reduces clip churn from high-frequency mouse/webcam/manual samples, keeps the smoothness controls connected to actual scheduled movement, and gives LoomLarge #590 one stable runtime target for profile-backed eye/head settings.

What This Is Not

This is not the LoomLarge profile persistence PR. The UI/profile work lives in meekmachine/LoomLarge#590, and the cross-repo validation harness lives in meekmachine/LoomLarge#708. This PR is the Latticework runtime side that those LoomLarge changes need to test against.

Main Changes

  • Route eye/head gaze state, config, lifecycle, and cleanup through GazeService.
  • Use Most streams for observable gaze state/events and Effect refs for config, lifecycle, and last-applied-target state.
  • Prefer the animation-agency scheduler-backed runtime when available, while keeping direct engine output as the compatibility fallback.
  • Drive eye/head axes as persistent normalized control snippets and retarget those snippets instead of rescheduling one-off target clips per sample.
  • Preserve production target filtering defaults (smoothFactor: 0.25, minDelta: 0.01) so pointer/webcam noise is damped before it reaches Loom3.
  • Wire agencyTransitionDuration into gaze planning so the UI smoothness control affects scheduled eye/head movement speed.
  • Preserve inherited first keyframes into Loom3 buildClip() so snippets anchor from the live morph/bone pose.
  • Clear disabled channels immediately, including neutral head snippets when head tracking is disabled without disturbing eye tracking.
  • Fix return-to-center scheduling so quiet mouse/webcam/manual input can return to neutral through the scheduler-backed runtime.
  • Force a camera-relative apply when eye/head channels are enabled so newly enabled head tracking catches up even if the shared target did not move.
  • Keep deprecated gazeMode and useAnimationAgency fields accepted for compatibility.
  • Add focused tests for planning/runtime behavior, scheduler-backed gaze output, inherited-frame handoff, channel disable cleanup, return-to-center timing, adjustable scheduler durations, and camera-relative channel enable catch-up.

Closes #15.
Refs #12.
Supersedes #27.

Runtime Dependency Note

This still requires a published Loom3 release that contains the inherited first-frame work from meekmachine/Loom3#142. That Loom3 PR is merged, but npm currently still reports @lovelace_lol/loom3@1.0.47, so this PR should remain draft until Loom3 is published again and the dependency range/lockfile can resolve that runtime.

Verification

Latest refresh after merging current main into this branch:

  • npm install
  • npm run typecheck
  • npm run build
  • npm test
  • git diff --check

Earlier focused validation:

  • npm test -- --run src/eyeHeadTracking/__tests__/eyeHeadTrackingService.test.ts src/eyeHeadTracking/__tests__/eyeHeadTrackingScheduler.test.ts src/gaze/__tests__/service.test.ts src/animation/__tests__/animationService.test.ts
  • npx tsc --noEmit --pretty false

@meekmachine

Copy link
Copy Markdown
Owner Author

Conflict/status update: this PR is currently mergeable/clean against main at 15710d8475c3021a68111021532018527eb43700.

Benefit: this moves eye/head tracking onto the scheduler-backed GazeService path instead of repeatedly writing direct AU/head values or scheduling replacement clips for every gaze sample. That should reduce runtime churn, let gaze blend with other Loom3 animation snippets, make disable/reset/return-to-neutral behavior deterministic per channel, and give LoomLarge's profile-backed eye/head settings one runtime path to project into.

The matching LoomLarge integration PR is meekmachine/LoomLarge#708. It validates LoomLarge #590 with this Latticework PR linked through CI, without committing temporary package-manifest git pins.

@meekmachine

meekmachine commented Jun 16, 2026

Copy link
Copy Markdown
Owner Author

I updated the PR body to make the benefit clearer. The short version: this moves gaze into a single GazeService path with Effect-managed service state and scheduler-backed Loom3 snippet projection, so eye/head tracking can blend with other animation snippets instead of fighting them through direct AU/head writes. It also gives LoomLarge #590 one stable runtime target for profile-backed eye/head settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Modernize gaze agency around Most.js streams and Effect state

1 participant