Track Polyester Clojure-way agency transition

[meekmachine]

Goal

Track the agency-by-agency work needed for Polyester to become the ClojureScript-first character runtime package, while LoomLarge can continue A/B testing Polyester against Latticework.

This is the Polyester-side successor to the Latticework umbrella: meekmachine/Latticework#46.

Agency issues

• Animation scheduling gateway: [agency] Finish CLJS animation agency as the scheduling gateway #4
• Blink: [agency] Harden CLJS blink agency and worker protocol parity #5
• Gaze: [agency] Harden CLJS gaze agency around worker-safe target planning #6
• Eye/head tracking: [agency] Harden CLJS eye/head tracking around scheduled snippets #7
• Hair: [agency] Harden CLJS hair agency as a worker-safe state planner #8
• Prosodic: [agency] Harden CLJS prosodic agency for speech gesture planning #9
• Vocal sentence-level lip-sync runtime: [agency] Harden CLJS vocal agency as the active lip-sync timeline runtime #10
• LipSync / provider viseme timing: [agency] Rework CLJS lipsync/viseme timing around provider timelines #11
• TTS: [agency] Create hybrid CLJS TTS planner with host-owned audio effects #12
• Transcription: [agency] Create hybrid CLJS transcription analyzer #13
• Conversation orchestration: [agency] Create CLJS conversation orchestrator over typed agency events #14
• EmotionExpression: Create emotion expression agency for emoji-triggered speech playback #3

Current architecture read

Polyester already contains CLJS planner/runtime namespaces for animation, blink, gaze, eye/head tracking, hair, prosodic, vocal, and lipsync. Those should be hardened and tested rather than restarted.

TTS, transcription, and conversation still need first-class CLJS agency namespaces and hybrid host adapters because they depend on browser APIs, backend calls, audio playback, mic/transcription APIs, and LiveKit/browser objects.

The “Clojure way” direction for Polyester should be:

• pure/serializable planner state in CLJS
• reducer/planner commands and outputs as data
• worker-safe agency boundaries
• main-thread adapters for browser APIs, Loom3/Three/WebGL, audio, LiveKit, and backend effects
• JS compatibility surfaces while LoomLarge and existing callers migrate

Speech/lipsync note

The most important quality risk is speech timing. Buffered Azure TTS returns backend-generated audio_base64 plus Azure visemes/word boundaries. LiveKit conversation streams audio over a LiveKit audio track and sends viseme/word timing over text streams. The Polyester speech agencies should normalize those into one clock-anchored utterance/timeline contract and route visible mouth playback through Vocal sentence timelines.

Acceptance criteria

• Every agency has a Polyester issue with current state, Clojure-way direction, and acceptance criteria.
• Child issues link back to the relevant Latticework research issues and LoomLarge discussions where applicable.
• The agency list is kept current as issues are closed, split, or superseded.
• The first implementation milestone can be selected from this list without re-researching the architecture.