Goal
Finish Vocal as the active ClojureScript sentence-level lip-sync runtime for text and provider-timed speech timelines.
Current state
Polyester already has src-cljs/latticework/vocal.cljs, in-process and worker client APIs, text-to-viseme planning, sentence timelines, word-boundary drift detection, snippet building, jaw curves, coarticulation constraints, and cleanup planning. The TypeScript VocalService is currently the active lip-sync runtime used by TTS and LiveKit paths.
This mirrors the Latticework research issue: meekmachine/Latticework#34 and the stabilization discussion: meekmachine/LoomLarge#521.
Clojure-way direction
- Keep sentence-level timeline planning in pure CLJS data: viseme events, word timings, snippet curves, active sentence state, drift decisions.
- Keep all animation side effects at the host edge: schedule/remove/seek/pause/resume.
- Provider timelines should be pre-timed; snippet playback rate should remain neutral to avoid double-applying speech rate.
- Word boundaries should verify and correct drift by emitting seek requests instead of rebuilding clips on every word.
Acceptance criteria
createVocalAgency and worker client can replace the TypeScript VocalService for text and provider timelines behind the existing JS API.
- Worker returns deterministic serializable snippets for text and provider viseme timelines.
- Word-boundary drift still emits seek requests when threshold is exceeded.
- Active sentence cleanup cannot remove unrelated snippets and does not leak timers.
- Add TS-vs-CLJS parity tests for representative phrases, jaw curves, coarticulation caps, sentence cleanup, pause/resume, and drift correction.
- Document that Vocal is the visible mouth runtime; legacy LipSync remains compatibility/research until folded into this path.
Goal
Finish Vocal as the active ClojureScript sentence-level lip-sync runtime for text and provider-timed speech timelines.
Current state
Polyester already has
src-cljs/latticework/vocal.cljs, in-process and worker client APIs, text-to-viseme planning, sentence timelines, word-boundary drift detection, snippet building, jaw curves, coarticulation constraints, and cleanup planning. The TypeScriptVocalServiceis currently the active lip-sync runtime used by TTS and LiveKit paths.This mirrors the Latticework research issue: meekmachine/Latticework#34 and the stabilization discussion: meekmachine/LoomLarge#521.
Clojure-way direction
Acceptance criteria
createVocalAgencyand worker client can replace the TypeScriptVocalServicefor text and provider timelines behind the existing JS API.