Goal
Create the Polyester TTS agency as a hybrid ClojureScript planner plus host adapter: CLJS owns speech/timeline planning, while browser/backend/audio effects stay on the main thread or backend.
Current state
Polyester root still inherits the TypeScript TTS service from Latticework. There is no src-cljs/latticework/tts.cljs agency yet. The TypeScript src/tts/ttsService.ts currently owns Web Speech, SAPI/Azure backend calls, AudioContext playback, display-media playback reference, timeouts, and a VocalService instance.
This mirrors the Latticework research issue: meekmachine/Latticework#37.
Clojure-way direction
- CLJS owns text token parsing, emoji/emotion event extraction, provider response normalization, timeline planning, utterance state, cancellation tokens, and commands to Vocal/Prosodic/EmotionExpression.
- Main thread owns Web Speech,
SpeechSynthesisUtterance, AudioContext, display-media capture, HTMLAudioElement, permission prompts, fetch calls, and playback start/end events.
- Backend owns Azure credentials and synthesis for the buffered Azure path. Polyester workers should consume provider timing data; they should not own secrets.
- Preserve current JS imports (
createTTSService, TTSService) through an adapter during migration.
Azure/audio contract to support
- Buffered Azure: host calls LoomLarge
POST /api/azure-tts/synthesize, receives audio_base64, visemes, word_boundaries, duration, decodes audio, starts playback, then sends a clock-anchored provider timeline into CLJS/Vocal.
- LiveKit drawer TTS: host calls
POST /api/tts/synthesize, receives audio_base64 WAV and estimated word boundaries, plays audio, then sends text-predicted timeline plus word-boundary ticks.
- Web Speech: host listens for
onboundary and start/end events, while CLJS plans fallback text visemes and prosodic/emotion events.
Acceptance criteria
- Existing TTS public API remains source-compatible.
- No CLJS worker code references Web Speech,
navigator.mediaDevices, AudioContext, DOM, backend fetch APIs, LiveKit objects, or Loom3.
- TTS emits data commands for Vocal, Prosodic, and EmotionExpression instead of directly scheduling animation snippets.
- Stop/interruption reliably cancels pending timelines and active speech state.
- Current TTS tests continue to pass through the adapter.
- Add CLJS tests for token parsing, emoji timing, provider response normalization, utterance cancellation, and speech lifecycle state.
Goal
Create the Polyester TTS agency as a hybrid ClojureScript planner plus host adapter: CLJS owns speech/timeline planning, while browser/backend/audio effects stay on the main thread or backend.
Current state
Polyester root still inherits the TypeScript TTS service from Latticework. There is no
src-cljs/latticework/tts.cljsagency yet. The TypeScriptsrc/tts/ttsService.tscurrently owns Web Speech, SAPI/Azure backend calls, AudioContext playback, display-media playback reference, timeouts, and a VocalService instance.This mirrors the Latticework research issue: meekmachine/Latticework#37.
Clojure-way direction
SpeechSynthesisUtterance,AudioContext, display-media capture,HTMLAudioElement, permission prompts, fetch calls, and playback start/end events.createTTSService,TTSService) through an adapter during migration.Azure/audio contract to support
POST /api/azure-tts/synthesize, receivesaudio_base64,visemes,word_boundaries,duration, decodes audio, starts playback, then sends a clock-anchored provider timeline into CLJS/Vocal.POST /api/tts/synthesize, receivesaudio_base64WAV and estimated word boundaries, plays audio, then sends text-predicted timeline plus word-boundary ticks.onboundaryand start/end events, while CLJS plans fallback text visemes and prosodic/emotion events.Acceptance criteria
navigator.mediaDevices,AudioContext, DOM, backend fetch APIs, LiveKit objects, or Loom3.