diff --git a/docs/private/next-gen-model-realtime.mdx b/docs/private/next-gen-model-realtime.mdx new file mode 100644 index 00000000..4176992e --- /dev/null +++ b/docs/private/next-gen-model-realtime.mdx @@ -0,0 +1,88 @@ +--- +description: Get started with our next-generation model for Realtime transcription +--- + +# Next-generation Model – Realtime + +## Introduction +We are offering existing customers an early preview of our next-generation model, `melia-1`, for Realtime transcription. This is the Realtime companion to the [Batch preview](./next-gen-model). + +:::warning +This is an early preview for evaluation only, and is not intended for production use. Only a subset of Realtime features is currently supported (see [Capabilities](#capabilities)). Remaining features will be added over time. +::: + +This page describes how to access the preview and what to expect in terms of capabilities. + +## Getting started +You can start right away using your existing API keys, no new credentials needed. If you don't have an API key, head to the [Portal](https://portal.speechmatics.com) to get one. + +## API reference +The API schema is consistent with the [V2 Realtime WebSocket API](https://docs.speechmatics.com/api-ref/realtime-transcription-websocket) with some additions and limitations described below. + +### Supported endpoint +The preview runs on a dedicated endpoint, separate from the production Realtime service: + +| Region | Endpoint | +|-------------------|------------------------------------------| +| Preview (EU & US) | **wss://preview.rt.speechmatics.com/v2** | + +The preview is deployed in France (EU) and Oregon (USA). + +### Minimal example +Example minimal `StartRecognition` message: + +```json +{ + "message": "StartRecognition", + "transcription_config": { + // highlight-start + "language": "multi", + "model": "melia-1" + // highlight-end + } +} +``` + +### LanguageInfo message +A new `LanguageInfo` message is sent from the transcriber to the client the first time a new language code is detected in the audio. It is emitted **once per language code**, immediately before that language first appears in an `AddTranscript` (Final) or `AddPartialTranscript` (Partial) message. + +Use it to configure how you process and display the transcript — for example, word spacing and text direction for right-to-left scripts. + +```json +{ + "message": "LanguageInfo", + "language": "ar", + "word_delimiter": " ", + "writing_direction": "right-to-left", + "partial": true +} +``` + +- `language` (String): The detected language code. +- `word_delimiter` (String): The delimiter to use between words when reconstructing the transcript for this language. +- `writing_direction` (String): Either `left-to-right` or `right-to-left`. +- `partial` (Boolean): Optional. When present, this is `true`, indicating the message was triggered by a Partial transcript. + +## Capabilities + +| Feature | Current capability | Future plans | +|---------------------------------------------|-----------------------------|----------------------------------------| +| Partials | ✅ Matches current behaviour | | +| Channel diarization | ✅ Matches current behaviour | | +| Output locale | ✅ Matches current behaviour | | +| Word timings | ✅ Matches current behaviour | | +| Language labelling | ✅ Word-level | | +| Speaker diarization | ⚠️ Not yet | Prioritised according to customer need | +| Speaker identification | ⚠️ Not yet | Prioritised according to customer need | +| Language hints | ⚠️ Not yet | Prioritised according to customer need | +| `max_delay` / `max_delay_mode` | ⚠️ Not yet | Under consideration | +| Mid-session config (`SetRecognitionConfig`) | ⚠️ Not yet | Prioritised according to customer need | +| End of Utterance (EOU / FEOU) | ⚠️ Not yet | Prioritised according to customer need | +| Custom dictionary | ⚠️ Not yet | Prioritised according to customer need | +| Alphanumerics | ⚠️ Not yet | Prioritised according to customer need | +| Entities / punctuation overrides | ⚠️ Not yet | Prioritised according to customer need | +| Disfluency removal | ⚠️ Not yet | Prioritised according to customer need | +| Profanity tagging | ⚠️ Not yet | Prioritised according to customer need | +| Audio volume filtering | ⚠️ Not yet | Prioritised according to customer need | +| Translation | ⚠️ Not yet | Prioritised according to customer need | +| Audio events | ⚠️ Not yet | Prioritised according to customer need | diff --git a/docs/private/next-gen-model.mdx b/docs/private/next-gen-model.mdx index 31c9e8ac..ec0a05fe 100644 --- a/docs/private/next-gen-model.mdx +++ b/docs/private/next-gen-model.mdx @@ -6,7 +6,7 @@ description: Get started with our next-generation model ## Introduction We are excited to offer early access to our next-generation model, codenamed Omni. This new architecture is designed to provide improved multilingual capabilities, faster turnaround time and improved alphanumeric accuracy across all supported languages. -Currently available for Batch transcription, the Omni model will be made available for Realtime transcription in Q3 2026. +Currently available for Batch transcription and Realtime transcription — see [Next-generation Model – Realtime](./next-gen-model-realtime). This page will guide you through how to access the new model, and what to expect in terms of capabilities.