Kyutai STT 2.6B EN

Kyutai STT is a decoder-only, streaming speech-to-text model from Kyutai Labs (Paris), leveraging the multistream architecture of Moshi (arXiv:2410.00037) to jointly model audio and text streams.

• Decoder-only Transformer (~2.6B params)
• Audio is tokenized by Mimi (Moshi's neural audio codec) at 12.5 Hz, with 32 audio tokens per frame
• Text stream is shifted 2.5 s relative to audio, so the model predicts text tokens from preceding audio — yielding natural streaming
• Sampling rate: 24 kHz (via Mimi)
• Produces capitalized, punctuated transcripts with recoverable token timestamps

• Pretraining: 2.5 M hours of public audio with synthetic transcripts from whisper-timestamped
• Finetuning: 24k hours of public datasets with ground-truth labels
• Long-form finetuning: concatenated LibriSpeech (1000 h) + 22k h synthetic dialogs
• 48× H100 for pretraining, 16× H100 for finetuning

• Best-in-class WER (6.4%) among the open streaming ASR models surveyed
• Streaming with only 2.5 s delay, robust to noisy conditions and audio up to 2 hours long
• Native integration in Hugging Face Transformers (≥ 4.53) via KyutaiSpeechToText*; also MLX and Candle builds
• Based on the same technology stack as Kyutai's Moshi full-duplex voice model

Live captioning, real-time voice assistants, meeting/call transcription, accessibility, and powering downstream LLM voice pipelines (e.g. Unmute).