Miso One

Miso One uses a Sesame CSM (Conversational Speech Model) architecture with two transformer components:

• Backbone: A Llama-8B-style transformer that processes text and audio-frame embeddings. This is the large language model that handles semantic understanding and prosodic planning.
• Audio Decoder: A smaller Llama-300M autoregressive decoder that predicts higher-order audio codebooks within each frame.

The model generates Mimi audio codes—32 codebooks per frame, with codebook 0 predicted from the backbone hidden state and codebooks 1–31 predicted autoregressively by the audio decoder. The text vocabulary is 128,256 tokens, the audio vocabulary is 2,051 tokens, and the maximum sequence length is 2,048.

This is a dense architecture, not Mixture of Experts. All 8.2B parameters are active during inference. At FP16, that means roughly 16 GB of VRAM just to load the weights, plus additional memory for activations and KV cache. The 2,048 token context window is relatively short by LLM standards, but for TTS it’s sufficient for multi-turn conversation and voice continuation tasks.

The Mimi audio tokenizer operates at 48 kHz output, which is higher than the standard 24 kHz or 16 kHz found in many open TTS models. This contributes to audio quality but also increases computational cost per second of generated speech.

Miso One is designed for three primary tasks:

Expressive conversational speech. The model can vary emotion, pacing, and delivery based on text content. This is the core differentiator—most open TTS models produce flat, scripted-sounding output. Miso One can make a character sound hesitant, excited, or commanding without manual parameter tweaking.

One-shot voice cloning. Given a short reference audio clip (roughly 10 seconds), the model can continue speaking in that voice. This works through audio context conditioning—the model processes the reference clip and generates continuation audio that matches the speaker’s timbre and style.

Low-latency voice agent research. Miso Labs’ 110 ms latency claim is for their hosted API, not local inference, but the architecture is designed for streaming use cases. The model generates audio frame-by-frame, which makes it suitable for real-time voice agent pipelines if you have the hardware to keep up.

Current limitations: English only, no multilingual support. The 8.2B parameter count means this is not a lightweight model—it’s built for quality, not portability. The Modified MIT License permits commercial use but includes specific terms around voice cloning consent and watermarking.

Miso One is not a model you run on a laptop. Here’s what you need to know for local deployment.

At FP16, you need a 24 GB GPU to have headroom for activations and batch processing. A 16 GB GPU (RTX 4060 Ti, RTX 4080) can load the model at FP16 but will be tight on memory for anything beyond single-utterance generation.

• RTX 4090 (24 GB): Runs FP16 comfortably. Expect 2–4 seconds of audio generated per second of wall time for single utterances.
• RTX 3090 (24 GB): Same capacity as 4090, slightly slower inference.
• RTX 4080 (16 GB): Can run Q8_0 or Q4_K_M. FP16 is possible but leaves little room for other processes.
• M4 Max / M4 Ultra (64–128 GB unified): Runs FP16 with plenty of headroom. Unified memory eliminates VRAM constraints, but raw throughput is lower than a 4090.
• Dual GPU setups: Possible with model parallelism, but not officially supported in the current inference code.

The official repository at MisoLabsAI/MisoTTS provides the inference code and setup instructions. The quickest path:

This downloads the model weights from Hugging Face and generates a sample conversation. For production use, you’ll want to integrate the generator.py module into your own pipeline.

Local inference latency is significantly higher than Miso Labs’ hosted API. Expect 500 ms to 2 seconds for the first audio frame on consumer hardware, depending on GPU and quantization. Streaming generation improves perceived latency but requires careful pipeline design.

GGUF and EXL2 quantizations are not yet available as of initial release, but community conversions are likely to appear quickly given the model’s popularity.

vs. XTTSv2 (Coqui): XTTSv2 is smaller (~1.6B parameters) and runs on far less hardware—8 GB VRAM is sufficient. It supports multilingual TTS and voice cloning. Miso One wins on emotional expressiveness and audio quality (48 kHz vs 24 kHz). XTTSv2 wins on accessibility, speed, and language coverage. Choose Miso One if you need natural conversational delivery and have the GPU budget. Choose XTTSv2 if you need multilingual support or are running on constrained hardware.

vs. Fish Speech 1.5: Fish Speech is also smaller (~500M–1B parameters) and supports multiple languages with voice cloning. It’s faster and more hardware-efficient. Miso One produces more emotionally varied output and higher sample rates. Fish Speech is the practical choice for production pipelines on consumer GPUs. Miso One is the quality choice for voice agents where natural delivery matters more than throughput.

vs. ElevenLabs (proprietary): ElevenLabs offers superior quality and latency but is a paid API. Miso One is open weights and can run locally. If you need the absolute best quality and have budget, ElevenLabs wins. If you need data sovereignty, no per-character costs, or the ability to fine-tune, Miso One is the better option.