NVIDIA Canary-Qwen 2.5B

NVIDIA Canary-Qwen 2.5B is a hybrid Speech-Augmented Language Model (SALM) that sets a new benchmark for automatic speech recognition (ASR). Developed by NVIDIA and released under the permissive CC-BY-4.0 license, this 2.5-billion-parameter model achieves a record 5.63% word error rate (WER) on the Hugging Face Open ASR leaderboard—the lowest reported to date. It combines the Canary-1B-Flash speech encoder with a Qwen3-1.7B large language model (LLM) decoder, effectively merging transcription capabilities with downstream language understanding.

Unlike traditional ASR systems that only output raw text, Canary-Qwen 2.5B operates in two distinct modes: a pure transcription mode (ASR) and an LLM mode that retains the decoder’s reasoning, summarization, and question-answering skills. This makes it a single, locally-runable solution for workflows that require both accurate speech-to-text and post-processing of the transcript—without relying on cloud APIs. It competes with other 2-3B parameter ASR models, but its hybrid architecture and top-tier benchmark results place it in a class of its own for practitioners who need on-premise, real-time speech intelligence.

Canary-Qwen 2.5B is a dense Transformer model with 2.5 billion parameters. Its architecture is a dual-component pipeline:

• Encoder: Canary-1B-Flash, a FastConformer-based speech encoder optimized for low-latency, high-throughput inference. It processes raw audio into a sequence of hidden representations.
• Decoder: Qwen3-1.7B, an LLM decoder that takes the encoded audio features and produces the final text output. In ASR mode, the decoder is constrained to transcription; in LLM mode, it can leverage its full generative capabilities on the transcript.

The model is trained on a diverse corpus of English speech data, including LibriSpeech, Fisher, Switchboard, VoxPopuli, Common Voice, and several proprietary NVIDIA datasets (Granary, YTC, Yodas2). It supports automatic punctuation and capitalization (PnC) out of the box.

Key performance specs:

• Inference speed: 418 RTFx (real-time factor ×) – meaning it can process 418 seconds of audio per second of compute.
• WER benchmarks: 1.61% on LibriSpeech clean, 3.10% on LibriSpeech other, 9.43% on GigaSpeech, 10.19% on AMI meetings, 10.45% on Earnings-22.
• Context length: Not explicitly specified, but the FastConformer attention mechanism scales linearly, making it efficient for long audio segments.

Because it is a dense model (not Mixture-of-Experts), all 2.5B parameters are active during every forward pass. This means VRAM requirements scale directly with model size and quantization level, with no sparsity tricks to reduce memory. Practitioners should plan for predictable memory usage based on standard 2.5B-parameter dense model calculations.

Canary-Qwen 2.5B is built for English speech recognition with optional LLM-driven post-processing. Its primary capabilities:

• High-accuracy transcription across diverse domains: clean read speech, noisy meetings, financial earnings calls, and spontaneous conversations.
• Punctuation and capitalization in the output text, improving readability for downstream tasks.
• Two operational modes: ASR mode for pure transcription (fast, deterministic) and LLM mode for transcript summarization, question answering, or other generative tasks (slower but more flexible).

Concrete use cases:

• Real-time meeting transcription – Run locally on a workstation to transcribe audio streams with WER under 11% on AMI benchmark.
• Call center analytics – Transcribe recorded calls and use LLM mode to extract action items or sentiment without a separate API call.
• Voice-controlled local AI assistants – Combine with a TTS model for a fully offline voice interface.
• Academic research – Serve as a baseline for ASR experimentation with an open, reproducible model.

Note: The model is English-only and text-output only. It does not generate speech or handle other languages.

This is where Canary-Qwen 2.5B shines: it can be run on a single consumer GPU with careful quantization. Here’s what you need to know to run it locally.

These estimates include inference overhead and assume a batch size of 1. For streaming or batch processing, add 1-2 GB for buffers.

• NVIDIA RTX 4090 (24 GB): Runs FP16 with headroom for long audio clips or multiple parallel streams. Ideal for production local setups.
• NVIDIA RTX 3060 (12 GB): Can run Q4_K_M comfortably, or Q8_0 with smaller audio chunks.
• Apple M4 Max (64 GB unified memory): Works well with llama.cpp or Apple’s MLX framework; no GPU VRAM limits. Expect real-time or better performance.
• NVIDIA RTX 3050 (8 GB): Possible with Q4_0 quantization, but may struggle with long sequences. Use shorter audio segments.

Performance depends heavily on GPU and quantization. For an RTX 4090 at Q4_K_M:

• ASR mode: ~400-500 tokens/second (audio seconds processed per second of compute).
• LLM mode (summarization): ~30-60 tokens/second (text generation speed on transcripts).

On an M4 Max, expect similar speeds in ASR mode, with slightly lower LLM generation throughput due to metal backend differences.

Use [Ollama](https://ollama.ai) with the model’s GGUF conversion. As of mid-2026, community GGUF files for Canary-Qwen 2.5B are available. The command:

1ollama run nvidia-canary-qwen-2.5b

Alternatively, use NVIDIA NeMo’s inference scripts for full control over quantization and streaming.

vs. Whisper Large V3 (1.5B parameters)

• Accuracy: Canary-Qwen 2.5B outperforms Whisper Large V3 on most English benchmarks, especially noisy speech (e.g., AMI, Earnings-22). Whisper V3 has a lower WER on LibriSpeech clean but falls short on varied domains.
• VRAM: Whisper V3 (1.5B dense) requires ~3 GB at FP16 vs. ~5 GB for Canary-Qwen. However, Canary-Qwen’s RTFx is higher (418 vs. ~250 for Whisper).
• LLM integration: Canary-Qwen’s dual-mode capability gives it a clear advantage for workflows that need summarization or reasoning without a separate model.
• Tradeoff: Choose Whisper if you need multilingual support (99 languages) or have limited VRAM. Choose Canary-Qwen if English-only is fine and you want state-of-the-art accuracy + built-in LLM post-processing.

vs. Parakeet-unified-en-0.6B (NVIDIA’s smaller model)

• Accuracy: Canary-Qwen is significantly more accurate, especially on spontaneous speech. Parakeet may be sufficient for clean audio on low-resource hardware (0.6B parameters, ~1.2 GB at FP16).
• Speed: Parakeet has higher RTFx (reportedly ~800) due to smaller size, but at a cost of WER.
• Tradeoff: Use Parakeet for extremely low-latency or edge deployments (e.g., Raspberry Pi with GPU). Use Canary-Qwen for any workload where transcription quality is critical.

For anyone building a local, English-only ASR pipeline that also needs to analyze transcripts, NVIDIA Canary-Qwen 2.5B is currently the best open model at this size. Its combination of record-breaking WER, commercial-friendly license, and dual-mode architecture makes it a practical choice for developers who demand local inference without compromise.