Benchmarks · 2023

WER: Word Error Rate (Open ASR Leaderboard)

Name: WER: Word Error Rate (Open ASR Leaderboard)
Creator: Hugging Face Open ASR Leaderboard
Published: 2023
Keywords: WER, AI benchmark, audio model evaluation, Hugging Face Open ASR Leaderboard

The standard accuracy metric for speech-to-text: lower is better.

Open Dataset

Models Tested

Top Score

1.0%

Published

2023

Source

Hugging Face Open ASR Leaderboard

Tools for AI Builders—All the hardware and tools you need to run private AI at your office or at home.Shop Now

How It Works

WER is the percent of words in a transcript that differ from the human reference: substitutions, deletions, and insertions all count as one error. It is the canonical accuracy metric for speech-to-text. Lower is better: a WER of 5% means 5 out of every 100 words are wrong.

For each audio clip, the ASR model produces a transcript, which is compared word-for-word to the reference. Errors are summed and divided by the reference length. The Open ASR Leaderboard reports WER averaged across a basket of public datasets to control for dataset-specific quirks.

Dataset size

Aggregates Word Error Rate across multiple public ASR datasets including LibriSpeech, TED-LIUM, and Common Voice.

Mean score

5.8%

Median score

6.4%

Open / Closed

34 / 2

Benchmark data sourced from: Hugging Face, Arena.ai, Artificial Analysis.

Top Scorers

#	Model	Lab	Source	Score
01	Fish Speech v1.4	Fish Audio	Open	1.0%
02	Fish Speech v1.5	Fish Audio	Open	1.0%
03	Apriel Voice 27B Preview	ServiceNow	Open	2.2%
04	MAI-Transcribe-1.5	Microsoft AI	Closed	2.4%
05	MAI-Transcribe-2-Streaming	Microsoft AI	Closed	2.5%
06	Nemotron 3.5 Transcribe	Nvidia	Open	4.7%
07	Granite Speech 5.0 2B	IBM	Open	5.1%
08	Cohere Transcribe (03-2026)	Cohere	Open	5.4%
AdTools for AI BuildersAll the hardware and tools you need to run private AI at your office or at home.Shop Now
09	Granite-Speech-5.0-470M-TurboCTC	IBM	Open	5.5%
10	Granite 4.0 1B Speech	IBM	Open	5.5%
11	NVIDIA Canary-Qwen 2.5B	NVIDIA	Open	5.6%
12	Granite Speech 3.3 8B	IBM	Open	5.7%
13	Qwen3-ASR-1.7B	Alibaba	Open	5.8%
14	Granite Speech 3.3 2B	IBM	Open	6.0%
15	Phi-4-multimodal-instruct	Microsoft	Open	6.0%

Score Distribution

Score vs Parameter Count

Average Score by Lab

Fish Audio
1.0%n = 2
Microsoft AI
2.5%n = 2
IBM
5.6%n = 5
NVIDIA
6.6%n = 10

Most Correlated Benchmarks

Not enough scored models yet.

What It Captures Well

Industry-standard metric with decades of comparability.
Cheap and fully automated.
Sensitive to the kinds of errors that matter for downstream search and indexing.

Where It Falls Short

Treats every word equally, even when some words matter much more than others.
Does not account for punctuation, capitalization, or speaker labelling.
Does not measure latency or streaming quality.

Frequently Asked Questions

What is a good WER?

On clean read speech, top systems get under 2% WER. On real conversational audio with accents and background noise, even the best systems are between 5% and 12%. Anything above 15% in production is going to feel unreliable to users.

Is lower always better?

Yes, but watch for over-optimization on a single dataset. A model that hits very low WER on LibriSpeech but stays high on Common Voice may be brittle to accents. The Open ASR Leaderboard cross-averages to expose this.