CrisperWhisper

Our Take

A solid 1.55B-parameter dense audio model from Nyra Health. Treat the modality benchmarks above as the leading indicator of fit — composite scoring across modalities is still maturing.

Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.

CrisperWhisper is a dense transformer model with 1.55B parameters, identical in size to Whisper large-v3. Unlike mixture-of-experts (MoE) architectures that activate only a subset of parameters per token, dense models use all parameters for every forward pass. This means VRAM consumption scales linearly with parameter count—no surprises—and inference speed is predictable across hardware.

Nyra Health made two key modifications to the base Whisper architecture:

• Tokenizer adjustment: The original Whisper tokenizer was reworked to preserve filler words and disfluencies in the output vocabulary, enabling the model to generate verbatim transcriptions rather than cleaned-up versions.
• Custom attention loss: During fine-tuning, an additional loss term was applied to the cross-attention scores used for dynamic time warping (DTW) alignment. This dramatically improves timestamp accuracy around disfluencies and pauses—a known weak point in standard Whisper.

The model’s context length is not specified by the provider, but it inherits Whisper’s typical 30-second audio chunk processing. For longer audio, you must segment the input or use a sliding window approach (supported in the official transcribe.py script and via Hugging Face transformers).

CrisperWhisper’s core capability is verbatim ASR with crisp word-level timestamps. It transcribes exactly what was said, including:

• Fillers: “um”, “uh”, “hmm”
• Stutters and false starts: “I-I-I think”
• Pauses and hesitations
• Overlapping or fragmented speech (within a single speaker segment)

It also reduces transcription hallucinations—spurious words or phrases that Whisper sometimes invents when audio is noisy or ambiguous.

Concrete use cases:

• Clinical documentation: Capture patient speech verbatim, including hesitations that may indicate cognitive or emotional state.
• Legal and compliance: Verbatim transcripts of depositions or witness statements where filler words and pauses can be relevant.
• Media post-production: Accurate alignment of captions with spoken content, especially around natural speech disfluencies.
• Conversation analysis: Research workflows that require fine-grained temporal markers for every word.
• Voice user interface debugging: Logging exactly what users said, including false starts, to improve intent recognition.

CrisperWhisper supports English and German out of the box. Other languages may work but are not benchmarked.

Because CrisperWhisper is a 1.55B dense model, it runs comfortably on consumer hardware. Here’s what you need to know to run it locally.

Minimum hardware: Any GPU with at least 4 GB VRAM (e.g., GTX 1060 6GB, RTX 3050) can run FP16 inference. For real-time or near-real-time performance, an RTX 3060 12GB or better is recommended.

Recommended hardware:

• Consumer GPU: RTX 4070 (12GB) or RTX 4090 (24GB) — Q4_K_M quantization achieves <1 second latency per 30-second audio chunk.
• Apple Silicon: M4 Max with 48GB unified memory runs FP16 easily; M2/M3 Pro with 16GB also works with quantization.
• CPU-only: Possible with GGUF Q4_K_M on a modern 8-core processor, but expect 5–10x slower than GPU.

• Tokens per second: On an RTX 4090 with Q4_K_M, CrisperWhisper processes approximately 200–300 tokens per second (roughly 5–8 seconds of audio per second of wall time). On an M4 Max, expect 150–200 tokens/sec.
• Latency per 30-second clip: ~0.5–1.5 seconds on a modern GPU.

The quickest way to run CrisperWhisper locally is via the official transcribe.py script from the GitHub repository, or using Hugging Face transformers. For optimized inference, use faster-whisper (CTranslate2) with the CrisperWhisper model weights converted to the ct2 format. Ollama does not currently support CrisperWhisper; the model is not in the Ollama library, but you can run it directly with Python.

For best performance, use torch.float16 and move the model to GPU.

CrisperWhisper competes directly with other Whisper variants at the 1.5B parameter scale. The two most relevant alternatives are:

• OpenAI Whisper large-v3 (1.55B): The base model. CrisperWhisper is a drop-in replacement that adds verbatim transcription and sharper timestamps. If you need clean, intended transcription (e.g., for subtitles where fillers are unwanted), stick with vanilla Whisper. If you need exact speech capture, CrisperWhisper is strictly better.
• Faster-Whisper (CTranslate2): An optimized inference engine for Whisper models. It supports quantization and batching, making it faster on CPU and low-end GPUs. You can run CrisperWhisper weights through faster-whisper by converting them to CTranslate2 format. The tradeoff: faster-whisper does not include the custom attention loss or tokenizer adjustments, so timestamp accuracy and verbatim output will revert to base Whisper behavior.

When to choose CrisperWhisper: You need word-level timestamps that are reliable even around hesitations and fillers, and you want to minimize hallucinations. It is the best open model for verbatim ASR at this size.

When to choose an alternative: If you only need standard non-verbatim transcription and want maximum speed on CPU, use faster-whisper with Whisper large-v3. If you need multilingual support beyond English and German, stick with vanilla Whisper large-v3 (which covers 99 languages).