Moonshine Streaming Medium

Moonshine Streaming Medium is a 245M-parameter streaming English automatic speech recognition (ASR) model from Useful Sensors, designed to bring low-latency, on-device transcription to edge hardware. The model achieves a word error rate of 6.65% on the OpenASR Leaderboard — outperforming Whisper Large v3 despite being roughly 6× smaller. It’s purpose-built for real-time voice applications where latency, privacy, and local execution matter more than cloud connectivity.

The model is part of the broader Moonshine Voice framework, an open-source toolkit (MIT license) from the team behind TensorFlow’s original founders. Moonshine Streaming Medium targets developers building voice agents, live captioning, smart assistants, and interactive systems that must respond while the user is still speaking. It’s a dense architecture with 0.245B parameters, optimized for streaming inference rather than batch processing.

Moonshine Streaming Medium uses a sequence-to-sequence Transformer with a novel “ergodic” sliding-window encoder. The encoder processes audio in overlapping chunks using bounded local attention and no positional embeddings — positional information is injected via an adapter before the autoregressive decoder. This design eliminates the need to wait for the full audio clip before starting transcription, enabling sub-100ms streaming latency on modest hardware.

Key architectural characteristics:

• 50 Hz audio frontend that converts raw waveform into a compact feature stream.
• Sliding-window Transformer encoder that caches state between windows, avoiding redundant computation.
• Autoregressive decoder with standard cross-attention, generating text tokens incrementally.
• Support for Hugging Face Transformers (pipeline tag: automatic-speech-recognition), plus native C API and pre-built packages for iOS, Android, Python, macOS, Windows, Linux, Raspberry Pi, and wearables.
• Single-language model (English only) trained from scratch on diverse speech datasets.

The context window is effectively unbounded for streaming: the model processes audio indefinitely by shifting the attention window forward. No context length is specified, but practical usage suggests it handles continuous streams of at least several minutes without degradation.

Moonshine Streaming Medium is built for one thing: converting English speech to text in real time on local hardware. It excels at:

• Live voice agents and conversational AI – The sub-200ms streaming latency makes it suitable for turn-based dialogue systems where the model transcribes speech while the user is still talking.
• On-device captioning – Run directly on a laptop, tablet, or Raspberry Pi to generate closed captions for meetings, lectures, or live events without sending audio to the cloud.
• Voice-controlled smart devices – IoT, wearables, and home automation endpoints can use the 34M–245M parameter family; the Medium variant is appropriate for devices with at least 512 MB of RAM.
• Privacy-sensitive transcription – Since everything runs locally, no audio leaves the device. Use cases include medical dictation, legal transcription, and personal assistant applications.
• Prototyping and research – The MIT license and Hugging Face integration allow easy experimentation with customizable voice pipelines.

The model is not suited for multilingual transcription, music/sound event detection, or offline batch processing of very long recordings (non-streaming models like Whisper may be more efficient for that).

The tiny parameter count makes Moonshine Streaming Medium extraordinarily accessible. At 0.245B parameters, even the FP32 weights consume roughly 1 GB of VRAM / system memory. Quantization drops this further:

Recommended hardware:

• Consumer GPU: RTX 3060 or better is overkill – even integrated GPUs in recent Intel/AMD processors handle it easily. An RTX 4090 will give near-instantaneous token generation but is unnecessary.
• Apple Silicon: M1 or later (M4 Max included) runs the model in <100ms per utterance using Core ML or the Moonshine Python package.
• Edge devices: Raspberry Pi 4/5 with 2 GB+ RAM can run the Q4_K_M variant at approximately 2–3× realtime (e.g., transcribing 10 seconds of audio in 3–4 seconds). For real-time streaming, the Tiny (26M) or Small (123M) variants are preferred.
• CPU-only: Works on any modern x86 or ARM CPU – expect 1–4× realtime depending on hardware.

Performance expectations: On an RTX 3090, the model achieves roughly 50–80 tokens per second for the decoder, while the encoder runs at ~10–20 ms per 30 ms audio window (0.3–0.6 real-time factor). Overall end-to-end latency from audio input to first word is typically below 100 ms on a mid-range GPU.

Quickest path to run locally: Install the Moonshine Python package via pip install moonshine (see [GitHub](https://github.com/usefulsensors/moonshine)). Alternatively, use Hugging Face Transformers with MoonshineStreamingForConditionalGeneration as shown in the model card. No API keys or cloud accounts are needed.

vs. Whisper Large v3 – Moonshine Medium beats Whisper Large v3 on the OpenASR Leaderboard (6.65% WER vs. ~7.4%) while being 30× smaller. Whisper is more robust to noisy environments and supports 99 languages; Moonshine is strictly English. For streaming, Moonshine’s sliding-window encoder provides lower latency and lower memory overhead. If you need multilingual speech recognition, Whisper remains the better choice.

vs. other edge ASR models (e.g., Paraformer-Large, Wav2Vec2-Large) – Moonshine Medium’s 245M parameters place it between small and medium edge ASR models. Paraformer-Large (~220M) offers competitive WER but is not designed for streaming. Moonshine’s explicit streaming architecture gives it a latency advantage. Wav2Vec2-Large (300M) requires fine-tuning for transcription (CTC-based) and lacks native streaming support; Moonshine is ready out of the box.

When to choose Moonshine Streaming Medium: You need real-time English transcription on a device with limited compute (Raspberry Pi, phone, low-end PC) and you want accuracy comparable to top cloud ASR APIs without sending data off-device. The MIT license and cross-platform support make it a low-friction option for production voice pipelines.