Granite 4.0 1B Speech

Architecture: Specialized acoustic encoder coupled with the Granite 4.0 1B language backbone:

1. Conformer acoustic encoder — 16 Conformer blocks trained with CTC over an ASCII vocabulary for European languages plus Katakana phonetics for Japanese, with 4-second block-attention windows and self-conditioned CTC.
2. Windowed Q-Former modality adapter that downsamples acoustic embeddings 10× and projects them into the Granite 4.0 text-embedding space.
3. Granite 4.0 1B Base (hybrid Mamba-2 / Transformer architecture, 128k context) finetuned jointly on the speech corpora.

Modality: Audio-to-text with text-only fallback (the Granite 4.0 backbone). Supports speculative decoding for faster inference.

Training: Trained on IBM's Blue Vela cluster for 30 days (26 days encoder + 4 days projector) on 8 H100 GPUs, using public ASR/AST corpora and synthetic data targeted at Japanese ASR, keyword-biased ASR, and speech translation.

Capabilities & use cases:

• Multilingual ASR in English, French, German, Spanish, Portuguese, Japanese.
• Bidirectional AST between those languages and English, plus En→It and En→Zh.
• New keyword list biasing for proper-noun, acronym and domain-vocabulary recognition.
• Designed for edge deployments (laptops, on-prem, Apple Silicon via mlx-audio); RTFx ≈ 280× real-time. Half the parameters of granite-speech-3.3-2b with higher English transcription accuracy.