LFM3-Omni-2B

LFM3-Omni-2B is Liquid AI's first native omni-modal foundation model, designed from scratch to collapse the traditional multi-stage local pipeline into a unified on-device architecture. Instead of chaining an automatic speech recognition (ASR) engine, a vision-language model, an LLM, and a text-to-speech (TTS) synthesizer together, this model processes text, still images, video (with or without audio), and 16 kHz acoustic data directly within a single 2.17B dense parameter network. It produces text, native 24 kHz speech, or interleaved text and speech responses with minimal overhead.

By operating as an end-to-end multimodal system, LFM3-Omni-2B cuts out the compounding latency and error propagation typical of cascaded edge pipelines. It occupies a rare niche in the small edge model ecosystem: a sub-3B parameter model capable of handling real-time conversational visual Q&A and multimodal tool calling without pinging external APIs. Released under the LFM Open License v1.0, it is built explicitly for resource-constrained devices such as edge gateways, laptops, robotics controllers, and in-vehicle systems.

This is a non-reasoning model optimized for responsive execution rather than deep multi-step algorithmic derivation. For developers aiming to run LFM3-Omni-2B locally, it offers an exceptionally low memory footprint while maintaining strong instruction-following and function-calling abilities across both auditory and visual inputs.

LFM3-Omni-2B uses a dense 2.17B parameter architecture that incorporates Liquid Foundation Model (LFM) sequence-modeling blocks rather than relying purely on standard multi-head self-attention. The overall architecture coordinates four main components: an LFM3 base backbone, an LFM3 vision encoder, an LFM3 causal audio encoder, and the LiquidCodec audio tokenizer paired with a depth-decoder network.

Unlike Mixture-of-Experts (MoE) architectures that swap active parameter pathways, dense models like LFM3-Omni-2B activate all 2.17 billion parameters on every token or audio code step. At this scale, a dense network guarantees uniform execution timing, straightforward memory management, and high compute density on edge hardware with limited memory bandwidth.

• Total Parameters: 2.17B dense parameters
• Context Window: 32,000 tokens
• Input Modalities: Text, images, temporal video patches with synchronized soundtrack, and 16 kHz raw audio
• Output Modalities: Text, 24 kHz speech audio codes via Liquid Audio Codec, or interleaved text and speech
• Generation Control: Greedy decoding for standard text output; specialized audio decoding via audio_temperature=0.4, audio_repetition_penalty=1.2, and audio_repetition_window=64

The 32K context window enables significant ingestion headroom. A developer can feed dozens of video frames alongside an interleaved audio track or hold extended voice conversations without exhausting context memory. Ingestion passes input data through specialized projectors directly into the main sequence stream: visual tokens map to <|image|> and <|video|> placeholders, while 16 kHz log-mel audio features insert into <|audio|> placeholders. Outbound speech is generated by driving the internal depth-former and Liquid Audio Codec via <|audio_out|> tokens, bypassing external TTS latency completely.

LFM3-Omni-2B handles real-time edge interactions across visual and auditory tasks that previously required multiple models working in concert.

The model processes image and audio inputs simultaneously. In practice, this means an operator can point a camera at machinery, ask a verbal question ("What valve position is indicated here?"), and have the model identify the visual feature and respond via synthesized 24 kHz voice. Because it accepts video with synchronized audio tracks, it can analyze surveillance or inspection footage where visual actions and acoustic events must be correlated, such as diagnosing mechanical rattling during an operational cycle.

LFM3-Omni-2B supports structured function-calling triggered by mixed-modality prompts. Developers can expose system APIs where tool arguments are extracted straight from live camera feeds and spoken commands. For example, speaking "Log this temperature reading to inventory" while aiming a sensor camera allows the model to extract the numeric readout visually and format a structured JSON payload for execution.

The model supports distinct sets of languages for text and speech processing:

• Text Understanding & Generation: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, and Polish.
• Speech Synthesis (24 kHz): British English, American English, Arabic, Danish, German, Greek, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, and Chinese, with options for female and male voices.

Running an omni-modal network locally requires accounting not only for the language backbone, but also the active vision encoders, audio encoders, and the codec depth-decoder.

Because this model contains multiple sub-networks, total memory usage depends on whether the encoders and decoders are unified in system RAM or separated across GPU VRAM.

The best quantization for LFM3-Omni-2B deployments on consumer hardware is Q4_K_M (or q4f16 when targeting ONNX DirectML/CUDA runtimes). The 4-bit decoder keeps the text embedding, LM head, and sensitive projection layers at higher precision (int8/fp16) while compressing standard linear weights to 4-bit. This configuration keeps total memory consumption well under 3 GB without degrading phonetic clarity in speech generation.

• NVIDIA Consumer GPUs: The model easily fits entirely within the VRAM of an RTX 3060 (12 GB), RTX 4060 (8 GB), or RTX 4090 (24 GB). On an RTX 4090, inference operates almost entirely cache-bound, providing instant response times for both speech and text tokens. If you are looking for the best GPU for LFM3-Omni-2B, any modern 8 GB VRAM card will give you maximum performance and full offload capacity.
• Apple Silicon: Macs with unified memory (M2, M3, or M4 series with 8 GB or 16 GB RAM) handle this model with virtually no memory pressure.
• Edge Devices: On a Raspberry Pi 5 (8 GB) or an Intel N100 mini PC running CPU-only quantized packages via ONNX Runtime or llama.cpp, text generation remains functional, though real-time 24 kHz speech synthesis requires careful thread allocation.

On modern desktop GPUs such as an RTX 4070 or 4080, LFM3-Omni-2B delivers high throughput:

• Text Generation: 90 to 140+ tokens per second on mid-to-high-tier GPUs.
• Apple Silicon (M3/M4 base or Pro): Expect 60 to 95 tokens per second on text generation.
• Speech Output Latency: Speech generation runs through the LiquidCodec decoder. On GPU and Apple Silicon via Metal, audio generation reaches real-time factors (RTF) well below 0.15, meaning audio generates significantly faster than it plays back.

For standard text and multimodal workflows, Ollama and llama.cpp provide the quickest way to get started. For end-to-end voice-to-voice pipelines requiring audio input and 24 kHz output, deployment through Hugging Face Transformers (transformers>=5.20.2) or ONNX Runtime with Liquid's custom processing graph ensures full access to audio encoder and decoder tensors.

Evaluating LFM3-Omni-2B against comparable small models highlights its specific engineering focus.

• LFM3-Omni-2B vs. Microsoft Phi-3.5-vision (4.2B): Phi-3.5-vision is roughly twice the parameter size, providing stronger zero-shot text reasoning and detailed document reading. However, Phi-3.5-vision lacks any native audio processing, speech input, or speech generation capabilities. It also requires nearly double the VRAM. If your workload demands a voice-in, voice-out assistant, Phi-3.5-vision forces you to run separate Whisper and TTS services alongside it, whereas LFM3-Omni-2B handles the entire loop inside 3 GB of memory.
• LFM3-Omni-2B vs. Qwen2-Audio-7B: Qwen2-Audio provides audio comprehension and speech analysis, but at 7 billion parameters, it demands an 8 GB to 16 GB hardware profile just for text and audio understanding, and it does not natively emit speech audio. LFM3-Omni-2B is substantially smaller, runs at higher token-per-second rates on low-tier consumer hardware, adds simultaneous visual understanding, and generates speech directly.

Choose LFM3-Omni-2B when building compact, multimodal edge assistants that require real-time physical interaction through voice and vision. If your local AI project centers strictly on complex coding, long-form document drafting, or formal mathematical reasoning, larger text-focused models remain the appropriate alternative.