Clef-Omni

Clef-Omni is an open-weight 31.83B parameter mixture-of-experts (MoE) decision model released by Cloudflare under the Apache-2.0 license. Built as a specialized post-train of Qwen3-Omni-30B-A3B-Instruct, the architecture routes inference through just 3B active parameters while keeping the full parameter pool available for representation capacity. Unlike standard autoregressive large language models that generate free-form text token by token, Clef-Omni operates as an end-to-end decision engine: it evaluates an input state against a typed schema of questions and produces probability distributions across all predefined choices in a single forward pass.

This operational shift eliminates output parsing, JSON schema validation errors, and the latency overhead of multi-token decoding. By mapping complex input states directly to discrete probability distributions, Clef-Omni addresses high-throughput classification, routing, and scoring tasks that typically require slow, fragile prompt pipelines. It establishes a distinct category in the open-weights ecosystem, functioning less like a conversational agent and more like a high-capacity, multi-task discriminator.

For engineers looking to run Clef-Omni locally, the model presents a unique hardware profile. Because it only computes over 3B parameters per forward pass, inference latency is exceptionally low compared to traditional 30B dense models. However, storing the weights requires allocating enough memory for the full 31.83B parameter footprint. This makes Clef-Omni an ideal candidate for systems where VRAM capacity is sufficient to hold the model weights, allowing users to reap the throughput benefits of sparse execution.

Clef-Omni pairs a modified foundational MoE backbone with a specialized classification head:

• Backbone Network: A 31.83B total parameter mixture-of-experts architecture based on Qwen3-Omni-30B-A3B-Instruct. During any single forward pass, the dynamic routing mechanism activates only 3B parameters.
• Joint Schema Head: Rather than decoding output tokens sequentially through an LM head, the model uses a lightweight transformer head trained on top of the backbone's final hidden states. This joint head correlates evidence from the input state across the entire schema, scoring all options for all typed questions simultaneously.
• Context Length: A native 64,000-token context window, allowing users to pack extensive system logs, lengthy document bodies, structured JSON blobs, or multi-turn interaction traces into a single inference call.
• Output Layer: Outputs calibrated logits for each option in a target schema. Applying a standard softmax per question yields exact probabilities, removing the non-determinism inherent in generative text sampling.

The primary architectural advantage is Clef-Omni MoE efficiency. In generative text models, generation speed is bottlenecked by memory bandwidth during auto-regressive decoding. Clef-Omni avoids auto-regressive decoding altogether. A single forward pass through 3B active parameters extracts features and produces decisions immediately, delivering latency numbers that rival small sub-3B dense models while retaining the semantic understanding of a 30B-class network.

Clef-Omni is engineered specifically for deterministic routing, evaluation, and structured classification pipelines:

• Zero-Parsing Agent Routing: Directing autonomous agent steps by evaluating state histories against available tool schemas. Instead of waiting for an LLM to generate function call strings, Clef-Omni outputs probabilities across available tools in one step.
• Automated Triage & Moderation: Processing incoming tickets, incident logs, or user reports across multi-label schemas (e.g., department assignment, priority level, severity score, and compliance flags) in a unified pass.
• Document & Data Extraction: Extracting categorical attributes from unstructured text and complex JSON records without schema deviations or syntax hallucination.
• SystemOne and Jev Schema Integration: Full drop-in compatibility with the Jev and SystemOne decision APIs, supporting choice (multi-class selection), score (continuous/graded evaluation), and noul (binary true/false) question types.

Because the model outputs normalized probabilities rather than strings, developers can establish programmatic confidence thresholds. If the top probability for a routing decision falls below a calibrated limit (e.g., 0.85), systems can safely escalate the payload to a human operator or a larger reasoning model.

Running a local AI model with 31.83B parameters in 2026 requires understanding the distinction between compute load and memory residency. Clef-Omni requires host or GPU memory large enough to hold all 31.83B parameters, even though computation touches only 3B parameters per forward pass.

Memory requirements depend heavily on your target precision and context utilization:

• bfloat16 (Unquantized): ~64 GB VRAM. Requires an NVIDIA RTX 6000 Ada, an A100 (80GB), an H100/H200, or multiple interconnected GPUs (e.g., 3x RTX 3090/4090 24GB).
• Q8_0 (8-bit Quantization): ~34 GB to 36 GB VRAM. Fits comfortably on dual 24GB GPU configurations (such as 2x RTX 3090 or 2x RTX 4090) or unified memory workstations.
• Q4_K_M (4-bit Quantization): ~19 GB to 22 GB VRAM. This is the best quantization for Clef-Omni on single-GPU consumer hardware, fitting within a single 24GB VRAM frame (RTX 3090, RTX 4090, or RTX 5090) with enough headroom for the 64,000-token context KV cache.

• Best GPU for Clef-Omni (Single GPU): NVIDIA GeForce RTX 4090 (24GB) or RTX 3090 (24GB) running Q4_K_M quantization. For unquantized bfloat16 inference, a single workstation with an NVIDIA A100 80GB or dual RTX 6000 Ada GPUs is recommended.
• Apple Silicon: Mac Studio or MacBook Pro models equipped with M3/M4 Max or M2/M3/M4 Ultra chips and at least 36 GB of unified memory will execute quantized Clef-Omni locally with zero offloading penalties. Systems with 64 GB or more unified memory can run higher precision weights (Q8_0 or FP16) smoothly.
• Multi-GPU Setups: Dual 24GB cards using PyTorch device mapping or tensor parallelism offer the best price-to-performance balance for running high-precision quantization without breaking workstation budgets.

Because Clef-Omni computes decisions in a single forward pass, traditional metrics like auto-regressive tokens per second do not capture its performance profile. Instead, evaluate the model on forward pass latency:

• On a modern 24GB GPU (e.g., RTX 4090 running 4-bit weights), end-to-end evaluation latency typically ranges between 35 ms and 90 ms for standard contexts, enabling throughputs of 15 to 25 full multi-question schema evaluations per second.
• On Apple Silicon (M-series Max/Ultra), batch latency remains consistent across larger schemas thanks to high unified memory bandwidth.

To run Clef-Omni locally, developers currently rely on the official custom PyTorch implementation provided in the model repository, which includes joint_schema_model.py for handling schema encoding and forward evaluations:

1import sys
2import torch
3from huggingface_hub import snapshot_download
4
5# Download repository and import custom schema module
6path = snapshot_download("Cloudflare/clef-omni")
7sys.path.insert(0, path)
8from joint_schema_model import load_release_model, encode_record, collate_records
9
10# Load model to GPU (requires ~64GB VRAM for raw bfloat16)
11model, processor = load_release_model(path, device="cuda")
12
13record = {
14    "state": "Payment failed due to gateway timeout on node-east.",
15    "questions": {
16        "severity": {
17            "type": "choice",
18            "instructions": "Determine operational impact.",
19            "criteria": {
20                "critical": "Complete outage or transaction loss.",
21                "minor": "Non-blocking degradation."
22            }
23        }
24    }
25}
26
27encoded = encode_record(processor.tokenizer, record, processor=processor)
28batch = collate_records([encoded], processor.tokenizer.pad_token_id, torch.device("cuda"))
29
30with torch.inference_mode():
31    logits = model(batch)[0]
32
33for question, q_logits in zip(encoded.questions, logits):
34    probs = q_logits.float().softmax(-1).tolist()
35    print(question.question_id, dict(zip(question.option_ids, probs)))

Standard GGUF loaders like pure Ollama or unpatched llama.cpp instances do not support Clef-Omni out of the box due to its custom joint schema transformer head. Practitioners using Ollama should look for community backports or run the lightweight PyTorch/Transformers server containerized alongside their local services.

Dense generative models like Qwen2.5-32B or traditional MoEs like Mistral 8x7B generate responses sequentially. When tasked with structured extraction, generative models require prompt constraints, grammars (like Outlines or GBNF), and multi-step sampling that consumes compute for every generated token. Clef-Omni trades open-ended text generation for absolute schema adherence and sub-100ms latency. If your pipeline needs creative summaries or free-form reasoning text, choose a standard LLM. If your pipeline needs to route, classify, or score data according to fixed criteria, Clef-Omni is significantly faster and computationally cheaper.

Traditional cross-encoders like DeBERTa-v3-Large are compact (under 500M parameters) and run on entry-level GPUs. However, they lack the deep semantic comprehension, complex instruction-following, and massive 64,000-token context windows found in modern 30B foundation models. Clef-Omni provides the contextual capacity and complex reasoning of a frontier base model while preserving the single-pass classification dynamics of an encoder head. The tradeoff is memory: Clef-Omni requires at least 20 GB of VRAM, whereas small cross-encoders run comfortably inside 2 GB to 4 GB.