Clef

Clef is Cloudflare's open-weights decision model, designed to eliminate the latency, parsing failures, and non-determinism of standard autoregressive generation in agentic workflows. Built as a dense 27-billion-parameter network post-trained from Qwen/Qwen3.8-27B, Clef bypasses free-form text generation entirely. Instead of outputting tokens sequentially, it reads a state (such as structured JSON, raw text, or multimodal inputs) alongside a typed schema of questions, then returns calibrated probabilities for every permissible option across all questions in a single forward pass.

Released under the Apache 2.0 license, Clef represents a shift toward dedicated "System One" decision components within agent architectures. Where standard large language models require multi-second token generation and fragile JSON schema adherence to make simple triage or routing decisions, Clef produces bounded, typed classification logits directly. Practitioners deploying a local AI model with 27B parameters in 2026 can run Clef locally to handle high-throughput routing, policy enforcement, and classification pipelines without paying the compute tax of autoregressive decoding.

Clef couples a dense 27B transformer backbone with a specialized routing head. The underlying foundation is post-trained from Qwen/Qwen3.8-27B, retaining its core weights and representation capabilities while repurposing the final layers for simultaneous schema evaluation.

Standard LLM classification prompts rely on decoding an answer token by token. Clef replaces the standard language modeling head with a joint schema head:

• Backbone Representation: The 27B dense backbone processes the full context through bidirectional-like pooling of state and question tokens.
• Evidence Routing: The joint schema head is a compact transformer module that consumes the backbone's final hidden states, cross-attending evidence from the input state to specific question schemas.
• Single Forward Pass: Output logits are computed jointly for all valid options across every submitted question in parallel. Applying a per-question softmax yields normalized probabilities across categories (Choice), boolean indicators (Noul), or regression bounds (Score).

Because the inference pass stops after evaluating the prompt and computing output logits, Clef avoids the autoregressive decoding phase altogether. This architecture fundamentally decouples decision latency from generation length: whether your schema contains one question or ten, the inference time remains essentially flat.

Clef supports a 65,536-token context window (64K). Dense 27B models usually encounter memory bandwidth bottlenecks when scaled to 64K contexts during generation. However, because Clef only performs prompt ingestion (prefill) and a single forward evaluation step, the memory overhead associated with maintaining a dynamic key-value (KV) cache across thousands of generated tokens is eliminated. The memory footprint scales strictly with the size of the ingested state and the batch size.

Clef is not a conversational chatbot or a code synthesis engine; it is a dedicated decision engine. The model is optimized for high-reliability, low-latency evaluation across discrete tasks:

• Agent Routing and Tool Dispatch: Routing an incoming user request to one of multiple distinct agent subroutines based on structured input states, eliminating tool-calling schema parsing errors.
• Real-Time Content Moderation and Guardrails: Evaluating text streams against enterprise compliance policies, scoring violations with precise probabilities across multiple safety categories simultaneously.
• Workflow Triage: Parsing complex, multi-page customer tickets or vendor invoices against business logic schemas (for example, determining billing status, urgency level, and routing destination in a single call).
• Synthetic Data Labeling: High-throughput dataset tagging where deterministic output distributions are required across rigid taxonomies.

Because the output is bounded by design, Clef cannot hallucinate keys, break JSON formatting, or produce out-of-distribution values.

To run Clef locally, practitioners must account for the 27B parameter footprint. Because Clef executes only the forward pass without generating downstream tokens, the traditional metric of Clef tokens per second is less relevant than end-to-end forward pass latency. On enterprise hardware like an NVIDIA H200, median latency clocks in at roughly 209 ms. On consumer hardware, performance depends on compute bandwidth during the prefill phase.

Memory requirements depend heavily on quantization. The model checkpoint contains both the backbone weights and the custom joint schema head.

For most self-hosted production setups, the best quantization for Clef is FP8 or a high-quality 4-bit format (such as Q4_K_M). Running at 4-bit allows the weights to fit within the 24GB buffer of a single consumer GPU while preserving the calibration of its decision logits.

For engineers wondering how to run 27B model on consumer GPU setups, single-card execution is entirely achievable if you quantize:

• Single NVIDIA RTX 4090 / 3090 (24GB): The sweet spot for running Clef locally at consumer tier. At Q4_K_M quantization, the model fits with roughly 5GB of headroom for context prefill up to 16K tokens. Forward pass latency on an RTX 4090 typically ranges between 350 ms and 600 ms depending on context length.
• Dual RTX 3090 / 4090 (48GB Total): Running the model across two GPUs via pipeline or tensor parallelism allows uncompressed 8-bit precision or low-context BF16 execution, handling long 64K contexts with ease.
• Apple Silicon (Mac Studio / MacBook Pro): An M3 or M4 Max with 36GB, 48GB, or 64GB of unified memory provides an accessible local platform. While memory bandwidth on Apple Silicon is lower than dedicated GDDR6X, Clef performance remains viable for low-concurrency decision workloads, completing forward evaluations in under one second.

The best GPU for Clef in high-density local production environments is a single 48GB card (such as an NVIDIA RTX 6000 Ada or L40S) for 8-bit inference, or an 80GB A100/H100 if running the original BF16 weights at high batch sizes.

Clef requires custom model definition code to handle its joint schema head, meaning standard inference runtimes require specific support:

1. SGLang (Recommended for Production): SGLang provides optimized serving containers for Clef that natively expose a Jev/SystemOne compatible endpoint at /v1/systemone. Launching the official SGLang container with --model-path Cloudflare/clef enables batched inference, kernel optimizations, and standard API compatibility.
2. Hugging Face Transformers / PyTorch: You can execute Clef via the joint_schema_model.py module included in the Hugging Face repository. This requires torch >= 2.11 and transformers >= 5.10.2, loading the model via load_release_model() and collating input records directly in Python.
3. Ollama and GGUF Runtimes: Because Clef utilizes a custom joint schema head rather than standard autoregressive next-token prediction, standard GGUF and Ollama model wrappers cannot run the model out of the box without an engine build that supports the joint head architecture. For quick testing on desktop hardware, use the SGLang container or the direct Python loader.

Evaluating Clef hardware requirements and architectural trade-offs requires comparing it directly to its architectural peers and conventional LLMs.

Typesafe AI's Jev pioneered the current System One decision model category. Clef was designed as a drop-in replacement, matching the Jev API request specification (state, questions, and typed schemas for choice, noul, and score).

• Weight Accessibility: Jev is primarily consumed as a hosted API. Clef provides fully open weights under the Apache 2.0 license, allowing full local sovereignty and on-premises deployment.
• Input Flexibility: While baseline Jev is text-focused, Clef extends the state input pipeline to process multimodal states (including image tensors) directly within the same schema evaluation pass.
• Footprint: Clef's 27B parameter footprint requires roughly 16GB to 54GB of VRAM depending on quantization, whereas smaller dedicated classifiers often trade contextual reasoning capacity for a sub-8B parameter footprint.

Practitioners often attempt to achieve structured decisions by prompting general-purpose models using JSON mode or grammar-constrained sampling.

• Latency: General LLMs must ingest the prompt and sequentially output tokens (often 50 to 200 tokens of formatted JSON). Even at 60 tokens per second, a standard 32B model takes 1 to 3 seconds to complete the request. Clef executes a single forward pass, returning probabilities across all questions in 200 ms to 500 ms.
• Reliability: Standard LLMs can fail via schema drift, truncated JSON, or probability hallucinations. Clef outputs logits directly against defined keys, guaranteeing 100% schema compliance without post-processing or retries.
• Compute Efficiency: When serving high-volume routing workflows, Clef requires significantly fewer GPU compute cycles per decision, making it far cheaper and faster to host locally at scale.