Vela 3.0

Vela 3.0 is a family of open decision models developed collaboratively by vLLM Semantic Router, KR Labs, and AMD. Released in late October 2026, the collection spans sizes from 380M up to 27B parameters, engineered specifically to handle classification, semantic routing, and token-level span extraction in a single inference call. Rather than using autoregressive generation to output structured JSON, Vela 3.0 processes prompts in a single forward pass, returning exact character offsets, classification scores, and probabilities without token generation overhead.

Operating under an Apache-2.0 license (with select fast vision components under MIT), Vela 3.0 targets production infrastructure where generative latency cannot be tolerated. The architecture is built for agentic triage: determining routing paths, scoring intent, detecting out-of-domain queries, and flagging hallucinated or unsupported claims across RAG pipelines.

For engineers seeking to run Vela 3.0 locally, the family fills a distinct gap between dedicated discriminative encoders like DeBERTa or GLiNER and general-purpose generative LLMs. By anchoring specialized classification and extraction heads onto frozen, highly capable dense bases, Vela delivers zero-shot entity extraction and routing accuracy that outpaces standard prompted baselines while maintaining sub-50 ms execution times on local accelerators.

The core architecture of the Vela 3.0 models uses a dense transformer foundation augmented with gated adapters and task-specific classification and span heads. Rather than fine-tuning the entire weight matrix, the architecture freezes the underlying base model (such as the 8.4B parameter base in vela3-flash or the 26.1B base in vela3-turbo) and routes intermediate representations through modular adapters.

This design yields several technical advantages for local deployment:

• Bit-for-Bit Determinism: Standard decision tasks (yes/no, multiple-choice, scoring) map directly through the frozen base weights, guaranteeing identical behavior to the base checkpoints.
• Single-Pass Span Extraction: Span targets (named entities, relations, personal data, unsupported claims) are predicted directly via adapter heads. The model extracts start and end token indices alongside confidence probabilities without generating text tokens.
• Zero Generative Latency: Because inference requires only a prompt evaluation step (prefill) rather than an autoregressive decode loop, latency scales strictly with prompt length, eliminating the memory bandwidth bottlenecks associated with decoding key-value caches.

Context handling is optimized for dense document analysis and triage, supporting long prompt spans for RAG auditing and classification.

Vela 3.0 replaces multi-model extraction pipelines with a consolidated decision engine. In local production systems, it serves four primary roles:

The model acts as a front-line gateway for local LLM clusters. With zero-shot intent accuracy hitting over 83% on complex out-of-domain splits like MASSIVE, Vela 3.0 classifies user requests and dispatches them to smaller task-specific models or larger generative backends without regex or vector search overhead.

Vela 3.0 includes dedicated heads trained on datasets like LettuceDetect and RAGTruth. It flags unsupported claims within generated responses, pointing directly to the character span where a factual break occurs. This enables local agentic pipelines to run automated fact-checking on model outputs before returning them to the user.

Unlike traditional NER models that require fixed label taxonomies, Vela 3.0 accepts dynamic label strings at runtime. On zero-shot entity recognition benchmarks like OOD-7, the 27B variant hits 64.6 F1 from raw label names alone, outperforming specialized baselines like GLiNER-multitask-large. It also extracts subject-predicate-object relations in one call, providing an end-to-end relation extraction score of 35.9 on standard GLiFormer protocols.

Because Vela 3.0 relies on prompt processing rather than continuous token decoding, Vela 3.0 hardware requirements differ significantly from standard generative models. Compute capability during the prefill phase and raw memory bandwidth dictate latency, while KV cache memory footprint remains practically zero.

Memory requirements depend on which family member you deploy:

The best quantization for Vela 3.0 depends on whether you run classification or fine-grained span detection:

• Q4_K_M: The ideal balance for local deployment on consumer GPUs. The 9B vela3-flash variant fits comfortably on a budget 12GB Nvidia RTX 3060, while the 27B vela3-turbo model runs within the 24GB VRAM ceiling of an Nvidia GeForce RTX 4090 or RTX 3090.
• Q8_0 or FP16: Recommended if your primary workload involves high-precision hallucination detection or nested relation extraction, where sub-token boundary precision is critical.

For Apple Silicon users, an M3 or M4 Pro/Max with unified memory runs vela3-turbo in 8-bit precision with negligible swap, providing an exceptional workstation setup for offline document auditing.

Because there is no autoregressive decoding step, measuring performance using standard "Vela 3.0 tokens per second" metrics is misleading. Instead, performance is evaluated in end-to-end request latency:

• vela3-fast (380M): Delivers roughly 18 ms per span question on consumer GPUs, making it suitable for inline typing predictions and CPU execution.
• vela3-flash (9B): Returns extraction spans and decisions in approximately 40 ms for a 200-token prompt on modern consumer hardware like an RTX 4080.
• vela3-turbo (27B): Executes full entity, relation, and routing analysis across multi-paragraph inputs in roughly 110-180 ms on an RTX 4090 or AMD Radeon RX 7900 XTX.

The model interfaces natively with the vllm-semantic-router engine, which exposes an optimized execution path for single-pass non-generative tasks.

To deploy using Python:

1from vllm_sr import DecisionEngine
2
3# Initialize the 9B decision model
4router = DecisionEngine.load("vllm-sr/vela3-flash", device="cuda", dtype="bfloat16")
5
6# Execute classification and span extraction in a single call
7result = router.system_one(
8    prompt="Identify PII and classify intent for: Contact Alice at [email protected] regarding invoice 402.",
9    spans=["PII", "Email", "Invoice Reference"],
10    choices=["Billing Query", "Technical Support", "General Inquiry"]
11)
12
13print(result.choice)       # 'Billing Query'
14print(result.found_spans)   # Returns offsets for 'Alice', '[email protected]', '402'

For containerized setups or standard local model management, community adapters allow you to query decision endpoints via Ollama or custom vLLM pipelines running behind your API gateway.

Understanding where Vela 3.0 fits against alternative models prevents inefficient resource allocation.

General LLMs must be prompted with complex JSON schemas to extract spans and classify text. This approach requires generating 50 to 200 tokens per request, consuming significant compute and introducing parser failure risks when models produce malformed markdown.

Vela 3.0 bypasses the decode loop entirely. It runs 5x to 15x faster on identical hardware while providing deterministic character offsets. Furthermore, on zero-shot relation extraction, vela3-turbo scores 35.9 on GLiFormer benchmarks compared to 25.7 for prompted Qwen3.5-9B.

Small cross-encoders and GLiNER models excel at token classification within tight parameter footprints, but they struggle with complex cross-domain routing and lack hallucination detection capabilities. Vela 3.0 bridges this gap by unifying classification, routing, and token-level verification into a single dense model.

While a DeBERTa checkpoint requires less base VRAM, Vela provides broader zero-shot generalization across diverse unseen intent scenarios and long contexts, reducing the operational burden of hosting separate models for NER, routing, and guardrails.