Decision-2.0-Vega-27B

Decision-2.0-Vega-27B is a 29.37B parameter dense model developed by the vLLM Semantic Router team, designed specifically for zero-generation classification, semantic routing, and structural scoring. Released under the Apache-2.0 license, Vega-27B is constructed as a trained LoRA adapter on top of Qwen3.8-27B. Instead of using autoregressive decoding to output structured text, JSON blobs, or tool-call tags, the model evaluates input data against multiple predefined questions simultaneously in a single forward pass, outputting raw probability distributions across all choices without generating text tokens.

This single-pass mechanism addresses a critical bottleneck in agentic and automated inference systems: routing latency. When orchestrating multi-agent workflows, utilizing a standard generative LLM to categorize intents, extract boolean assertions, or assign triage ratings consumes substantial compute cycles and introduces parsing fragility. Vega-27B eliminates text generation overhead entirely, clocking a median single-GPU latency of 71.4 ms for single-question requests while scoring a class-leading 74.0 on JevArena and 56.5 on the Jev Decision Index (0.2.1 kit).

For local deployments, Vega-27B functions as a high-capacity System-1 routing engine. It acts as an authoritative front-line filter capable of evaluating user state against complex routing trees, leaving heavier, latency-intensive generative models free to execute actual task fulfillment.

Decision-2.0-Vega-27B is built on a dense transformer architecture totaling 29.37B parameters, using Qwen3.8-27B as its foundation base. Rather than generating response sequences token by token, the model couples its underlying representation layers with specialized decision heads via parameter-efficient fine-tuning (PEFT).

The model ingests two primary inputs: an arbitrary state (plain text or structured JSON) and a dictionary of structured questions. In a single forward pass, the model processes the context and computes normalized log-probabilities across every candidate option.

Key technical specifications include:

• Total Parameters: 29.37B (dense)
• Base Architecture: Qwen/Qwen3.8-27B (LoRA adapter implementation)
• Context Length: 32,768 tokens
• Inference Mode: Non-autoregressive forward pass (zero generated tokens)
• Decision Types: Discrete Choice (up to hundreds of options), Yes/No assertions, and Ordinal Scoring
• License: Apache-2.0

Because Vega-27B does not generate sequences autoregressively, its operational profile differs sharply from standard text models. Traditional LLM inference requires maintaining and expanding an autoregressive KV cache across dozens or hundreds of generated output steps. Vega-27B runs strictly in prefill mode: the input prompt and question definitions are processed together, the output logits are mapped directly to the defined answer slots, and the process completes immediately. This keeps memory footprints deterministic and drastically reduces execution variance.

Vega-27B is engineered for fast, programmatic infrastructure decisions where reliability and low latency are non-negotiable. Its native decision heads support three primary modalities:

• Choice: Selecting the optimal destination or category from a set of discrete options (for example, triage routing between billing, engineering, support, or security).
• Yes/No (Boolean): Evaluating explicit conditional assertions against the input text (for example, verifying whether a refund request contains an invoice number).
• Score: Rating attributes along an ordinal scale based on qualitative criteria (for example, evaluating urgency from routine to immediate).

These capabilities enable several concrete deployment architectures:

In multi-agent systems, incoming user queries must be dispatched to specialized agents or toolchains. Vega-27B handles complex intent routing across dozens of candidate endpoints in under 100 ms, supplying calibrated confidence scores for each route. If the top probability falls below an engineering threshold, the system can immediately trigger an escalation fallback.

Instead of running heavy moderation prompts through an external API, you can run Decision-2.0-Vega-27B locally to evaluate inputs against strict compliance policies. A single forward pass can simultaneously check for prompt injection patterns, policy violations, and PII presence across the full 32,768-token window.

Structured data pipelines often suffer from JSON syntax errors or hallucinated keys when using generative models for classification. Because Vega-27B outputs log-probabilities directly over predefined answer sets, output parsing failures are mathematically impossible.

Deploying a local AI model with 29.37B parameters in 2026 requires understanding its memory profile. Because Vega-27B does not generate text, you do not need to budget VRAM for dynamic autoregressive generation tokens, though processing large contexts up to 32k tokens still requires adequate working memory.

The memory required to load the model depends on the precision level:

• FP16 / BF16 (Unquantized): Requires ~58 GB to 62 GB of VRAM. Needs a multi-GPU setup (such as 2x RTX 3090 / 4090 24GB or an enterprise A100/H100/L40S).
• 8-bit Quantization (INT8 / FP8): Requires ~32 GB to 35 GB of VRAM. Fits comfortably across two consumer GPUs or a single workstation card (RTX 6000 Ada, A6000 48GB, or Apple Silicon with 48GB+ unified memory).
• 4-bit Quantization (Q4_K_M / AWQ / GPTQ): Requires ~18 GB to 20 GB of VRAM. This is the sweet spot for single consumer cards, fitting within the 24 GB pool of an RTX 3090, RTX 4090, or Apple Silicon unified memory configurations (36GB+ M3/M4 Max).

• Best Consumer GPU Configuration: A dual NVIDIA GeForce RTX 3090 or RTX 4090 rig (48 GB VRAM pooled via split execution or tensor parallelism) provides the best GPU for Decision-2.0-Vega-27B at native or 8-bit precision.
• Single GPU Setup: If your goal is how to run 29.37B model on RTX 4090 hardware, you must use 4-bit quantization (such as AWQ or Q4_K_M). A quantized 4-bit model leaves roughly 4 GB to 5 GB of headroom on a 24 GB card for context processing buffers.
• Apple Silicon: Mac Studio or MacBook Pro models equipped with an M3/M4 Max or Ultra chip and at least 36 GB (preferably 64 GB) of unified memory run this model smoothly using MLX or ONNX ports.

Searches regarding Decision-2.0-Vega-27B tokens per second reflect a misunderstanding of how the model functions. Because the engine does not output generative text tokens, traditional metrics like generation tokens per second do not apply.

Instead, Decision-2.0-Vega-27B performance is measured in batch forward-pass latency and queries per second (QPS):

• Single-Question Latency: Median of ~71.4 ms on a modern workstation GPU.
• Batch Processing: Because decision heads evaluate choice, boolean, and score tasks in parallel during the initial prompt processing pass, adding three or four simultaneous questions to the same input context adds negligible latency compared to running multiple independent generation passes.

Loading the model requires Python with transformers >= 5.17, PyTorch, Safetensors, and PEFT. Because the custom decision heads and system_one execution path are bundled with the repository, you must enable trust_remote_code=True.

1import json
2from transformers import AutoModel
3
4model = AutoModel.from_pretrained(
5    "vllm-sr/Decision-2.0-Vega-27B", 
6    trust_remote_code=True,
7    device_map="auto"
8)
9
10result = model.system_one(
11    state="Customer received broken display unit. Demanding refund immediately.",
12    questions={
13        "department": {
14            "type": "choice",
15            "instructions": "Route to correct department",
16            "criteria": {
17                "support": "General hardware defects and returns",
18                "billing": "Invoices and credit card disputes",
19                "sales": "New orders and product inquiries"
20            }
21        },
22        "escalation_required": {
23            "type": "yes_no",
24            "instructions": "Does the request require manager escalation?"
25        },
26        "severity": {
27            "type": "score",
28            "instructions": "Rate urgency",
29            "criteria": ["Low", "Medium", "Critical"]
30        }
31    }
32)
33
34print(json.dumps(result["answers"], indent=2))

For environments where Ollama or standard GGUF runtimes are preferred for local execution, community GGUF conversions and ONNX runtimes are available, though running via native Transformers or vLLM preserves direct access to the model.system_one() method.

When evaluating Decision-2.0-Vega-27B vs AutoJev-27B, Vega-27B holds the top benchmark ranking among identical-size decision models:

Engineers frequently attempt routing tasks using general-purpose models like Qwen2.5-32B-Instruct or Mistral-Small-24B paired with JSON mode or function calling.

• Latency: Generative models must process the context and then generate 20 to 100 output tokens to form a valid routing block, yielding typical latencies of 400 ms to 1,500 ms. Vega-27B completes the same evaluation in roughly 71.4 ms.
• Reliability: Generative routing requires parsing JSON strings, which can fail due to token corruption or syntax drift under high concurrency. Vega-27B extracts probabilities straight from network heads, eliminating parsing errors.
• Compute Efficiency: Because Vega-27B halts after the prefill phase, it consumes significantly fewer compute cycles per query than an autoregressive model generating full sequences.

If your local architecture requires generating freeform text, Vega-27B is the wrong choice. If your pipeline demands deterministic categorization, dynamic routing, or safety screening at hardware line-rate, Decision-2.0-Vega-27B provides an optimal balance of 29.37B classification capacity and sub-100ms local execution.