Decision-2.0-Nox-4B

Decision-2.0-Nox-4B is a purpose-built 4.21B parameter classification and routing model developed by the vLLM Semantic Router team. Instead of generating autoregressive text token by token, the model functions as a dedicated decision layer. Built on top of Qwen3.5-4B-Base using Low-Rank Adaptation (LoRA) and a specialized multi-task decision head, it takes an input state (raw text or serialized JSON) alongside structured evaluation questions and outputs normalized probabilities for every potential answer in a single forward pass.

Released under the permissive Apache-2.0 license, this dense architecture is engineered specifically for sub-system triage, guardrailing, intent detection, and dynamic agentic routing. Because it eliminates text generation decoding loops entirely, Decision-2.0-Nox-4B operates at ultra-low latency, recording a median processing time of 12.9 ms per single-question query on a single workstation GPU. For developers looking to run Decision-2.0-Nox-4B locally, it offers an efficient drop-in replacement for the sluggish, nondeterministic prompt-based LLM classifiers that typically introduce bottlenecks into production agent pipelines.

As a local AI model with 4.21B parameters released in 2026, Nox 4B hits a sweet spot between the ultra-lightweight edge routers in the Decision 2.0 suite (such as the 0.6B Kai and 0.8B Eos) and heavier reasoning engines like Vega 27B. It handles multi-question schemas simultaneously, returning probability distributions across discrete categories, binary determinations, and ordinal scales while keeping host memory usage low enough to fit comfortably on standard consumer hardware.

The underlying backbone of Decision-2.0-Nox-4B is a dense 4.21B parameter transformer initialized from Qwen3.5-4B-Base. The engineering team retained the base model's sequence modeling and semantic comprehension, augmenting the upper layers with trained LoRA adapters connected directly to a customized classification head.

Unlike traditional generative language models that must sample and output tokens to answer a prompt, Decision-2.0-Nox-4B uses a "System 1" non-generative architecture. The execution pipeline pools representation vectors from the processed input context and routes them straight through the classification heads:

• Single-Pass Multi-Head Evaluation: The model evaluates three native question types concurrently: choice (categorical selection against distinct text descriptions), noul (yes/no binary classification), and score (ordinal evaluation across an ordered sequence of qualitative levels).
• Zero Token Generation Overhead: Because no auto-regressive decoding or key-value (KV) cache expansion across hundreds of output steps takes place, execution latency is determined entirely by prefill compute time.
• 16,384 Token Context Window: A context window of 16k tokens allows the model to digest complex multi-turn conversation logs, lengthy API traces, legal policies, or dense JSON state blobs before resolving classification queries.
• Direct Probability Calibration: The model does not emit strings like "True" or "Category B" that require regex parsing. It outputs exact softmax probability distributions per question field, enabling programmatic thresholding inside application logic.

Decision-2.0-Nox-4B is optimized for deterministic routing, real-time safety evaluation, and state analysis where low latency and predictable output structures are non-negotiable.

In automated support and agent architectures, incoming user messages must be quickly categorized before reaching larger, expensive generation models. Nox 4B can evaluate an incoming ticket, simultaneously select the target engineering department, flag whether the customer is requesting a financial refund, and rate user frustration on a multi-tier scale in roughly 13 ms.

The model handles structured constraint checking against corporate safety guidelines. It can analyze multi-turn dialogue histories to verify whether sensitive credentials have been exposed or whether a request deviates from allowable enterprise operational boundaries.

However, practitioners should note empirical security boundaries revealed during benchmark stress testing: while Nox 4B scores 96.9% accuracy on structured technical compliance and inspection audits, it can struggle with multi-layered social engineering prompts that apply urgent authority pressure. For high-stakes security perimeters, probability thresholds must be tuned conservatively or supplemented with multi-agent consensus checks.

Because the model ingests structured JSON data as natively as unstructured text, it excels at validating data schemas against operational business rules. Examples include assessing whether incoming webhook payloads meet compliance criteria for automated processing, scoring mortgage approval risk tiers, and flagging incomplete records without writing bespoke deterministic heuristic scripts.

Evaluating Decision-2.0-Nox-4B hardware requirements is straightforward because memory consumption is fixed and predictable. Because there is no extended generation phase, VRAM is consumed almost entirely by base weights and the single prefill pass, eliminating the massive KV-cache bloat typical of 16k-context generative LLMs.

• Best GPU for Decision-2.0-Nox-4B: An Nvidia GeForce RTX 4070 (12 GB) or RTX 4080 (16 GB) provides optimal performance, effortlessly executing unquantized BF16 passes in under 15 ms. For budget setups, an RTX 3060 (12 GB) or RTX 4060 Ti (16 GB) offers outstanding value, running the full unquantized model at native speed.
• Entry-Level Consumer GPUs: To run this 4.21B model on consumer GPUs with 6 GB to 8 GB of VRAM (such as an RTX 2060, RTX 3050, or RTX 4060 8GB), loading the model in 8-bit precision or 4-bit quantization allows full throughput with virtually zero accuracy degradation.
• Apple Silicon: Macs with unified memory (M2/M3/M4, Pro, or Max chips) with 16 GB or more of RAM run the model with ease via PyTorch MPS backends, offering latencies around 20 ms to 35 ms.

Unlike open-ended creative models where quantization artifacts manifest subtly in prose style, aggressive quantization on classification heads can skew calibrated softmax probabilities near threshold boundaries.

For critical decision pipelines, FP16 or native BF16 is strongly recommended whenever 10 GB to 12 GB of VRAM is available. If memory is constrained, FP8 or INT8 retains full decision fidelity while halving memory footprint. Using a 4-bit quantization format (such as Q4_K_M) is appropriate when running alongside other memory-intensive models on a shared 8 GB or 12 GB GPU, but classification probabilities near ambiguous boundaries should be benchmarked against ground truth validation sets.

Traditional tokens per second metrics do not apply to Decision-2.0-Nox-4B because it is not an auto-regressive generator. Instead, performance is evaluated by end-to-end forward-pass execution latency:

• Single GPU (RTX 4090 / A100): Median latency of 12.9 ms for single-question requests on short contexts; 20 ms to 45 ms when handling complex multi-thousand token contexts.
• Mid-Tier GPU (RTX 3060 / 4060): 25 ms to 50 ms per decision request.
• Throughput: In high-concurrency production deployments with dynamic batching enabled via vLLM or custom PyTorch batching, Decision-2.0-Nox-4B easily sustains 80 to 120 routed decisions per second on a single card.

The model natively integrates with standard Hugging Face workflows using trust_remote_code=True:

1import json
2import torch
3from transformers import AutoModel
4
5# Load directly onto GPU in native precision
6model = AutoModel.from_pretrained(
7    "vllm-sr/Decision-2.0-Nox-4B", 
8    torch_dtype=torch.bfloat16, 
9    device_map="auto", 
10    trust_remote_code=True
11)
12
13state_input = (
14    "Customer reported an unexpected 403 API response after updating "
15    "their payment profile. They have an enterprise SLA."
16)
17
18questions = {
19    "target_tier": {
20        "type": "choice",
21        "instructions": "Determine support routing level.",
22        "criteria": {
23            "tier_1": "General inquiries and self-service",
24            "tier_2": "Standard bug reports",
25            "enterprise_escalation": "SLA-backed urgent issues"
26        }
27    },
28    "billing_related": {
29        "type": "noul",
30        "instructions": "Is this issue related to billing or payment methods?"
31    },
32    "severity": {
33        "type": "score",
34        "instructions": "Rate urgency level.",
35        "criteria": ["Low", "Moderate", "Critical"]
36    }
37}
38
39result = model.system_one(state=state_input, questions=questions)
40print(json.dumps(result["answers"], indent=2))

In structured classification benchmarks, Decision-2.0-Nox-4B stands out among non-generative routing architectures.

Decider 4B remains a close competitor, holding a slight edge in raw human-labelled transfer tasks (55.5 vs 52.3 Macro-F1). However, Decision-2.0-Nox-4B scores higher on the broader JevArena index (63.6 vs 61.9). More importantly, Nox 4B features native execution support inside the modern vLLM semantic router ecosystem, making deployment simpler if your tech stack already leverages vLLM primitives.

Running a standard generative 3B to 8B model (such as Llama 3.2 3B or Qwen 2.5 7B) prompted to produce JSON classification requires emitting 30 to 80 output tokens. Even at 100 tokens per second, that introduces a 300 ms to 800 ms latency tax per check, along with JSON formatting failure risks. Nox 4B solves the exact same classification task in 12.9 ms with mathematically guaranteed schema alignment, making it the superior architectural choice for high-throughput routing pipelines.