Valen-Decision-1.5-4B

Valen-Decision-1.5-4B is a 4.55B parameter open-weights decision model developed by Valen. Built as a full fine-tune of Qwen3.5-4B, it departs from standard generative text models by functioning as a dedicated "System One" decision engine. Instead of generating autoregressive text tokens sequentially to produce answers, the model computes direct probability distributions across structured candidate labels or numerical scores in a single forward pass. It scores 69.01 on the Jev Decision Index 0.4.1 public text track, establishing itself as a specialized utility for deterministic routing, evaluation, and policy enforcement.

Published under the permissive Apache-2.0 license, Valen-Decision-1.5-4B ships in native Hugging Face Transformers formats, ONNX runtimes, and GGUF binaries. By shifting classification, verification, and grounding workloads away from heavy generative LLMs, this 4.55B model drastically cuts inference latency and compute overhead. For developers building agentic workflows, deterministic API routers, or automated screening pipelines, it fills the gap between rigid rule-based classifiers and unpredictable, token-hungry reasoning models.

When engineers evaluate how to run Valen-Decision-1.5-4B locally, its primary appeal lies in extreme computational efficiency. Because decisions and coordinate bounding boxes execute in a single forward evaluation rather than through multi-token generation loops, the model delivers near-instant inference times while maintaining deep semantic understanding. This makes it a direct, practical competitor to general-purpose compact models like Qwen2.5-3B, Llama 3.2 3B, and Phi-3.5-mini for classification and routing workloads.

Valen-Decision-1.5-4B uses a dense transformer architecture containing 4.55B parameters derived from Qwen3.5-4B. The architecture introduces block-bidirectional attention alongside a dual-head output structure. In this setup, full attention layers utilize a block-structured bidirectional mask (full_attention_mask) while linear attention layers retain causal masking. This hybrid attention mechanism allows the network to evaluate prompt contexts and candidate choices holistically without suffering the directional bias common to pure autoregressive backbones.

The model is distributed with its transformer backbone stored in BF16 precision, while its dedicated decision classification heads and bounding box grounding heads are maintained in FP32 precision. This hybrid precision design prevents numerical instability when computing exact softmax probability distributions over small or large label sets.

Instead of generating text characters that must subsequently be parsed with regular expressions or JSON schemas, Valen-Decision-1.5-4B processes structured query formats:

• Choice Questions: Maps and distributes normalized probabilities across arbitrary candidate sets.
• Predicate (Noul / Yes-No) Questions: Directly evaluates binary truth values for task instructions.
• Score Questions: Assigns calibrated probability mass across ordered evaluation grades.
• Bounding Box (Bbox) Questions: Predicts normalized visual coordinates via its dedicated grounding head when using compatible multimodal runtimes.

Context length handling in this model leverages shared-state execution. A single context or prompt state can be evaluated against multiple subsequent questions without recalculating prefix representations, maximizing hardware throughput during batch ticket processing or tabular row evaluation.

Rather than writing creative prose or functioning as a chat conversationalist, Valen-Decision-1.5-4B excels at deterministic, structured tasks where low latency and well-calibrated confidence scores are mandatory.

The model can take raw customer queries, inbound emails, or support tickets and classify them across dozens of target intents in a single pass. Because it outputs calibrated probabilities, engineering teams can configure clear routing policies: automatically dispatch predictions with over 95% confidence, while routing lower-confidence edge cases to human review.

Production applications running automated content moderation can evaluate text inputs against multi-criteria safety guidelines simultaneously. The model evaluates whether incoming text breaches acceptable use policies, assigns severity scores across predefined violation tiers, and outputs deterministic binary verdicts.

In multi-agent architectures, agents waste substantial latency and token budgets when relying on generative models to choose the next tool or decide branching logic. Valen-Decision-1.5-4B acts as a deterministic decision router, receiving agent state variables and selecting tool pathways or conditional transitions instantly without the risk of syntax hallucination or malformed JSON outputs.

Evaluating Valen-Decision-1.5-4B performance on local workstations is straightforward due to its 4.55B dense parameter footprint. Unlike standard autoregressive generation where time-to-first-token and subsequent generation speeds dictate throughput, Valen-Decision-1.5-4B processes decisions in single evaluation passes. Valen-Decision-1.5-4B tokens per second metrics translate directly into prompt evaluation speed, where it evaluates hundreds of decision queries per second on modest accelerators.

To determine the best GPU for Valen-Decision-1.5-4B, review the required memory footprint across operational precisions:

• FP16 / BF16 (Native Transformers weights): Requires approximately 9.5 GB to 10.5 GB of VRAM. An NVIDIA RTX 3060 12GB, RTX 4070 12GB, or Apple Silicon Mac with 16GB unified memory handles full-precision native inference with sufficient headroom for shared-state caching.
• Q8_0 (8-bit quantization): Requires roughly 5.5 GB to 6.2 GB of VRAM. Compatible with standard 8GB GPUs like the RTX 3070 or RTX 4060.
• Q4_K_M (4-bit medium quantization): Requires approximately 3.2 GB to 3.8 GB of VRAM. This represents the best quantization for Valen-Decision-1.5-4B for general production deployments, offering near-lossless scoring accuracy while fitting into lightweight 6GB mobile GPUs, laptops, or multi-tenant servers.

• Consumer Desktop GPUs: An NVIDIA GeForce RTX 4060 8GB or RTX 4070 delivers sub-20ms latency per decision. If you are exploring how to run 4.55B model on consumer GPU setups alongside other local services, an RTX 3060 12GB or RTX 4090 offers abundant room to host Valen-Decision-1.5-4B concurrently with an embedding model or vector database.
• Apple Silicon: Macs equipped with M2, M3, or M4 chips (including base 16GB models, M4 Pro, and M4 Max) run the GGUF export efficiently via Metal acceleration with virtually no memory pressure.
• CPU Inference: Because single forward passes require minimal arithmetic compared to 500-token text generations, running the ONNX or GGUF format on modern x86 (AVX-512) or ARM CPUs yields production-ready latencies (under 80ms per decision).

To get started quickly, developers can load the official GGUF build via the Valen-patched llama.cpp runtime (v0.2.0+) or integrate the ONNX runtime inside existing Python or C++ applications. While Ollama remains a popular way to launch local generative models, specialized decision runtimes, vLLM plugins, or direct transformers pipelines are recommended to properly ingest candidate options and read the dual-head probability logits directly.

1# Example: Local inference via Transformers
2from transformers import AutoModel, AutoTokenizer
3
4model_id = "Valen-Team/Valen-Decision-1.5-4B"
5tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
6model = AutoModel.from_pretrained(model_id, trust_remote_code=True, torch_dtype="bfloat16").cuda()
7
8prompt = "Ticket: 'My order #8841 arrived with broken glass.' Candidate intents: [Return, Address Change, Shipping Delay, Billing]"
9# Single forward pass computes logits over choices directly

Understanding where Valen-Decision-1.5-4B fits in your stack depends on whether your pipeline requires conversational synthesis or strict decision classification.

Qwen2.5-3B-Instruct is a general-purpose instruction model. When tasked with classification, Qwen2.5-3B must decode output tokens sequentially (for example, generating {"classification": "Return"}). This process introduces parsing overhead, susceptibility to schema violations, and variable inference latency (often 150ms to 400ms depending on token length). Valen-Decision-1.5-4B eliminates autoregressive decoding entirely for classification, evaluating all candidates in a single pass in under 25ms while delivering calibrated probabilities for each candidate class.

DeBERTa-v3-Large (435M parameters) has long served as the industry standard for local zero-shot classification and NLI scoring. However, DeBERTa struggles with complex context understanding, multi-turn dialogue comprehension, and broad world knowledge. With 4.55B parameters inherited from Qwen3.5, Valen-Decision-1.5-4B offers superior semantic reasoning, stronger cross-domain generalizability, and the ability to process long prompts without task-specific fine-tuning.

For developers seeking an efficient local AI model with 4.55B parameters in 2026, Valen-Decision-1.5-4B stands out as a purpose-built utility that strips away generative overhead to maximize speed, reliability, and precision in autonomous decision loops.