StartLux-Decision v1.1 9B Instant

StartLux-Decision v1.1 9B Instant is an open-weights dense decision model built by StartLux Labs. Rather than generating text autoregressively token by token, the model executes a single forward pass over an input state and returns calibrated probabilities across explicitly typed options. It addresses a persistent problem in production agent workflows: the latency, compute overhead, and parsing fragility of using traditional generative language models for discrete routing, classification, and branching decisions.

Weighing in at 9.65B dense parameters, the model is engineered specifically for fast, deterministic choices across structured options, binary yes-or-no confirmations, or scaled ratings. It scores 69.98 on the public subset of Decision Index 0.3, outperforming general-purpose language models adapted for tool-calling and outranking dedicated decision baselines in its parameter tier. By replacing generative decoding with normalized logits computed directly at the final prompt position, it removes generation overhead entirely.

For practitioners looking to run a local AI model with 9.65B parameters in 2026, StartLux-Decision v1.1 9B Instant occupies a specialized niche. It is not a conversational chatbot or an unstructured text generator. Instead, it serves as a high-throughput, low-latency control plane for autonomous agent architectures, local routing pipelines, and high-frequency policy evaluation running on consumer hardware.

The model uses a dense transformer architecture containing 9.65 billion parameters with a native context window of 65,536 tokens. This expansive context length allows developers to inject complete execution histories, large JSON state trees, structured database schemas, and multi-turn conversational traces into the prompt without truncating historical state.

The key architectural departure from typical causal autoregressive models lies in its attention mechanism and output heads:

• Hybrid Attention Pattern: In full-attention layers (one out of every four layers), the model uses bidirectional attention across prompt tokens. This allows earlier context tokens to attend to later prompt positions, ensuring the input state is evaluated with full visibility of the subsequent question. The linear-attention layers remain causal to preserve memory efficiency. In the configuration, this is exposed as jev_prompt_attention: "bidirectional".
• Single Forward-Pass Readout: The model does not execute an autoregressive generation loop. Instead, the question is evaluated with its labeled target choices. At the final token position, logits representing candidate options are isolated, scaled using per-type temperature scaling parameters defined in its configuration, and normalized via softmax to produce true probabilities.
• Two-Stage Training: Stage 1 fine-tuned the base weights across roughly 2.7 million decision instances. Stage 2 introduced an adapter trained with bidirectional prompt attention across an additional 1.7 million examples before being fully merged into the released base weights.
• Inference Engine Compatibility: Because standard causal engines will yield incorrect logit distributions without bidirectional prompt support, running the model locally requires an engine with custom patch support, such as the official StartLux inference engine, custom patches for llama.cpp, vLLM, or MLX.

StartLux-Decision v1.1 9B Instant handles structured evaluation rather than open-ended prose. It operates on three distinct typed question formats: choice (selecting one target from multiple specified options), noul (a binary yes/no decision), and score (evaluating a level across a discrete numerical scale from 2 to 10).

Concrete production workloads where this architecture excels include:

• High-Throughput Agent Routing: Directing autonomous agent workflows by evaluating the current state against downstream tools. Instead of prompting a generative LLM to output a JSON object containing a tool call, StartLux-Decision evaluates candidate tools in parallel in one forward pass, avoiding syntax errors and malformed output.
• Deterministic Policy and Permission Verification: Evaluating complex ledgers, authorization trees, and rule engines. It tests robustly on the DecisionPivot benchmark, which measures whether a decision correctly flips when a single critical variable or permission boundary in the context changes.
• Real-Time Game and Simulation Control: Making frame-by-frame strategic calls in real-time environments where text generation latency is intolerable. The model can process state snapshots and return directional, tactical, or movement choices inside strict frame-rate budgets.
• Content Moderation and Guardrails: Running classification checks on input and output streams. The 65,536-token context allows scanning entire documents or conversation transcripts against safety criteria with calibrated confidence values rather than opaque token outputs.

Because StartLux-Decision v1.1 9B Instant operates in a single forward pass without autoregressive decoding loops, standard metrics like tokens per second do not capture how it performs. Inference is measured in total request latency per evaluation. On desktop and workstation hardware, evaluating a decision typically takes between 15 and 45 milliseconds depending on prompt length and quantization.

The 9.65B dense parameter footprint dictates memory overhead. Because the model executes one forward pass, KV cache growth during generation is absent, though processing prompts up to 65,536 tokens requires sufficient working memory.

• BF16 (Unquantized): Requires approximately 20 GB to 22 GB of VRAM. Needs an Nvidia RTX 3090, RTX 4090, or professional 24 GB GPU.
• 8-bit Quantization (Q8_0 / MLX 8-bit): Requires roughly 11 GB to 13 GB of VRAM. Fits cleanly on 16 GB GPUs such as the RTX 4080 or Radeon RX 7800 XT.
• 4-bit Quantization (Q4_K_M / MLX 4-bit): Requires roughly 6.5 GB to 8 GB of VRAM. Can run comfortably on 8 GB and 12 GB consumer GPUs like the RTX 4060 Ti, RTX 3060, or Apple Silicon Macs with unified memory.

• Nvidia GPUs: The best GPU for StartLux-Decision v1.1 9B Instant in consumer rigs is an RTX 4090 or RTX 3090 (24 GB VRAM). These cards allow running the model in full BF16 or INT8 with massive context windows up to 64k without offloading to system RAM. For users wanting to know how to run a 9.65B model on mid-range hardware: an RTX 4070 (12 GB) paired with an 8-bit or 4-bit quantization profile will return single-pass decisions in under 30 ms for typical prompt lengths.
• Apple Silicon: On macOS, the model runs natively via MLX. An M3 Max or M4 Max chip with unified memory executes 8-bit and 4-bit models with minimal memory bandwidth bottlenecks, making it well-suited for local agent hosting alongside other tools.
• Best Quantization Target: For most production setups, the Q4_K_M or MLX 4-bit build offers the best balance of speed and precision. Decision calibration degrades only marginally under 4-bit quantization, while latency drops and memory requirements shrink below 8 GB.

To run StartLux-Decision v1.1 9B Instant locally, you must use an engine that respects its bidirectional prompt attention mechanism. Standard out-of-the-box Ollama builds that assume strictly causal generation will load the weights but produce uncalibrated probabilities. To run the model properly:

1. Clone the official repository (StartLuxLabs/StartLux-Decision) or apply the StartLux attention patch to your llama.cpp build.
2. Download the GGUF weights or Hugging Face repository files.
3. Start the local server using the built-in HTTP daemon, which provides an API endpoint compatible with /v1/systemone.
4. Send your payload containing the state and the typed question schema. The response will return exact floats representing option probabilities without streaming overhead.

Evaluating StartLux-Decision v1.1 9B Instant requires comparing it against both general-purpose generative models and alternative routing architectures.

• StartLux-Decision v1.1 9B Instant vs. Llama 3.1 8B (Instruct): Llama 3.1 8B is a standard choice for local agent routing, but using it for decisions requires prompting it to generate structured JSON or a single character token. This incurs generation latency, risks output formatting failures, and suffers from poor probability calibration on edge cases. StartLux-Decision 9B Instant scores significantly higher on Decision Index benchmarks (69.98) and finishes in a single forward pass, delivering 5x to 10x lower latency per routing call. However, Llama 3.1 8B remains necessary if your application requires actual text generation, conversational dialogue, or open-ended reasoning.
• StartLux-Decision v1.1 9B Instant vs. StartLux-Decision v1.1 9B AutoThink: Within the same family, the AutoThink variant introduces a dynamic reasoning loop. When top decision confidence fails to meet a predetermined gate (typically 0.60), AutoThink generates a brief hidden scratchpad before taking a second forward pass. The Instant model skips this mechanism entirely, enforcing zero text generation and fixed, predictable latency. If your workflow involves low-complexity routing or requires hard real-time latency ceilings, choose Instant. If you are dealing with nuanced logic puzzles or complex policy arbitration where latency variance is acceptable, AutoThink is the superior alternative.