StartLux-Decision v1.1 4B Instant

StartLux-Decision v1.1 4B Instant is a dense 4.66B parameter open-weights decision model engineered by StartLux Labs. Built specifically to eliminate the overhead of generative text generation in agentic pipelines, it evaluates complex system states and returns calibrated probability distributions over structured questions in a single forward pass. Rather than autoregressively generating textual explanations or JSON tokens, the model directly scores predefined outcomes, delivering instant categorical determinations for runtime agents, policy engines, and triage workflows.

Released in October 2026 under a CC BY-NC 4.0 license (with an Apache-2.0 inference package), this local AI model with 4.66B parameters occupies a unique niche between standard small language models and heuristic classifiers. On the public subset of Decision Index 0.3, the 4B Instant variant achieves an aggregate score of 67.81, leading competing small-scale decision architectures by wide margins. It is purpose-built for engineers who need deterministic, low-latency branching logic in high-throughput environments where generating verbose chain-of-thought tokens is both too slow and too resource-intensive.

To run StartLux-Decision v1.1 4B Instant locally is to bypass conventional token generation pipelines entirely. Whether parsing system metrics, historical telemetry, or multi-turn agent logs, the model reads the full state alongside up to 26 lettered options per question, outputting normalized confidence scores ready for immediate programmatic routing.

The core architecture is a dense transformer containing 4.66 billion parameters configured for a native 65,536-token context window. This large context window allows developers to feed extensive transaction ledgers, codebases, or state histories directly into prompt memory without aggressive truncation.

Unlike pure causal models that read sequences strictly from left to right, StartLux-Decision v1.1 implements a hybrid attention scheme. One out of every four attention layers uses full bidirectional attention over the prompt segment, while the intervening linear-attention layers maintain standard causality. Declared in the engine configuration as jev_prompt_attention: bidirectional, this allows earlier prompt tokens containing system state data to attend to the target question placed at the end of the context. As a result, the model contextualizes state data through the specific lens of the decision query before arriving at its read-out.

The model evaluates queries using three typed question primitives:

• choice: A categorical selection over multiple predefined options (up to 26 choices, mapped A through Z).
• noul: A binary yes or no judgment.
• score: An ordinal rating calibrated on an integer scale from 2 to 10.

At inference time, the model extracts the logits for the relevant option characters at the final prompt position. These raw logits are divided by specialized, per-type temperatures configured in decision_config.json and normalized via softmax. Because the output is derived strictly from the final sequence position, inference consumes only a single forward pass without allocating memory for an ongoing autoregressive KV cache expansion.

StartLux-Decision v1.1 4B Instant excels at verifiable, rules-driven tasks where decisions depend on fine-grained textual details. The model was trained through a two-stage process: initial full-weight fine-tuning across 2.7 million decision instances, followed by adapter training with bidirectional prompt attention over 1.7 million additional examples.

Key real-world applications include:

• Autonomous Tool Selection and Routing: Instead of relying on a generative model to produce function-call schemas, developers can pass available API endpoints as a choice query. The model selects the correct tool and provides calibrated confidence probabilities to gate execution.
• Permission and Policy Verification: Evaluating user actions against complex enterprise security policies or role-based access control ledgers. Its strong performance on the DecisionPivot evaluation suite demonstrates an ability to detect subtle contrast differences in rules, flags, and permissions.
• Fast Triaging and Guardrails: Acting as a high-speed firewall for downstream 70B+ parameter models. It can tag incoming queries for jailbreak attempts, policy violations, or thematic routing (noul or choice) in under 40 milliseconds.
• Real-Time Simulation and Robotics: Driving low-latency game agents, tactical bots, or hardware state monitors that require continuous evaluation of environmental JSON states at fixed intervals.

Because this is a non-generative decision model, conventional StartLux-Decision v1.1 4B Instant tokens per second metrics do not apply. Instead, StartLux-Decision v1.1 4B Instant performance is measured in forward-pass latency per request. On modern hardware, single-batch forward passes execute in 20 to 50 milliseconds, making it orders of magnitude faster than a traditional decoder emitting text tokens.

StartLux-Decision v1.1 4B Instant VRAM requirements vary based on the quantization level and the context length utilized:

• BF16 (Original Weights): Requires approximately 9.8 GB to 11.5 GB of VRAM. An NVIDIA RTX 4080 (16GB), RTX 4090 (24GB), or an Apple Silicon Mac with 16GB+ unified memory provides ample headroom.
• Q8_0 (8-bit Quantization): Requires approximately 5.8 GB to 6.5 GB of VRAM. Runs comfortably on mid-range consumer cards like the RTX 3060 12GB or RTX 4060 Ti 16GB.
• Q4_K_M (4-bit Quantization): Requires approximately 3.2 GB to 4.0 GB of VRAM. Ideal for edge environments, mobile platforms, or entry-level 6GB/8GB GPUs.

When sizing memory, budget additional VRAM if you plan to saturate the full 65,536-token context limit. At 65k context, activation memory and the initial prompt processing buffer will require an extra 2 GB to 4 GB depending on batch size.

The best quantization for StartLux-Decision v1.1 4B Instant in production is Q4_K_M or MLX 8-bit. Testing confirms that Q4_K_M retains calibration fidelity on Decision Index benchmarks while dropping the model footprint under 4 GB, allowing high batch concurrency on a single GPU.

To run the model locally, standard inference servers require custom engine handling due to the bidirectional prompt attention layer:

1. vLLM / Official Engine: The dedicated startlux_decide package (available via GitHub) runs an HTTP server compatible with the System One /v1/systemone endpoint format, offering native vLLM acceleration.
2. llama.cpp: The official repository maintains patches in patches/llama.cpp to correctly parse jev_prompt_attention: bidirectional. Running unpatched builds will fall back to pure causal attention, which degrades accuracy.
3. MLX: Official 4-bit and 8-bit MLX weights are provided for Apple Silicon users, enabling high-efficiency inference on M-series chips.

For users asking how to run 4.66B model on consumer GPU setups, the best GPU for StartLux-Decision v1.1 4B Instant at edge scale is an RTX 4060 Ti (16GB) or an Apple Mac Mini (M4, 24GB). Both deliver sub-50ms decisions with minimal power consumption.

Understanding where the 4B Instant fits requires comparing it against both specialized decision engines and generic small generative models.

H2O-Lightning-4B is another contender in the compact decision space. While H2O-Lightning-4B relies on standard causal formatting with custom readout prompts, StartLux-Decision leverages bidirectional full-attention layers. This architectural difference allows the 4B Instant model to outperform H2O-Lightning-4B by over 8 points on the public Decision Index 0.3. Furthermore, StartLux natively supports typed question definitions (noul, score, choice) directly out of the box with calibrated temperatures.

FlyMyJev-4B serves similar routing tasks but suffers from strict structural bottlenecks, including a hard cap of 26 options that frequently fails on composite multi-choice evaluations. FlyMyJev also lacks native support for the extended 65k context window found in the StartLux architecture. For practitioners parsing massive JSON state snapshots, StartLux provides significantly more functional headroom.

Standard autoregressive models can perform classification by prompting them to output a single letter or structured JSON. However, this approach is fundamentally inefficient. Standard LLMs suffer from logit bias, require complex prompt engineering, and introduce latency penalties by generating tokens sequentially. In contrast, StartLux-Decision v1.1 4B Instant evaluates the state in a single mathematical pass, providing properly calibrated probability distributions across all candidates without output parsing errors or token hallucination.