Nimble-v4-max 27B

Nimble-v4-max 27B is a 27.36-billion-parameter dense decision model developed by Bespoke Labs. Built as a full fine-tune of Qwen3.8-27B, the model abandons standard conversational text generation in favor of fast, structured judgment. Rather than producing verbose explanations or multi-step reasoning traces, Nimble processes input text or JSON against up to 64 typed questions simultaneously. It directly outputs calibrated probabilities across categorical options, boolean checks, and ordinal rating scales.

On the public suite of Decision Index 0.3, Nimble-v4-max achieved a top score of 68.46 during Bespoke Labs' benchmark runs on October 12, 2026. This performance puts it well ahead of competing decider architectures, including TypeSafe's Jev 1.13 (57.96), Perplexity Decider v1.1 (62.25), and RSI-Jev v6.1 VL 27B (65.64). For engineers managing production triage, classification, and routing, the model offers a way to replace brittle prompt chains and slow LLM-as-a-judge pipelines with a deterministic local engine.

The model weights are released under a Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0), restricting commercial deployment while providing an open-weight resource for evaluation, internal tooling, and academic research. To run Nimble-v4-max 27B locally at unquantized 16-bit precision, you will need enterprise-tier GPU memory, though quantized representations make local deployment accessible on high-end workstation hardware.

Nimble-v4-max uses a dense transformer architecture totaling 27.36B parameters, derived from the base Qwen3.8-27B model. Rather than functioning as a conventional autoregressive text generator, the model is trained as a "System One" decision engine.

Traditional language models generate decisions by emitting text tokens sequentially, which introduces high inference latency and makes probability extraction difficult. Nimble alters this dynamic:

• Logit-Level Option Scoring: Instead of generating free-form strings, the model evaluates input context alongside a pinned schema, computing normalized logit probabilities directly over candidate tokens mapped to specific answers.
• Custom Attention Pattern: Inference requires Bespoke Labs' custom evaluation pipeline (provided in their reference code repository). The engine pins prompt structure via schema_config.json hashes and evaluates the input with the exact attention masking used during training. Standard text-generation loaders or naive chat templates will produce invalid results.
• Multi-Question Vectorization: A single forward pass can answer up to 64 discrete questions concerning the same context payload, eliminating the redundant prompt prefill overhead typically associated with batch classification tasks.
• Context Handling: While context length limits are not explicitly published by the provider, the model inherits the dense sequence capacity of Qwen3.8, allowing it to digest substantial input documents and dense JSON trees before scoring answers.

Because the model only needs to evaluate logits for targeted answer tokens rather than generating sequential decode steps, inference time is dominated by prefill speed rather than auto-regressive generation.

Nimble-v4-max 27B is engineered for tasks where structured code must act on machine-readable confidence values. Each schema submitted to the model supports three distinct question types:

• Multiple Choice: Categorical classification across defined options (e.g., ticket routing across support queues, department dispatch, or multi-label intent detection).
• Yes/No (Boolean): Binary classification checks (e.g., compliance validation, refund eligibility verification, and content safety moderation).
• Rating Scales: Ordered ordinal levels (e.g., urgency grading from 1 to 5, sentiment scoring, or LLM-as-a-judge evaluation of model responses).

Because the output delivers normalized probabilities rather than raw text, engineering teams can configure precise downstream automation thresholds. For instance, a ticket routing engine can automatically dispatch an issue when the top option exceeds an 85% probability score, while routing borderline decisions (e.g., two choices scoring 45% and 42%) to human review queues. This reliability bypasses the parsing errors, hallucinated keys, and schema drift common when forcing standard generative LLMs to output strict JSON.

Deploying this local AI model with 27.36B parameters in 2026 requires understanding its memory footprint and its non-standard inference workflow.

The unquantized model is distributed in BF16 and consumes approximately 55 GB of VRAM for weights alone. Quantization significantly lowers the barrier for workstation deployment:

• BF16 / FP16 (Raw Weights): Requires ~55 GB of VRAM. A single 80 GB GPU (such as an NVIDIA A100 80GB, H100, or H200) or dual 48 GB workstation cards (RTX 6000 Ada, A6000) are required.
• 8-bit Quantization (Q8_0 / INT8): Requires ~31 GB to 34 GB of VRAM. Runs on a dual RTX 3090/4090 setup or a single 48 GB workstation card.
• 4-bit Quantization (Q4_K_M / INT4): Requires ~17 GB to 19 GB of VRAM. This makes it possible to run on a single 24 GB GPU like the NVIDIA GeForce RTX 3090 or RTX 4090.

When looking for the best GPU for Nimble-v4-max 27B, your choice depends on whether you run quantized or full-precision weights:

• Single Consumer GPU (RTX 3090 / RTX 4090 24GB): The best quantization for Nimble-v4-max 27B on consumer hardware is Q4_K_M or Q5_K_M. This leaves sufficient headroom for context buffers while keeping model weights around 18 GB.
• Dual Consumer GPUs (2x RTX 3090 / 4090): Splitting the model across two 24 GB cards via tensor parallelism or pipeline parallelism allows running 8-bit quantized weights or uncompressed Q6_K with minimal precision loss.
• Apple Silicon (M2/M3/M4 Max and Ultra): Systems configured with 36 GB, 64 GB, or 128 GB of unified memory can run the model locally. A 64 GB Mac Studio or MacBook Pro handles 8-bit weights comfortably, while a 128 GB configuration can execute full BF16 weights entirely in unified memory.

Traditional measurements like Nimble-v4-max 27B tokens per second do not capture how this architecture performs. Because the model scores answer options directly without generating auto-regressive text sequences, throughput is measured in requests per second or decision latency per document:

• On an NVIDIA RTX 4090 using 4-bit weights, evaluating a 2,000-token document across 10 typed questions takes approximately 120ms to 250ms total.
• On an enterprise NVIDIA H100 (BF16), inference latencies routinely drop below 70ms for moderately sized prompts.

For initial testing, developers often look for Ollama as a quick way to get started. Note that while earlier 9B iterations (such as Bespoke-Nimble-9B-v2) received native Ollama library support via ollama pull nimble, running Nimble-v4-max 27B locally requires the official Bespoke Labs inference scripts (found in their GitHub repository and Hugging Face dataset releases). Standard chat interfaces or basic GGUF prompt runners will attempt to generate text tokens instead of extracting single-token option logits, breaking the scoring mechanism.

To evaluate Nimble-v4-max 27B performance, practitioners should benchmark it against both dedicated decider models and standard instruction-tuned base models.

TypeSafe's Jev popularized the fast System One decision approach. However, Jev 1.13 remains a proprietary model with closed weights and an unpublished parameter count. On Decision Index 0.3, Nimble-v4-max outscores Jev 1.13 by 10.50 points (68.46 vs. 57.96). For practitioners who require local, self-hosted execution without cloud telemetry or data leakage, Nimble provides an accessible, fully inspectable open-weight alternative.

Using standard generative LLMs for classification requires techniques like structured outputs, guidance schemas, or chain-of-thought prompting.

• Latency and Resource Use: A standard 70B model generating a 50-token reasoning trace takes 1 to 3 seconds per query and burns significant compute. Nimble-v4-max evaluates up to 64 questions in a single forward pass, finishing in a fraction of that time.
• Calibration: Standard generative models suffer from poor probability calibration. Extracting confidence scores from generated output requires complex logprob parsing that often fails to sum to 1.0. Nimble natively exposes calibrated odds across every declared choice.
• Tradeoffs: Nimble-v4-max cannot synthesize text, explain its reasoning in natural language, or draft responses. It is strictly an evaluation and routing engine. Additionally, its CC BY-NC 4.0 license prevents commercial monetization, whereas standard open models like Apache-2.0 base Qwen3.8 allow commercial deployment.