Perplexity Decider v1.1 27B

Perplexity Decider v1.1 27B is an open-weights dense decision model built by Perplexity, released under the Apache-2.0 license. Rather than operating as a conventional autoregressive text generator that outputs tokens sequentially, Decider v1.1 acts as a scoring and classification engine. It evaluates input state against structured criteria, returning calibrated probability distributions for binary questions, categorical choices, and discrete rubric scores.

Built as a full fine-tune of Qwen3.8-27B, the model achieves an overall score of 61.56 on the Hugging Face Decision Index 0.3 benchmark, surpassing both its predecessor (v1 at 56.4) and comparable architectures like TypeSafe's Jev (57.9). By replacing open-ended text generation with direct logit readouts, Decider v1.1 eliminates parsing errors, JSON schema violations, and output non-determinism in mission-critical routing and evaluation pipelines.

For engineers seeking to run Perplexity Decider v1.1 27B locally, this checkpoint presents distinct architectural traits compared to standard causal models. Local deployment requires accounting for custom readout heads, noncausal attention masks, and substantial VRAM footprints during inference.

Perplexity Decider v1.1 27B uses a dense 27.8B parameter backbone based on Qwen3.8-27B. Unlike standard decoder-only large language models that map hidden states to a massive vocabulary matrix, Decider swaps out the standard language model head (lm_head) for a specialized readout module.

Key architectural characteristics include:

• Custom Decision Head: The model discards the full vocabulary projection in favor of a specialized readout.safetensors head with dimensions [255, 5120] in BF16 format. This head extracts classification logits directly from the final transformer layer.
• Lifted Causal Masking: In standard generative transformers, causal masking prevents tokens from attending to subsequent tokens. In Decider v1.1, full-attention layers operate in a noncausal, bidirectional mode across the input state and question. Linear-attention layers retain their native structure. This bidirectional visibility across the full prompt is the primary factor behind the model's jump in benchmark accuracy.
• Context Length: The architecture supports a context length of 8,192 tokens. Inputs exceeding 8,192 tokens are rejected by the bundled preparation script rather than silently truncated, ensuring the model never evaluates truncated context.
• Inference Runtime Constraints: Standard causal inference engines like vanilla vLLM, llama.cpp, or Hugging Face pipeline("text-generation") cannot reproduce the evaluated results out of the box. Running the model requires either using Perplexity's bundled autojev Python package or exporting the readout rows into specific vocabulary token slots with an engine patched for bidirectional attention.
• Calibration: The model includes a pre-calibrated temperature vector applied directly to the readout logits, ensuring output probabilities accurately reflect true confidence intervals.

Decider v1.1 27B is optimized for high-throughput decision-making where deterministic, numerical outputs are mandatory. Because it does not generate text, it cannot draft responses, write code, or engage in conversation. Instead, it provides high-precision evaluations across three core primitives:

In multi-agent architectures and complex support systems, routing user intent to the correct agent or API tool typically requires parsing a generative LLM's response. Decider v1.1 handles categorical selection natively. A single forward pass yields exact probabilities across predefined categories (such as triage between billing, technical bug reports, or account security), allowing application logic to route traffic based on raw floating-point thresholds.

For automated safety policies, Decider processes inputs against binary policy criteria. Instead of asking a model to output "SAFE" or "UNSAFE" and parsing string tokens, Decider returns a calibrated probability of violation. If the probability exceeds an operational risk tolerance (such as 0.85 for mild violations or 0.50 for severe risks), the system triggers intervention instantly.

Evaluating complex artifacts, such as grading user-written essays, checking code against stylistic constraints, or assessing customer support transcripts, usually requires structured output parsing. Decider natively evaluates input against multi-tiered rubric criteria, computing probabilities across discrete point scales and providing a statistically sound expected score.

Deploying this local AI model with 27.8B parameters in 2026 requires understanding its memory footprint and execution path. Because Decider uses a forward evaluation pass rather than continuous token decoding, hardware demands differ significantly from standard generative LLMs.

In its native BF16 format, the model weights take approximately 49 GiB of storage. Inference requires additional working memory for the bidirectional attention computation across the 8,192-token context window.

For practitioners running on consumer silicon, Q4_K_M (or 4-bit AWQ/EXL2 equivalent) is the best quantization for Perplexity Decider v1.1 27B. At 4-bit precision, the parameter weight footprint drops to around 16 GiB. This fits comfortably inside the 24GB VRAM buffer of an NVIDIA RTX 4090 or RTX 3090, leaving roughly 6 GiB to 8 GiB for the KV cache and working activation buffers at the maximum 8,192 context limit.

INT8 (Q8_0) offers near-zero degradation from BF16, but requires at least 32 GiB of addressable VRAM, which forces users onto multi-GPU systems or professional workstation cards.

• Best GPU for Perplexity Decider v1.1 27B: For single-card deployment at full native precision, the NVIDIA A100 (80GB) or H100 provides the necessary headroom. For local workstation setups, two 24GB consumer GPUs (such as two RTX 3090 or RTX 4090 cards) offer 48GB of pooled memory, though running full BF16 requires offloading a fraction of layers or utilizing an 8-bit quantized checkpoint.
• Consumer Hardware: To answer how to run a 27.8B model on consumer GPU setups: a single RTX 4090 can execute the model efficiently using 4-bit weights. On unified memory systems, an Apple Mac Studio or MacBook Pro equipped with an M3 Max or M4 Max and 64GB or more of RAM can run the native BF16 checkpoint out of the box using PyTorch on the Metal Performance Shaders (MPS) backend.

Evaluating Perplexity Decider v1.1 27B tokens per second requires a different mindset than testing conversational chatbots:

• Autoregressive Generation vs. Single-Pass Scoring: The model does not generate streams of output tokens. It processes the prompt context and evaluates the readout head in a single forward pass.
• Latency: On an RTX 4090 running a 4-bit quant, a classification decision over a 1,000-token input resolves in approximately 80ms to 180ms.
• Prefill Throughput: In high-batch ingestion pipelines, prefill speeds reach 1,200 to 2,500 tokens per second depending on prompt length and GPU memory bandwidth.

Standard inference runners like Ollama require custom configuration for this model. Because Ollama assumes a standard autoregressive model with an lm_head, loading Decider directly produces invalid results unless the checkpoint is exported into an expanded vocabulary mapping and paired with an engine that supports lifting the causal attention mask.

For reliable production serving, deploy using the official bundled runtime:

1. Clone the repository from Hugging Face (perplexity-ai/pplx-decider-v1.1-27b).
2. Add source/src from the repository to your Python path.
3. Initialize the DecisionModel class pointing to the local weight directory on a CUDA or MPS device.
4. Pass input dictionaries containing the state text and the question schema directly to model.predict().

Understanding where Decider v1.1 fits requires comparing it against its dedicated architectural predecessor, TypeSafe's Jev, and conventional generative models like Qwen 2.5 32B.

Both models utilize custom readout heads to bypass generative tokenization. However, Decider v1.1 scores 3.66 points higher on the Decision Index 0.3 benchmark overall. Decider v1.1 demonstrates substantial advantages in the Language (69.45 vs. 62.0) and Retrieval (61.26 vs. 55.4) subcategories, largely due to training on broader task-oriented corpora like tasksource and optimizing noncausal layer transitions. Jev holds a narrow edge in raw Knowledge retrieval (51.4 vs. 48.18), but Decider v1.1 is significantly more reliable across programmatic classification workflows.

A general-purpose generative LLM like Qwen 2.5 32B can perform classification by prompting it to output a JSON object or a single label. However, doing so incurs severe operational tradeoffs:

• Latency and Compute: Qwen must generate 10 to 50 tokens to articulate a structured response, consuming GPU compute for multiple autoregressive steps. Decider v1.1 resolves the decision in a single forward pass.
• Reliability: Generative models can suffer from hallucinations, formatting failures, or probability skew caused by tokenization artifacts. Decider v1.1 outputs pure, normalized mathematical probabilities across candidate classes.
• Tradeoff: Qwen 2.5 32B offers a 128k context window and can synthesize explanations, write code, or adjust to novel free-form instructions. Decider v1.1 cannot generate text and is strictly bounded by its 8,192-token context window.

For engineers building agentic control planes, routing fabrics, and data labeling pipelines, Perplexity Decider v1.1 27B provides a faster, mathematically sound alternative to prompting generative language models.