Perplexity Decider Flash v1.3 4B

Perplexity Decider Flash v1.3 4B is an open-weight, dense decision and classification model built on top of Qwen3.5-4B and distilled from Perplexity's larger 27B Decider v1.3. Released under the Apache 2.0 license, it departs from conventional generative text models. Instead of streaming text tokens through iterative autoregressive loops, Decider Flash evaluates structured classification questions about an input context in a single forward pass, returning calibrated probabilities, ordinal scores, and verbatim character spans.

This model targets agentic workflows, deterministic routing, and high-throughput evaluation pipelines where using a general-purpose generative LLM introduces unnecessary latency, schema parsing failures, and unpredictable token costs. By leveraging bidirectional attention across shared input states, it handles complex categorical decisions, binary conditions, and multimodal inputs containing both text and images with latency profiles measured in milliseconds.

For engineers evaluating a local AI model with 4B parameters in 2026, Decider Flash v1.3 occupies a distinct niche. It is not built to write essays or generate conversational responses. It functions as an ultra-fast System 1 triage engine designed to sit in front of larger local models, handling tasks like intent detection, input guardrailing, document routing, and extraction without the overhead of prompt-engineered JSON generation.

Decider Flash v1.3 is founded on the dense 4B-parameter Qwen3.5 architecture. While standard autoregressive language models mask future tokens to force unidirectional processing, Perplexity adapted the model with bidirectional attention mechanisms across the input context. This allows every token in the prompt to attend to all other tokens before reading out decision logits.

Instead of generating text characters to represent choices, Decider Flash utilizes custom readout heads attached to the transformer backbone. When passed a query schema, the model evaluates questions against the input state concurrently.

• Parameter Count: 4B dense parameters
• Base Architecture: Finetuned and distilled from Qwen3.5-4B (teacher model: Perplexity Decider v1.3 27B)
• Modality: Multimodal input (accepts raw text, structured JSON objects, and images via Pillow/Torchvision); outputs structured probabilities and span offsets
• Memory Footprint (Unquantized): Approximately 8.5 GiB for BF16 weights
• Context Handling: Designed for long-context ingestion (evaluations scale up to 262k tokens), enabling multi-page document classification and large codebase evaluation without truncating context
• Licensing: Apache 2.0

Because outputs are derived directly from output logits rather than multi-token generation paths, inference time remains virtually invariant to the complexity of the criteria. You eliminate the compute overhead of decoding, sampling, and output parsing.

Decider Flash operates on a request schema defining a shared state (the raw context to analyze) and one or more questions. The model executes four distinct task primitives:

1. Choice (`choice`): Multi-class classification supporting between 1 and 255 discrete options. The model evaluates provided natural language criteria for each label and returns normalized probabilities and a confidence score.
2. Binary Decision (`noul`): Boolean questions returning a direct probability $P(\text{true})$. Practitioners can also extract raw logit margins (true - false) to rank documents or query candidates without probability saturation.
3. Ordinal Scoring (`score`): Maps an input against ordered evaluation scales (from 2 to 255 levels). It calculates an expected level index based on probability mass distribution, which is ideal for grading tone, urgency, or semantic adherence.
4. Span Localization (`locate`): Extractive entity and phrase identification. Instead of generating spans, it returns exact character offsets (start, end, score) for named entity recognition (NER), personal information (PII) detection, and extractive question answering directly from the state.

• Agentic Workflow Routing: Determining which specialized sub-agent or local model should handle an incoming user prompt.
• Multimodal Visual Inspection: Passing an image alongside text to verify visual UI bugs, label diagram elements, or extract metadata from receipts.
• PII Scrubbing and Redaction: Pinpointing precise string offsets of sensitive data using the locate primitive before storing logs or forwarding context to external endpoints.
• Guardrails and Content Moderation: Running low-latency safety evaluations on both incoming prompts and generated model responses to intercept toxic, off-policy, or jailbreak attempts.

To run Perplexity Decider Flash v1.3 4B locally, practitioners need to consider how its architecture interacts with standard local inference software. Because the model relies on custom decision heads and bidirectional attention implemented through the pplx_decider library, it is primarily executed via PyTorch and Transformers rather than drop-in GGUF loaders like Ollama.

Running the model locally requires sufficient memory for the 8.5 GiB weight base, plus KV cache and activation overhead based on your input length.

• Best Consumer GPU: An NVIDIA GeForce RTX 4070 (12GB) or RTX 3060 (12GB) represents the minimum comfortable threshold for running native BF16 weights alongside moderate context windows. For massive context windows or batch processing, an RTX 4090 (24GB) is the best GPU for Perplexity Decider Flash v1.3 4B.
• Apple Silicon: M-series chips handle this model cleanly via PyTorch MPS or unified memory execution. A base Mac with 16GB or 24GB of unified memory (such as an M3 or M4) provides sufficient headroom for the model, system OS, and image processing libraries.
• Multi-tenant / Production Node: For enterprise inference clusters, deploying on an NVIDIA A10G (24GB) or L4 (24GB) offers dedicated acceleration and support for concurrent batched evaluations.

Because Decider Flash v1.3 bypasses autoregressive token generation, calculating traditional "Perplexity Decider Flash v1.3 4B tokens per second" is not applicable. Instead, benchmark Perplexity Decider Flash v1.3 4B performance using task latency. In production benchmarks, single-request decision latency routinely hits roughly 150 ms to 180 ms on modern hardware.

For most local deployments, running in native BF16 delivers the highest classification calibration without quantization-induced logit drift. If you are memory-constrained and evaluating how to run a 4B model on a consumer GPU with 8GB VRAM or less, the best quantization for Perplexity Decider Flash v1.3 4B involves using 8-bit weights (via bitsandbytes or torchao). Extreme 4-bit quantization can compress the logit margins, degrading fine-grained probability calibration on subtle multi-class routing choices.

Understanding where Decider Flash v1.3 fits requires comparing it to standard small generative models repurposed for classification and routing.

Practitioners often attempt to handle classification by prompting general-purpose small instruct models (like Qwen2.5-3B or Llama 3.2 3B) to return structured JSON.

• Reliability and Validation: Generative models frequently fail schema constraints, hallucinate keys, or generate introductory conversational text. Decider Flash guarantees schema-conformant arrays, floats, and integer offsets directly out of the box.
• Latency and Throughput: A 3B instruct model must generate 20 to 100 output tokens to explain or format its JSON response. Even running at 60 tokens per second, generation adds 300 ms to 1500 ms of latency. Decider Flash evaluates multiple classification heads in a single forward pass, returning results in under 200 ms.
• Probability Calibration: Generative models cannot easily provide normalized confidence scores across 50 arbitrary options without complex logit masking. Decider Flash natively returns calibrated probabilities across all specified options.

If your workflow requires synthetic data generation, open-ended reasoning, or conversational dialogue, Decider Flash v1.3 cannot perform those tasks. It is strictly an analytical instrument. However, when your pipeline demands high-speed triage, programmatic content routing, or precise span extraction without cloud API dependencies, Decider Flash v1.3 4B delivers unmatched efficiency for its hardware footprint.