Torchcast Decision 27B

Torchcast Decision 27B is an open-weight, 27.8B parameter dense decision engine built specifically for structured probabilistic evaluation rather than standard discursive text generation. Released on October 3, 2026, by Torchcast AI, the model is a LoRA fine-tune merged into StartLux-Decision-27B, an architecture derived from Alibaba's Qwen3.8-27B foundation. Instead of returning free-form text completions, it evaluates complex system states and returns calibrated probability distributions over defined choices, binary outcomes, or ordinal ratings via a TypeSafe-compatible /v1/systemone interface.

With a native context window of 262,144 tokens, this local AI model 27.8B parameters 2026 release addresses a specific engineering bottleneck: giving autonomous agents fast, deterministic decision loops without relying on brittle prompt-engineering tricks or external API calls. On the Decision Index 0.2.1 benchmark suite, Torchcast Decision 27B scores 65.00, outperforming TypeSafe's Jev 1.13 by 7.09 points while executing single binary evaluations in 33 milliseconds on an NVIDIA H100. For engineers deploying autonomous routing, safety guardrails, or state verification pipelines locally, it provides a purpose-built alternative to general-purpose chat models.

The model is distributed under a CC-BY-NC-4.0 license, meaning self-hosted non-commercial research and local internal testing are unrestricted, but commercial production deployments require commercial agreements with both Torchcast AI and StartLux. The reference inference and evaluation codebase is licensed under Apache-2.0.

Torchcast Decision 27B operates as a dense 27.8B parameter transformer. Unlike mixture-of-experts (MoE) architectures that route active tokens across sparse parameter sub-networks, this dense design routes every token through the full parameter set. While dense execution increases compute intensity during the prefill phase, it guarantees consistent latency profiles and predictable memory bandwidth requirements across large context batches.

The model strips out the Multi-Token Prediction (MTP) head present in some base architectures, streamlining the output path to focus purely on scoring discrete options. It evaluates three specific question schemas:

• choice: Selects among a discrete set of categorical possibilities.
• noul: Answers binary yes/no assertions.
• score: Produces a probability spread over an ordinal rating scale.

A critical design advantage is single-pass execution. For typical requests containing multiple state questions, Torchcast Decision 27B evaluates all queries simultaneously in one forward pass rather than generating autoregressive tokens one by one.

Supporting an expansive context length of 262,144 tokens requires significant attention to memory scaling. When processing large state representations, full context processing demands FlashAttention, Flash Linear Attention, or causal-conv1d kernels to keep KV cache footprint and compute manageable. The official inference stack relies on flash-linear-attention and causal-conv1d packages to maintain execution speed.

Rather than generating descriptive text, Torchcast Decision 27B excels at decision calibration: assigning mathematically dependable probabilities to distinct options based on unstructured or structured evidence.

• Autonomous Agent Tool Routing: Evaluating whether an agent should call a specific API, continue planning, or surface an alert to a human operator. The model handles tool invocation checks, appliance simulation, and interactive state transitions with high precision.
• Contract and Policy Auditing: Using the 262K context window to ingest entire service-level agreements, legal disclosures, or codebase diffs, then scoring whether the document complies with a defined regulatory rule or passes a binary safety check.
• Aspect-Based Sentiment and Stance Analysis: Outperforming generalist models on structured stance detection (such as the VAST benchmark) and aspect-sentiment extraction (ACOS), making it suitable for high-throughput filtering of feeds and user tickets.
• Verification and Fact Checking: Assessing claims against supplied reference documentation and outputting a confidence distribution rather than a hallucinated narrative.

Torchcast Decision 27B is not designed for open-ended creative writing, conversational dialogue, or deep multi-step mathematical derivations. On deep reasoning and academic knowledge benchmarks like GPQA Diamond and MMLU-Pro, it trails knowledge-dense generalist models. It is designed to act as an analytical judge over provided context, not an encyclopedic retrieval engine.

To run Torchcast Decision 27B locally, practitioners need to plan around the sheer memory requirements of a 27.8B dense parameter network, particularly when scaling context.

In unquantized BF16 precision, the model weights alone consume 54.7 GB of storage and memory. Running native BF16 inference alongside a deep context buffer requires a minimum of 64 GB to 80 GB of VRAM.

• BF16 (Original Precision): Requires an 80GB GPU such as an NVIDIA A100 80GB or H100 80GB PCIe/SXM.
• FP8 Precision: Requires roughly 30 GB to 34 GB of VRAM. Achievable on dual 24GB GPUs (e.g., 2x RTX 3090 or 2x RTX 4090) or a single professional workstation card like the RTX 6000 Ada (48GB).
• 4-bit Quantization (Q4_K_M / AWQ / EXL2): Requires approximately 16 GB to 18 GB of VRAM for weights, leaving headroom on a single 24GB consumer GPU for dynamic KV cache allocation at moderate context lengths.

For local practitioner rigs, 4-bit precision (such as Q4_K_M via llama.cpp or 4-bit AWQ) offers the best balance of speed, footprint, and fidelity. While extreme sub-4-bit quantizations (such as 2-bit or 3-bit) often distort probability tails and harm decision calibration, 4-bit implementations retain over 98 percent of baseline classification accuracy.

If your rig can support it, FP8 is the sweet spot for preserving calibrated probability distributions across long document inputs.

• Single Consumer GPU (Budget Local): An NVIDIA GeForce RTX 3090 24GB or RTX 4090 24GB running a 4-bit quantized version. This is the baseline answer for how to run 27.8B model on consumer GPU setups.
• Unified Memory Mac: Apple Silicon configurations (M3 Max, M4 Max, or M2/M3 Ultra) configured with 48GB, 64GB, or 128GB of unified memory run this model smoothly using local runtimes without quantization limits.
• Enterprise / Workstation Rig: A single NVIDIA RTX 6000 Ada (48GB) for FP8, or an NVIDIA H100/A100 (80GB) for native BF16 execution.

Because Torchcast Decision 27B outputs probability distributions over pre-allocated candidate tokens rather than streaming hundreds of text tokens, token generation metrics differ from standard LLMs. What matters is prefill speed and decision latency:

• On an H100 80GB (BF16), single binary questions resolve with an end-to-end latency of ~33 ms, while multi-question batches run in roughly 49 ms.
• On an RTX 4090 (4-bit quant), expect prompt ingestion rates between 60 and 110 tokens per second for prefill, with decision resolutions returning in 60 to 120 ms for short contexts.

The quickest way to run the native server locally is through the official torchcast_decision package:

1# Clone weights and setup environment
2hf download torchcast-ai/torchcast-decision-27b --local-dir torchcast-decision-27b
3cd torchcast-decision-27b
4python3 -m venv .venv && source .venv/bin/activate
5pip install -r requirements.txt
6
7# Verify acceleration kernels
8python -m torchcast_decision.check .
9
10# Launch local TypeSafe-compatible server
11python -m torchcast_decision.server --model . --port 8090

Once running, query the endpoint directly:

1curl -s localhost:8090/v1/systemone \
2  -H 'Content-Type: application/json' \
3  -d '{
4    "state": "System memory usage is at 94% with 400MB swap remaining.",
5    "questions": [
6      {"id": "q1", "type": "noul", "text": "Is memory exhaustion imminent?"}
7    ]
8  }'

For environments utilizing Ollama or standard GGUF tools, community conversion scripts can expose the underlying base architecture, though full compatibility with the custom /v1/systemone logit extraction heads requires running the provided Python runner.

Understanding Torchcast Decision 27B performance requires comparing it directly to relevant decision engines and similarly sized foundation models.

TypeSafe's Jev 1.13 has served as the reference implementation for typed decision serving. Torchcast Decision 27B establishes a notable lead on general decision metrics, scoring 65.00 on Decision Index 0.2.1 compared to Jev 1.13's 57.91. Torchcast leads on 31 of 38 index sub-benchmarks, notably outperforming Jev in stance evaluation, contract logic, and simulated tool calling. However, Jev 1.13 retains an advantage on broad factual retrieval and complex knowledge tasks (leading Torchcast by 5.8 points in the Knowledge index area). Choose Torchcast for agent control flow and local automation; stick with Jev if your decision points rely heavily on open-domain world knowledge.

StartLux-Decision-27B is the direct architectural parent of Torchcast Decision 27B. While StartLux achieves a respectable 63.88 on Decision Index 0.2.1, Torchcast's LoRA fine-tuning and recalibrated head refine probability distributions in edge cases, reducing probability overconfidence by 1.12 index points overall. If choosing between the two for new local deployments, Torchcast Decision 27B offers better probability calibration across diverse prompts.

Deploying a standard open-weight model (such as a generic Qwen or Gemma chat variant) for classification tasks requires structured JSON prompting or logit masking wrappers. These approaches are prone to schema drift, require continuous generation of multi-token reasoning chains to achieve high accuracy, and consume significant compute. Torchcast Decision 27B provides native classification logits in a single forward pass, delivering predictable latency and calibrated output probabilities without conversational overhead.