
Base Labs released bison-1 on 2026-11-20 as a decision model built on Google's Gemma 4 31B backbone. It is a 31B dense model with a 128k token context window that accepts text and images and returns typed answers for classification, routing and rubric scoring. It ranks first of 19 models on the Artificial Analysis Decision Index v2.1 with a score of 76. Weights are open on Hugging Face and the model is served through Baseten Model APIs.
A workable 31B-parameter dense decision model from Base Labs. Treat the modality benchmarks above as the leading indicator of fit — composite scoring across modalities is still maturing. Newly released, so production-readiness is still being shaken out.
Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.
Benchmark data sourced from: Hugging Face, Arena.ai, Artificial Analysis.
The top devices for this model at 4-bit, ranked by fit and speed.
| Device | Grade | VRAM |
|---|---|---|
| Acer Veriton GN100 AI MiniAcer | SS | 51.0 GB |
| AI TOP ATOM ATAGB10-9003Gigabyte | SS | 51.0 GB |
| AMD Instinct MI300XAMD | SS | 51.0 GB |
| AMD Instinct MI325XAMD | SS | 51.0 GB |
| AMD Instinct MI350PAMD | SS | 51.0 GB |
Cheapest current cloud rentals with at least 51 GB VRAM, refreshed hourly.
Advertising disclosure: we earn commissions when you shop through the links below.
| Option | Cost / GPU-hour |
|---|---|
NVIDIA A100 80GB PCIeVast.ai · Spot · 80 GB VRAM | $0.20 |
NVIDIA A100 80GB SXMVast.ai · Spot · 80 GB VRAM | $0.28 |
NVIDIA A100 80GB SXMVast.ai · On-Demand · 80 GB VRAM | $0.40 |
NVIDIA A100 80GB PCIeVast.ai · On-Demand · 80 GB VRAM | $0.40 |
NVIDIA H100 SXMVast.ai · Spot · 80 GB VRAM | $0.94 |
Per-GPU rate across RunPod, the Vast.ai marketplace, DigitalOcean, and Vultr.
Spot tier is interruptible. Plan for restarts when comparing against on-demand prices.
Base Labs released bison-1 on November 20, 2026, as a specialized 31B dense decision model built on Google's Gemma 4 31B backbone. Rather than generating conversational prose or speculative text, bison-1 is engineered specifically for deterministic classification, request routing, and rubric evaluation. It processes a shared input state alongside multiple target questions, returning calibrated probability distributions for discrete options without wasting compute on auto-regressive token generation.
The model occupies a distinct niche in the local inference ecosystem. Instead of relying on general-purpose chat models constrained by JSON schemas or fragile regex parsers, bison-1 handles classification tasks in a single evaluation pass. It ranks first out of 19 models on the Artificial Analysis Decision Index v2.1 with an aggregate score of 76, leading the index with a 0.14 Brier score. That calibration metric indicates that its output probabilities track actual empirical accuracy far more reliably than standard instruction-tuned language models.
For engineers looking to run bison-1 locally, this 31B parameter model provides a predictable, zero-leakage routing layer for local agent pipelines. With open weights released on Hugging Face, teams can self-host decision and evaluation infrastructure on local workstations, entirely eliminating cloud API costs and network latency.
The foundation of bison-1 is Google's Gemma 4 31B dense architecture, post-trained by Base Labs specifically for single-pass structured scoring. While general-purpose dense language models use causal masking across all layers to predict the next token sequentially, bison-1 employs a block-bidirectional mask on its full-attention layers. Its sliding-window layers remain causal. This hybrid design allows the model to attend symmetrically across all options within a decision block, computing relative probabilities with higher internal consistency.
During training, Base Labs performed a full fine-tune of the core language model weights while keeping the vision encoder, layer norms, and input embeddings frozen. The result is a parameter-efficient decision core that retains input compatibility across text and image tokens while concentrating its updated weights on probabilistic scoring.
Its native context length extends to 131,072 tokens (128k). A 128k context window on a dedicated decision model fundamentally alters local workflow architecture. Rather than chunking long documents and aggregating separate router calls, developers can feed an entire technical specification, a complete customer chat history, or multi-page PDF scans into a single state payload, querying multiple routing and classification checks simultaneously against the same prefix.
Standard text models waste significant inference time generating chain-of-thought reasoning before arriving at a simple classification decision. bison-1 bypasses text decoding entirely. It consumes a state (context, documents, or images) and a dictionary of structured questions, returning typed decisions and confidence values across three core patterns:
Because all questions evaluate against a single shared-prefix prefill, batching multiple classification criteria costs virtually the same compute as asking a single question. This makes bison-1 an ideal local engine for agent gatekeeping, synthetic data filtering, and automated pipeline triage.
Deploying a 31B dense architecture on local hardware demands careful planning around memory bandwidth and VRAM allocation. Unlike mixture-of-experts models that only route tokens through a fraction of their weights per step, a dense 31B model activates all 31 billion parameters on every pass.
Memory consumption breaks down into two components: the base model weights and the key-value (KV) cache for your context. Because bison-1 supports up to 131,072 tokens, pushing context limits will require substantial additional VRAM beyond the base model footprints listed below:
When planning hardware for bison-1, consider these target tiers:
--quantize-kv 1).Evaluating bison-1 tokens per second differs from evaluating traditional models. In standard models, you measure sequential auto-regressive generation speeds (typically 25 to 40 tokens per second for a 31B model on an RTX 4090). Because bison-1 is a decision model that answers from prefill logits without decoding paragraphs of text, overall throughput is dominated by prefill speed (time-to-first-token).
On an RTX 4090 running optimized frameworks like llama.cpp or vLLM, prefill speeds on a 31B model hover between 1,200 and 2,500 tokens per second depending on prompt complexity. A routing call on a 2,000-token prompt completes in under one second, instantly returning probabilities for all configured questions.
The quickest path to get started locally is through llama.cpp or Ollama once quantized GGUF weights are converted from the base Hugging Face repository (baselabs/bison-1). Running via Ollama simply requires importing the GGUF Modelfile:
1FROM ./bison-1-Q4_K_M.gguf2PARAMETER temperature 0.03PARAMETER top_p 1.0
Understanding where bison-1 sits relative to other mid-sized models helps clarify when to deploy it over general-purpose alternatives.
The Gemma 4 31B base model is a general-purpose text generator. If your task requires writing explanations, summarizing text, or chatting with users, Gemma-4-31B is the correct choice. However, if you attempt to use Gemma-4-31B as an internal router or classification engine, you must enforce structured outputs using JSON schemas or prompt engineering. This introduces parsing errors, adds decoding latency, and frequently yields poorly calibrated probabilities. bison-1 discards conversational capabilities in favor of calibrated scoring (0.14 Brier score), making it significantly faster and more reliable for structural pipelines.
Qwen 2.5 32B is one of the strongest general open-weights models in the 30B class, and Llama 3.1 70B remains a standard for heavy reasoning. When used as zero-shot judges, both models demand high-overhead prompt formatting and require 50 to 200 tokens of generation to output an evaluation label. bison-1 matches or beats these larger systems on pure decision accuracy benchmarks while executing in a fraction of the time. You choose bison-1 when the goal is to evaluate, score, or route at minimum latency; you choose Qwen or Llama when you need open-ended generation and iterative reasoning chains.
Base Labs also introduced groundhog-1 alongside bison-1. Groundhog-1 is an architecture variant that utilizes a 26B mixture-of-experts backbone with only ~4B active parameters. For developers constrained to low-end hardware (such as 12GB or 16GB VRAM GPUs) or who require maximum routing throughput, groundhog-1 offers lower latency and reduced memory requirements. However, bison-1 remains the flagship for maximum accuracy, holding the number one rank on the Artificial Analysis Decision Index v2.1 for critical routing tasks where decision errors carry high operational costs.

Explore the Provider
Aggregate stats, leaderboard, release timeline, and benchmark coverage across every Base Labs model we track.

Check which of your devices can run this model and how fast.

Compare hosted per-token prices before you commit to local hardware.

Break-even math for self-hosting, renting a GPU, or paying per token.
Live status for the hosted APIs you might use instead of running this locally.