Bosun v3.1 1.7B

Bosun v3.1 1.7B is a dense, 2.03B-parameter decision model from Hanno Labs, released under Apache-2.0. It is a LoRA fine-tune of Qwen/Qwen3-1.7B (base model: 2,031,739,904 parameters) that has been retrained around a single job: given a state, a set of criteria, and a list of valid choices, return a calibrated probability distribution over those choices instead of free-form text.

That distinction matters more than the parameter count suggests. Most small models are generalists — you prompt them, they generate, and you hope the answer lands in a parseable shape. Bosun is built around learned decision tokens with stable slots, supporting up to 255 runtime choices plus a null slot, and it exposes the full candidate logits. Applications get P(choice) for every option, not just the argmax. The same decision-token contract covers three question types through the Bosun runtime: choice, score, and noul.

The headline number is on DecisionBench, Hanno Labs' frozen 23,900-row decision suite: 84.90% primary accuracy, 0.050 expected calibration error, and 0.478 mean negative log-likelihood at 100% coverage. For comparison, the model card lists Jev 1.13 at 72.03% accuracy with ECE 0.128, and GPT-5.6 Luna at 69.90% with ECE 0.203. Bosun's calibration gap is the interesting part — at 0.050 ECE, its stated probabilities are close to usable as confidence scores, which is what makes it viable as a routing or gating component rather than a suggestion engine.

This is not a chat model. If you need an assistant, a writer, or a code generator, look elsewhere. If you need a small, fast, locally-runnable component that answers "which of these N options is correct, and how confident is it" with a real distribution, this is one of the few open-weight options purpose-built for that.

Bosun v3.1 1.7B is dense, not Mixture-of-Experts. Every one of its 2.03B parameters is active on every forward pass. For inference that means two things: memory footprint scales linearly with the weight count (no expert offloading tricks), and compute per token is predictable and uniform. At this size, that's a feature — there's no routing overhead, no expert imbalance, and quantization behaves cleanly.

The architecture inherits Qwen3-1.7B's transformer stack and adds decision-slot behavior through LoRA adapters. The output head is the interesting deviation: rather than decoding a text answer, the model produces logits over stable decision slots, which the runtime maps to your supplied candidates. Because the logits are exposed rather than collapsed, downstream code can apply its own thresholds, renormalize, or abstain via the null slot.

Context length is not published on the model card. Practically, this matters less than it would for a generative model — a Bosun call is a single forward pass over a prompt containing state, criteria, and candidate labels, and the output is one decision. Long-context recall isn't the workload. Size your prompts to the task and don't assume the base model's context window carries over as a documented guarantee.

The training data comes from task families represented in the DecisionBench release. Hanno Labs states this explicitly: the reported results are frozen held-out rows from seen task families, and are not evidence of out-of-distribution generalization to novel decision types.

The DecisionBench breakdown shows where the model is strong and where it isn't:

• Binary classification — 92.19% accuracy across 6,388 rows. This is the sweet spot. Yes/no gating, relevance judgments, policy checks, moderation triage.
• Candidate selection — 84.22% across 15,814 rows. Routing a ticket to the right queue, picking a tool from a menu, selecting a retrieval result from a shortlist.
• Ordinal scoring — 63.84% across 1,698 rows. The weak primitive. If your task is "rate this 1–5," expect meaningfully worse results than the other two modes, and consider whether a binary decomposition gets you further.

Concrete uses that fit the profile: agent memory curation (the original motivating problem for the Bosun line — deciding whether a new edge in a knowledge graph is warranted), RAG passage filtering where you need a calibrated score rather than a reranker's raw logit, support-ticket routing with a null escape hatch for "none of these teams," and confidence-gated escalation where low-probability decisions get punted to a human or a larger model.

The null slot deserves emphasis. Being able to return "none of the supplied choices" with a probability attached is what keeps a small model from confidently picking the least-bad option when the real answer isn't in your list.

Official GGUF builds are published at Hanno-Labs/bosun-v3.1-1.7b-GGUF in F16, Q8_0, and Q5_K_M. That's the practical path for local deployment.

KV cache is negligible at the prompt lengths this model sees. The overhead above weight size is runtime and framework allocation, not activations.

• RTX 4090 / 4080 (16 GB) — massively overprovisioned. Run F16 and stop thinking about it.
• RTX 4060 Ti 8 GB, RTX 3060 12 GB — Q8_0 or F16, both comfortable. This is the realistic target tier.
• RTX 3050 6 GB, older 4 GB cards — Q5_K_M fits with room for the runtime.
• Apple M4 Max / M3 Pro (unified memory) — Q8_0 runs well; Metal acceleration handles the prefill efficiently.
• Apple M1/M2 with 8 GB unified — Q5_K_M is the right choice; expect contention with other apps.
• CPU-only, 8–16 GB RAM — viable. A 2.03B dense model at Q5_K_M is genuinely usable on modern x86 without a GPU, just slower.

Q4_K_M is the usual default for local LLMs, but this model doesn't ship one — Q5_K_M is the smallest official build, and at ~1.5 GB it's already cheap enough that there's little reason to go lower. Use Q5_K_M if you're memory-constrained, Q8_0 for the best accuracy-per-byte tradeoff, and F16 if you have the VRAM. Since calibration is the model's core value proposition, avoid aggressive quantization you'd have to build yourself; probability quality is exactly what degrades first.

Don't measure this model in tokens per second. A Bosun call is a single forward pass — there's no autoregressive decode loop. Latency is dominated by prefill over your prompt.

• RTX 4090, Q8_0, ~500-token prompt — expect tens of milliseconds per decision.
• M4 Max, Q8_0, same prompt — roughly 2–4x that.
• CPU-only, Q5_K_M — hundreds of milliseconds to low seconds, depending on core count.

The practical throughput ceiling is how many decisions per second you can batch, not generation speed.

The quickest path is llama.cpp or Ollama against the published GGUF. Note the caveat: a plain chat frontend will show you text, not the probability distribution. To get the full candidate logits and the choice/score/noul contract, use the reference runtime — jev-compatible-server from Hanno Labs serves the model behind an OpenAI-style endpoint (POST /v1/systemone) with a Transformers backend, and the Hugging Face repo includes a Transformers model class that loads the complete package.

vs. Qwen3-1.7B (the base model). Same parameter count, radically different behavior. Qwen3-1.7B is a general instruct model — flexible, conversational, and completely uncalibrated on constrained-choice tasks. You can force it into classification with grammar-constrained decoding, but you get a token, not a probability you should trust. Bosun trades general capability for a narrow, measurable, calibrated decision contract. If your workload is actually generative, use the base model.

vs. Jev 1.13. The closest direct competitor in the decision-model category, and the comparison Hanno Labs publishes themselves: 72.03% primary accuracy and 0.128 ECE against Bosun's 84.90% and 0.050. Roughly 13 points of accuracy and a 2.5x calibration improvement at the same 100% coverage. Jev's advantage would be ecosystem maturity; Bosun's is the numbers.

vs. a 3B–4B general model with constrained decoding. Larger general models can match or beat Bosun on raw accuracy for some tasks, but they cost more VRAM and give you no calibrated distribution without post-hoc work. If you need confidence scores for gating or escalation, that post-hoc calibration is real engineering effort — Bosun ships it.

Choose Bosun when the task is genuinely a decision over an enumerated set of options and you need the probability, not just the answer. Choose something else the moment your task is open-ended.