Bosun v3.1 0.6B

Bosun v3.1 0.6B is a decision model, not a chat model. Hanno Labs built it as a LoRA fine-tune of Qwen/Qwen3-0.6B and replaced the usual autoregressive text head with a readout over 256 learned decision tokens. Hand it a state, an instruction, and a set of labelled candidates, and it returns a normalized probability distribution across those candidates in a single forward pass. No JSON schema, no output parser, no retry loop when the model emits a trailing comma.

That contract is the whole point. Agents and software systems are full of decision points — which team owns this ticket, how urgent is this request, does any retrieved chunk actually apply — and the useful output at those points is a calibrated distribution, not generated prose. Bosun v3.1 occupies the same slot as a cross-encoder reranker or a task-specific classifier, except you program it with a sentence instead of a training run. Change the candidate labels at runtime and the model adapts without retraining.

The model is dense at 0.6B parameters, text-only, English, and licensed Apache-2.0. Hanno Labs ships GGUF builds (F16, Q8_0, Q4_K_M) for local use, and the underlying weights are small enough that VRAM is not the constraint on any modern machine. If you are evaluating a local AI model in the 0.6B parameter class for 2026, the question with Bosun isn't whether your hardware can run it — it's whether your problem maps to a bounded set of choices.

Bosun v3.1 is a dense transformer — no mixture-of-experts, no active-parameter tricks. Every one of the 0.6B parameters participates in every forward pass. At this scale that is the right trade: memory bandwidth, not compute, dominates latency, and dense weights mean predictable, uniform inference with no routing overhead.

The specifics:

• Base model: Qwen/Qwen3-0.6B at pinned revision c1899de289a04d12100db370d81485cdf75e47ca
• Adaptation: PEFT LoRA adapter (base_model_relation: adapter)
• Output head: 256 learned decision tokens — 255 runtime candidate slots plus one null slot
• Decision types: choice, score, noul
• Training data: 130,000 rows (80,000 original plus 50,000 ecosystem rows), three completed epochs, checkpoint at epoch 2 selected by macro-family accuracy
• License: Apache-2.0 (Copyright 2026 Clause Logic Inc.)

The readout is what separates this from a fine-tuned chat model. Candidate answers are assigned to stable presented slots, scored through the decision-token logits, and normalized into a probability distribution. There is no sampling step and no autoregressive decoding of the answer, which is why the model never produces malformed output.

The Transformers class loads the pinned base, the PEFT adapter, the tokenizer, the decision-token embeddings, and the serving contract together — expect to use trust_remote_code=True. One operational detail worth flagging: Bosun requires eager attention with JEV 0.1.1, so don't assume a flash-attention default will work in your stack.

Context length is not published for this release. The Qwen3-0.6B base supports 32,768 tokens, which is the practical ceiling unless Hanno Labs documents otherwise. In practice your state, instructions, and candidate descriptions all share that window — a 255-way classification with rich descriptions per candidate consumes it faster than you'd expect.

The three decision types cover most structured decision points in an agent loop:

• `choice` — pick one option from a defined set. Ticket routing, intent classification, tool selection, triage assignment, category tagging.
• `score` — rate on an ordered scale. Urgency, priority, quality tiers, confidence buckets. Useful anywhere you'd otherwise ask a model for "a number from 1 to 5" and get prose back.
• `noul` — produce a probability that no supplied candidate applies, including the null slot. This is the abstention path, and it's the one most teams skip until it bites them.

Concrete places this earns its keep:

• Support ticket routing. State is the ticket text, candidates are the teams that could own it, and you get a distribution to route on — including a real option to escalate when nothing fits.
• Agent tool selection. Instead of prompting a 7B model to emit a function name as JSON, score the available tools directly and threshold on the distribution.
• Retrieval gating. Before you dump context into a larger model, ask whether any retrieved chunk actually answers the question. The null slot gives you a principled way to say no.
• Policy checks. Encode the rule set as candidates and read the probabilities rather than parsing a verdict out of free text.
• Moderation tiering. Ordered severity scales are exactly what score is for.

What it does not do: open-ended generation, summarization, chat, translation, or code. This is a decision model with a custom readout, and it will not behave like an instruct model no matter how you prompt it.

One practical caution: the quality of the distribution depends on the quality of your candidate set. Crisp, mutually distinct labels produce sharp distributions. Near-duplicate candidates produce flat ones, and a flat distribution across 200 slots tells you very little.

GGUF builds from Hanno Labs (decimal download sizes, not runtime memory):

Runtime figures include the KV cache and activations for short states and are engineering estimates derived from file sizes — Hanno Labs does not publish VRAM requirements. Treat them as planning numbers, not vendor specs.

Nearly everything. VRAM is simply not the constraint at 0.6B parameters.

• Any modern x86 CPU with 2 GB of free RAM handles Q4_K_M without noticing.
• Apple Silicon (M1 through M4 Max) runs F16 comfortably; bandwidth is so far above what this model needs that prefill dominates.
• Consumer GPUs — an RTX 3060 12GB, RTX 4060, or anything newer is massively overspecced. A 4 GB card is sufficient. If someone asks you for the best GPU for Bosun v3.1 0.6B, the honest answer is the one already in your machine; spend the budget on the larger model downstream of this decision instead.

Pick Q4_K_M for most workloads — it's the smallest build and loses little at this parameter count. Move to Q8_0 or F16 if you depend on the absolute values of the probabilities rather than just the argmax. Decision models are calibration-sensitive by definition, so if you quantize, validate the resulting distribution on a held-out set before trusting it in production. This matters more than the raw accuracy delta.

The GGUF files are conversions of the source model, and the model card is explicit that a client must implement Bosun's decision-token readout described in serving.json. Running the GGUF through a standard Ollama or llama.cpp chat path will load the weights and produce meaningless output — you'll be reading base-model behavior, not decisions. Verify this yourself before committing to it in a pipeline.

The supported paths are:

JEV-compatible server (Transformers backend):

1uvx --from 'jev-compatible-server[transformers]' \
2  jev-compatible-server --model bosun-v3.1-0.6b

Then POST decision requests to /v1/systemone with a state and a map of typed questions.

Transformers directly:

1from transformers import AutoModelForCausalLM
2
3model = AutoModelForCausalLM.from_pretrained(
4    "Hanno-Labs/bosun-v3.1-0.6b",
5    trust_remote_code=True,
6    dtype="auto",
7    device_map="auto",
8)
9
10result = model.predict(
11    state={"request": "Customer cannot access their account."},
12    instructions="Choose the best support route.",
13    decision_type="choice",
14    row_id="example-1",
15    candidates=[
16        {"id": "billing", "label": "Billing", "description": "Payment issue"},
17        {"id": "access", "label": "Account access", "description": "Login issue"},
18    ],
19)

Tokens per second is the wrong metric here, because Bosun emits nothing autoregressively. The output is one readout step over 256 logits. What you actually care about is decisions per second, which is prefill-bound:

• RTX 4090, F16: prefill in the thousands of tokens/sec; hundreds of short-state decisions per second.
• M4 Max, F16: same order of magnitude, somewhat slower, still far beyond what most agent loops need.
• Desktop CPU, Q4_K_M: tens of decisions per second on short states — fine for batch routing jobs, too slow if you're gating every token of a live stream.

For most deployments the model is not the bottleneck. The network hop to a hosted API would be.

These share weights — Bosun is a LoRA adapter on the same pinned Qwen3-0.6B base. Qwen3-0.6B is a general instruct model that can do chat, light reasoning, and summarization; Bosun does exactly one thing. If you need text out, take the base model. If you need a calibrated distribution over 255 supplied options with a native abstention path, the base model will fight you: you'll be prompting for JSON, parsing it, handling malformed output, and still have no meaningful probability estimate. The tradeoff is flexibility against reliability, and for routing and scoring work reliability wins.

A task-specific cross-encoder or encoder classifier in the 0.1B–0.3B range — or Hanno Labs' own GLiNER-based decision work — will typically beat a general decision model on a single fixed task with a large labelled set, and it will be cheaper to serve. Choose the encoder when the task is stable, labels are frozen, and you have training data. Choose Bosun when you have many decision types, change candidates at runtime, or want to skip the training pipeline entirely. That runtime reprogrammability is the differentiator.

Gemma 3 270M sits in the same size class and is a fine-tuning target rather than a decision API. It generates text. If your pipeline needs the model to write something, that class of model is the right pick. If your pipeline needs a decision, Bosun gives you the distribution directly without a parsing layer between you and the result.