K2-Horizon-MoVA-36B-A4B

K2-Horizon-MoVA-36B-A4B is a sparse mixture-of-experts model from the Institute of Foundation Models, released under Apache 2.0. It stores 36B parameters and activates roughly 4B per token, which puts it in a category that barely existed two years ago: models that fit on a single high-end workstation while producing output quality that open dense models needed three to four times the parameter count to reach. The attention design is not standard multi-head attention. IFM uses Mixture-of-Values attention (MoVA), which is the architectural detail that separates this checkpoint from the rest of the 30B-class field.

The pitch is narrow and specific. This is not a general-purpose chat model competing on vibes. IFM trained it for agentic tool use, coding, and reasoning, and the published numbers back that up. On tau3-Banking agentic tool use it scores 26.8, ahead of Nemotron 3 Ultra (550B total, 55B active) at 14.2 and Muse Glimmer-30B at 23.5. On Terminal-Bench 2.1 it posts 58.6 against 53.9 for Nemotron 3 Ultra and 44.9 for Qwen3.6-35B-A3B. GPQA Diamond lands at 80.8, within a few points of dense models twice its active footprint.

The 524,288-token native context is the other headline. That is 512K, available from the midtraining stages onward rather than bolted on with a rope-scaling patch after the fact. For anyone building document ingestion, repo-scale code analysis, or long-horizon agent loops, that changes what is feasible on local hardware.

The MoE design is the reason this model is interesting for local deployment, and also the reason its VRAM math confuses people.

• Total parameters: 36B
• Active parameters per token: 4B
• Architecture: Mixture-of-Experts with Mixture-of-Values attention (MoVA)
• Context length: 524,288 tokens native
• Modality: text-only
• License: Apache 2.0

Active parameters govern compute, not memory. Every one of the 36B weights must be resident in VRAM or system RAM because the router can select any expert on any token. What you save is arithmetic: the model does roughly the work of a 4B dense network per forward pass. In practice that means decode throughput closer to a 4B model than a 36B one, while the memory footprint stays at 36B.

MoVA replaces the standard attention value projection with a mixture over value subspaces. IFM has not published the full routing detail in the model card, but the practical effect claimed is better quality per active parameter at long context, which is where attention variants usually fall apart.

That 512K window is not free. KV cache scales linearly with context, and at 524K tokens a full-precision cache will exhaust any consumer card long before the weights do. Budget for quantized KV cache (q8_0 or q4_0 in llama.cpp terms) or cap the working context at 32K-128K for most workloads and reserve the full window for batch jobs.

The declared capabilities are chat, code, reasoning, function-calling, and math. Where it earns its keep:

• Agentic tool use. The tau3-Banking result is the strongest signal. If you are wiring an agent to a set of function schemas and need reliable argument generation across multi-step calls, this checkpoint is built for that loop.
• Terminal and shell automation. Terminal-Bench 2.1 at 58.6 means it handles command synthesis, error recovery, and environment navigation better than most open models in its class.
• Repository-scale code work. The 512K context lets you hold a meaningful slice of a codebase in a single prompt instead of relying on retrieval. Useful for refactors, cross-file bug hunts, and migration scripts.
• Scientific and quantitative reasoning. SciCode at 38.9 and GPQA Diamond at 80.8 put it in range for research assistance, though Humanity's Last Exam at 25.2 shows it is not a frontier reasoning model.
• Long-document analysis. Contracts, logs, transcripts, and specification documents that exceed what a 32K model can ingest.

It is text-only. No vision, no audio. If your pipeline needs image input, this is not the model.

The weights are on Hugging Face under IFM/K2-Horizon-MoVA-36B-A4B, with intermediate checkpoints tagged (base_final, mid_1_final, mid_4_final) for staged comparison. Training data and code are slated for release.

VRAM requirements by quantization:

• BF16 (unquantized): roughly 72GB of weights. Needs an H100, A100 80GB, or a 96GB+ unified-memory Mac.
• Q8_0: roughly 38GB. Fits a single RTX 5090 (32GB) only with partial offload, or comfortably on a Mac Studio with 64GB+ unified memory.
• Q4_K_M: roughly 20-22GB. This is the sweet spot for most users and runs on a 24GB RTX 4090 or 3090 with a modest context window.
• Q3_K_M: roughly 16-17GB. Fits 16GB cards but you will notice quality loss on reasoning tasks.

Realistic hardware:

• RTX 4090 / 3090 (24GB): Q4_K_M with 16K-32K context. Expect roughly 40-70 tokens/sec decode at short context, dropping as the KV cache grows. This is the most common local configuration.
• RTX 5090 (32GB): Q4_K_M with 64K+ context, or Q5_K_M at shorter context. Faster memory bandwidth helps noticeably at long context.
• Mac Studio M3 Ultra / M4 Max (64GB-128GB): Q8_0 runs comfortably. Prompt processing is slower than a discrete GPU, but decode is respectable and unified memory removes the offload penalty.
• Dual RTX 4090: Q8_0 with tensor parallelism across the pair. Worth it only if you need the quality headroom.

Getting started: the fastest path is Ollama or llama.cpp with a GGUF build. Pull a Q4_K_M quant, set num_ctx to your actual working window rather than the full 524K, and enable KV cache quantization. vLLM and SGLang both have serving recipes in the model's technical appendix if you need throughput or continuous batching.

Quantization guidance: start at Q4_K_M. Move to Q5_K_M or Q8_0 only if your evals show measurable degradation on reasoning tasks. MoE models are generally more robust to aggressive quantization than dense models because routing decisions are less sensitive to small weight perturbations, but the 4B active budget leaves less margin than a 70B model would have.

vs. Qwen3.6-35B-A3B. Nearly identical parameter class (35B total, 3B active) and the closest direct competitor. K2-Horizon leads on agentic benchmarks by a wide margin (tau3-Banking 26.8 vs 9.3, Terminal-Bench 58.6 vs 44.9) and on GPQA Diamond. Qwen3.6 has a larger ecosystem of fine-tunes and better tooling maturity. Choose Qwen if you need community support and broad fine-tune availability; choose K2-Horizon if agentic reliability is the priority.

vs. Gemma 4 31B-it (dense). Gemma is dense, so all 31B parameters compute on every token. It is slower to decode and heavier on memory per unit of quality, but simpler to serve and easier to reason about. K2-Horizon wins on tool use and terminal tasks; Gemma holds its own on scientific reasoning (85.7 on GPQA Diamond vs 80.8). If your workload is single-turn QA rather than agents, the dense model is a defensible pick.

The Apache 2.0 license is the tiebreaker for commercial deployment. No gating, no usage restrictions, no per-seat fees.