K2 Horizon 375B-A23B is the flagship of a six-model fleet released by the Institute of Foundation Models (IFM) on September 3, 2026. It is a sparse mixture-of-experts model with 375 billion total parameters and 23 billion active, aimed at coding and agentic work in enterprise deployments. IFM published the weights, training code, data recipes, checkpoints and training logs under the Apache 2.0 license. The fleet also includes 36B-A4B, 32B, 7B, 3.7B and 0.9B models.
A niche 375B-parameter MoE language model from IFM. Sits below the open-model average on our benchmarks; pick it for specific deployment constraints rather than peak quality.
Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.
Benchmark data sourced from: Hugging Face, Arena.ai, Artificial Analysis.
Average benchmark score against active parameters for every text model we track. Models higher up deliver more quality for their size.
The top devices for this model at 4-bit, ranked by fit and speed.
| Device | Grade | Speed | VRAM |
|---|---|---|---|
| AMD Instinct MI355XAMD | SS | 441.3 tok/s | 218.8 GB |
| NVIDIA Rubin GPUNVIDIA | SS | 9708.3 tok/s | 218.8 GB |
| Rubin R100 UltraNVIDIA | SS | 1213.5 tok/s | 218.8 GB |
| TPU 8i MaxGoogle | SS | 948.8 tok/s | 218.8 GB |
| Intel GPU D270 |

Explore the Provider
Aggregate stats, leaderboard, release timeline, and benchmark coverage across every IFM model we track.
| SS |
| 82.7 tok/s |
| 218.8 GB |
See how different quantization levels affect VRAM requirements and quality for this model.
| Format | VRAM Required | Quality | |
|---|---|---|---|
| Q2_K | 140.0 GB | Low | |
| Q4_K_MRecommended | 218.8 GB | Good | |
| Q5_K_M | 256.3 GB | Very Good | |
| Q6_K | 301.3 GB | Excellent | |
| Q8_0 | 395.0 GB | Near Perfect | |
| FP16 | 751.3 GB | Full |
Released on September 3, 2026, by the Institute of Foundation Models (IFM), K2 Horizon 375B-A23B is an open-weights mixture-of-experts model built for enterprise-grade autonomous agents, multi-step reasoning, and software engineering. It anchors IFM's six-model fleet, which scales from 0.9B parameters up to this 375B flagship. IFM released the model under a permissive Apache 2.0 license, publishing not just the model weights, but also the full training code, data recipes, training logs, and intermediate checkpoints.
The architectural profile of K2 Horizon 375B-A23B targets a specific operational bottleneck: delivering frontier-tier reasoning and code synthesis without requiring the compute footprint of a monolithic 400B dense model during generation. By pairing 375 billion total parameters with only 23 billion active parameters per token, it lowers generation latency while retaining the broad factual knowledge and tool-calling competency of massive frontier systems.
For engineers evaluating local deployments, this design defines both the hardware constraints and the execution profile. Hosting this model locally demands significant VRAM to hold the 375B parameters in memory, but once loaded, generation speed tracks closer to a compact 23B model. This balance makes it a compelling candidate for private air-gapped clusters, local agent orchestration, and enterprise environments handling sensitive codebases.
K2 Horizon 375B-A23B uses a sparse Mixture-of-Experts (MoE) design built on 61 transformer layers, a hidden dimension of 6,144, 48 attention heads, and 8 key-value (KV) heads using Grouped-Query Attention (GQA). Compute efficiency hinges on its routing structure:
Understanding K2 Horizon 375B-A23B MoE efficiency requires distinguishing between storage footprint and compute load. All 375 billion parameters must be resident in memory across your GPU cluster or unified memory pool. Every token generated routes through the single shared expert plus the top 8 selected routed experts, resulting in 23B active parameters per forward pass. Memory bandwidth requirements during inference are therefore dictated by loading the active paths rather than the entire 375B network, resulting in high generation throughput when sufficient hardware bandwidth is available.
The native 512K context window enables repository-scale analysis, comprehensive multi-turn agent histories, and large document ingestion without relying on sliding-window approximations or aggressive context compression.
K2 Horizon 375B-A23B targets complex autonomous agent execution rather than simple conversational chit-chat. Its training emphasizes structured outputs, system interaction, and multi-step tool pipelines.
The model achieves high marks on agentic benchmarks such as Toolathlon, tau3-Banking, and MCPMark (testing integration with the Model Context Protocol). It reliably constructs structured function calls, handles schema validations, parses API responses, and course-corrects when encountering external execution errors. The combination of its 512K context window and top-tier agentic routing makes it suitable for running persistent background agents that interact with shell environments, databases, and continuous integration pipelines.
As a flagship local AI model with 375B parameters in 2026, K2 Horizon 375B-A23B is well suited for repository-wide refactoring, test-suite generation, and architectural analysis. Because it maintains stability over hundreds of thousands of context tokens, developers can feed an entire codebase or large dependency trees into the prompt. It produces syntactically strict code across modern languages (Rust, Go, Python, TypeScript, C++) and accurately maintains state when modifying files across sprawling directory structures.
On benchmarks requiring deep step-by-step logic, the model leverages its 23B active capacity to execute structured chain-of-thought derivations. In enterprise settings, this translates into verifiable financial calculation checks, formal verification tasks, logic-puzzle decomposition, and rule-based compliance auditing.
To run K2 Horizon 375B-A23B locally, your storage and memory bus must handle a 375-billion-parameter static footprint. The minimum and recommended configurations vary heavily depending on precision and quantization formats.
Note that operating at high context lengths (128K to 512K tokens) requires additional memory allocations for the KV cache. Even with 8 KV heads via GQA, allocating space for 512K tokens can demand 20 GB to 40 GB of extra VRAM depending on cache quantization (such as FP8 or Q4 KV caches).
Practitioners asking how to run a 375B model on consumer GPUs must confront the memory barrier. A single or dual RTX 4090 setup (24 GB to 48 GB) cannot host this model. To build a local machine capable of running K2 Horizon 375B-A23B, consider these configurations:
The best quantization for K2 Horizon 375B-A23B in production is Q4_K_M. In testing across MoE models, 4-bit medium quantization preserves expert gating accuracy and mathematical logic without the noticeable degradation observed in sub-3-bit quants.
Expected throughput figures for K2 Horizon 375B-A23B tokens per second vary significantly by memory subsystem:
For rapid prototyping without manual CUDA environment tuning, Ollama and llama.cpp provide the fastest paths to execution once appropriate GGUF files are downloaded:
1# If serving via Ollama (configured with custom Modelfile pointing to quant)2ollama run k2-horizon:375b-q4_k_m34# Direct execution via llama.cpp for maximum performance control5./llama-cli \6 -m ./models/k2-horizon-375b-a23b-Q4_K_M.gguf \7 --ctx-size 131072 \8 --n-gpu-layers 62 \9 --threads 16 \10 -p "You are an autonomous coding assistant. Analyze the repository structure provided below."
In multi-GPU environments running production traffic, deploy using vLLM or TensorRT-LLM with expert-parallelism (--pipeline-parallel-size and --tensor-parallel-size) enabled to prevent expert routing stalls.
Evaluating K2 Horizon 375B-A23B against similar large-scale models clarifies where it fits in an engineering stack.
Nemotron 3 Ultra activates 55B parameters per token out of 550B total parameters. K2 Horizon requires roughly 30% less active compute (23B active) while matching or outperforming Nemotron on agentic tool evaluation benchmarks like Toolathlon and tau3-Banking. Nemotron holds an advantage in raw enterprise knowledge extraction across massive conversational contexts, but K2 Horizon delivers higher token generation throughput on equivalent GPU clusters due to its lean 23B active routing footprint.
MiniMax-M3 matches K2 Horizon's 23B active parameter target almost exactly, but spans a slightly larger 428B parameter base. While MiniMax-M3 offers competitive reasoning benchmarks, K2 Horizon provides two distinct advantages for private deployments: a fully open training recipe under the Apache 2.0 license, and a native 512K context window optimized from mid-training. For organizations building complex tool-calling agents requiring transparent data lineage and deep context ingestion, K2 Horizon is the more flexible option.
Energy cost on AMD Instinct MI325X (~331 tok/s, Q4_K_M) vs flagship API pricing.
| Source | Cost per 1M tokens |
|---|---|
Local (energy only)K2 Horizon 375B-A23B on AMD Instinct MI325X · ~331 tok/s · 1000W | $0.101 |
GPT-6.1 TerraOpenAI · in $1.00 · out $5.00 | $2.20 |
Claude Fable 5.2Anthropic · in $8.00 · out $40.00 | $17.60 |
Gemini 4 HydrogenGoogle · in $1.50 · out $7.50 | $3.30 |
Grok 4.5xAI · in $2.00 · out $6.00 | $3.20 |
API prices blended at 70% input / 30% output.
Hardware amortisation not included. Run the full ROI calculator for payback math.
Cheapest current cloud rentals with at least 219 GB VRAM, refreshed hourly.
Advertising disclosure: we earn commissions when you shop through the links below.
| Option | Cost / GPU-hour |
|---|---|
AMD Instinct MI350XDigitalOcean · Spot · 288 GB VRAM | $3 |
AMD Instinct MI325XDigitalOcean · On-Demand · 256 GB VRAM | $3.8 |
AMD Instinct MI355XDigitalOcean · Spot · 288 GB VRAM | $4.5 |
AMD Instinct MI350XRunPod · Secure · 288 GB VRAM | $5.49 |
NVIDIA B300DigitalOcean · Spot · 288 GB VRAM | $6.17 |
Per-GPU rate across RunPod, the Vast.ai marketplace, DigitalOcean, and Vultr.
Spot tier is interruptible. Plan for restarts when comparing against on-demand prices.
Cumulative cost of buying the smallest device in our directory that runs this model at 4-bit, compared with paying flagship APIs for the same token volume.
Assumes 1 million tokens a day, a 70/30 input and output mix and $0.12 per kWh. The device only runs as long as the workload needs. Use the ROI calculator for your own numbers.