North Mini Code

North Mini Code is Cohere’s first model built specifically for developers who need to run agentic coding workloads locally or in private infrastructure. At 30B total parameters with only 3B active per token, it’s a sparse Mixture-of-Experts (MoE) model that delivers coding performance competitive with much larger dense models while keeping inference cost low. Released under Apache 2.0, the weights are fully open and available on Hugging Face in bf16, FP8, and W4A16 quantizations.

This model targets a specific gap: local code agents that can reason about software engineering tasks, orchestrate sub-agents, and handle 256K-token contexts without relying on cloud APIs. It scores 67.6 on SWE-Bench Verified, 40.2 on SWE-Bench Pro, and 36 on Terminal-Bench v2. On the Artificial Analysis Coding Index, it posts 33.4 — competitive with Qwen3.6 35B A3B and ahead of Devstral Small 2, Gemma 4, and even larger models like Mistral Small 4.

North Mini Code is not a general-purpose chatbot. It is a coding agent engine.

The model uses a MoE architecture with 128 experts, of which 8 are activated per token. The 30B total parameters are sparsely distributed — only 3B participate in any forward pass. This has two practical consequences:

• VRAM requirements are lower than a 30B dense model. You still need to load all parameters into memory, but because the compute graph is sparse, you can use aggressively quantized variants (e.g., W4A16) to fit on consumer GPUs.
• Inference throughput is high. Cohere reports up to 2.8x higher output throughput than Devstral Small 2 under identical concurrency, and a 30% advantage in inter-token latency. In practice, that means faster iteration on agent rollouts.

Context length is 256K tokens total, with a maximum generation of 64K tokens. That’s enough to dump an entire codebase into context and still have headroom for multi-turn debugging and patch generation. The architecture pair 30B total / 3B active is a deliberate tradeoff: you get the capacity of a 30B model (knowledge of libraries, language idioms, reasoning patterns) while paying the compute cost of a 3B model only at inference time.

The model is text-only. No vision, no audio. It’s optimized for code generation, function-calling, and structured reasoning.

Cohere recommends running on a single H100 at FP8 (minimum), with FP4 as a lighter option. The weights are available in bf16, FP8, and W4A16 formats on Hugging Face.

North Mini Code excels at tasks that require tool use, long-context reasoning, and multi-step code changes. Its primary strengths:

• Agentic software engineering. It can interpret bug reports, navigate repository structures, write patches, and run tests. The SWE-Bench scores reflect real-world pull-request resolution, not toy problems.
• Terminal-based agents. Terminal-Bench v2 and Terminal-Bench Hard measure the ability to chain shell commands, read logs, install dependencies, and recover from errors. North Mini Code scores 36 on v2 and 15.7 on Hard.
• Complex code generation. LiveCodeBench v6 and SciCode test the model’s grasp of algorithms, data structures, and scientific computing. It handles these without needing to invoke external tools.
• Function-calling and instruction following. The model supports structured tool definitions and can route between tool calls and open-ended responses. This makes it suitable for frameworks like LangChain, CrewAI, or custom agent harnesses.

Where it underperforms: non-coding agentic tasks. On GDPval-AA (a generalist agent benchmark) it scores 14%, and on τ²-Bench Telecom it scores 37%. If your workload is purely conversational or domain-specific outside software engineering, this isn’t the best model for the job.

For developers building code agents — PR reviewers, bug-fixing bots, CI/CD assistants, or autonomous refactoring tools — North Mini Code is purpose-built.

The key question for practitioners: can I run this on my hardware? The answer depends on quantization.

• RTX 4090 (24 GB). Run the W4A16 variant or a GGUF Q4_K_M. Expect 15–30 tokens per second depending on context length and prompt size. This is functional for interactive agent sessions but not real-time.
• RTX 3090 / 3090 Ti (24 GB). Same VRAM but slower memory bandwidth. Expect 10–20 tok/s. Still usable for background agent tasks.
• Apple Silicon (M4 Max with 64 GB unified). Can run the bf16 variant via on-device memory, but inference is slower than an H100. The unified memory model makes it easy to run large contexts without swapping.
• Dual consumer GPUs (e.g., two RTX 3090). With NVLink or tensor parallelism, you can load the FP8 variant and get close to H100-level throughput.

Start with Ollama. The North Mini Code model is available in GGUF format and can be pulled with ollama pull cohere/north-mini-code. This is the fastest way to test on consumer GPUs. For production agent workloads, consider deploying the FP8 version on a single H100 via vLLM or SGLang.

At FP8 on an H100, Cohere reported ~199 output tokens/sec on the Artificial Analysis evaluation suite. On an RTX 4090 with Q4_K_M, you’ll get roughly 10–20 tok/s — slower, but still viable for non-real-time agent loops where latency is less critical than throughput over many calls.

Devstral Small 2 is a dense 24B model from the same agentic coding niche. Cohere’s internal tests show North Mini Code delivers up to 2.8x higher output throughput under identical concurrency, with 30% better inter-token latency. On the Artificial Analysis Coding Index, North Mini Code (33.4) beats Devstral Small 2 (slightly above 25, per Cohere’s figure). If you’re choosing between the two and have a single H100, North Mini Code gives better throughput and coding benchmarks.

Tradeoff: Devstral Small 2 may have better general-purpose reasoning or broader knowledge, since it’s a larger dense model. For agentic coding specifically, North Mini Code is ahead.

Qwen3.6 35B A3B is the same MoE size class (35B total, 3.6B active). It scores 35.2 on the Artificial Analysis Coding Index — slightly above North Mini Code’s 33.4. It also has broad multilingual support and strong general reasoning. North Mini Code is faster (199 tok/s on Cohere’s API vs Qwen3.6’s typical 120–150), and it’s specifically tuned for agentic tool use.

Tradeoff: Choose Qwen3.6 if you need better general coding scores and multilingual coverage. Choose North Mini Code if you need maximum throughput for a single-agent loop, especially on constrained hardware.

• You need an open-weight model that can run locally under Apache 2.0.
• Your workload is agentic software engineering (SWE-Bench style) with long contexts.
• You have a single H100 or a 24 GB consumer GPU and want the best coding/throughput ratio in this size class.
• You want to avoid cloud API vendor lock-in.

This model does not replace a 70B dense coder like DeepSeek-Coder-V2, but it occupies a sweet spot: near-state-of-the-art coding performance at a fraction of the resource cost.