
Inkling 2 Mini is an open-weight multimodal model from Thinking Machines Lab, released November 20, 2026 under Apache 2.0. It is a 26.4B parameter Mixture-of-Experts model with 3.4B active parameters whose middle 12 layers run three times per token, giving 5.4B effective active parameters. It takes text and image input with a 1M token context window and is aimed at agentic coding, tool use, chat and retrieval systems. API access through Tinker is priced at $0.10 per million input tokens and $0.40 per million output tokens.
A workable 26.4B-parameter MoE language model from Thinking Machines Lab. A pragmatic middle-ground choice when you need open weights without a flagship-sized footprint. Newly released, so production-readiness is still being shaken out.
Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.
Access model weights, configuration files, and documentation.
Benchmark data sourced from: Hugging Face, Arena.ai, Artificial Analysis.
Average benchmark score against active parameters for every text model we track. Models higher up deliver more quality for their size.
See how different quantization levels affect VRAM requirements and quality for this model.
| Format | VRAM Required | Quality | |
|---|---|---|---|
| Q2_K | 38.8 GB | Low | |
| Q4_K_MRecommended | 44.3 GB | Good | |
| Q5_K_M | 47.0 GB | Very Good | |
| Q6_K | 50.1 GB | Excellent | |
| Q8_0 | 56.7 GB | Near Perfect | |
| FP16 | 81.8 GB | Full |
The top devices for this model at 4-bit, ranked by fit and speed.
| Device | Grade | Speed | VRAM |
|---|---|---|---|
| NVIDIA A100 SXM4 80GBNVIDIA | SS | 53.0 tok/s | 44.3 GB |
| NVIDIA H100 SXM5 80GBNVIDIA | SS | 87.0 tok/s | 44.3 GB |
| Google Cloud TPU v5pGoogle | SS | 71.8 tok/s | 44.3 GB |
| Intel Gaudi 2 AI AcceleratorIntel | SS | 63.6 tok/s | 44.3 GB |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionNVIDIA | SS | 46.8 tok/s | 44.3 GB |
Energy cost on Corsair AI Workstation 300 (Ryzen AI Max 385) (~6.6 tok/s, Q4_K_M) vs flagship API pricing.
| Source | Cost per 1M tokens |
|---|---|
Local (energy only)Inkling 2 Mini on Corsair AI Workstation 300 (Ryzen AI Max 385) · ~6.6 tok/s · 150W | $0.752 |
GPT-6.2 SolOpenAI · in $2.00 · out $10.00 | $4.40 |
Claude Fable 5.2Anthropic · in $8.00 · out $40.00 | $17.60 |
Gemini 4 NeonGoogle · in $3.00 · out $15.00 | $6.60 |
Grok 4.5xAI · in $2.00 · out $6.00 | $3.20 |
API prices blended at 70% input / 30% output.
Hardware amortisation not included. Run the full ROI calculator for payback math.
Cheapest current cloud rentals with at least 44 GB VRAM, refreshed hourly.
Advertising disclosure: we earn commissions when you shop through the links below.
| Option | Cost / GPU-hour |
|---|---|
NVIDIA A100 80GB PCIeVast.ai · Spot · 80 GB VRAM | $0.20 |
NVIDIA A100 80GB SXMVast.ai · Spot · 80 GB VRAM | $0.28 |
NVIDIA RTX A6000Vast.ai · Spot · 48 GB VRAM | $0.31 |
NVIDIA RTX A6000RunPod · Community · 48 GB VRAM | $0.33 |
NVIDIA RTX PRO 5000 BlackwellVast.ai · Spot · 48 GB VRAM | $0.40 |
Per-GPU rate across RunPod, the Vast.ai marketplace, DigitalOcean, and Vultr.
Spot tier is interruptible. Plan for restarts when comparing against on-demand prices.
Inkling 2 Mini is an open-weight multimodal model developed by Thinking Machines Lab and released under the permissive Apache 2.0 license. Built as a 26.4B parameter sparse Mixture of Experts (MoE) network with an atypical looped transformer architecture, it delivers the reasoning and coding performance typically restricted to dense models twice its size. By running its middle layers multiple times per token, Inkling 2 Mini activates just 3.4B physical parameters per forward pass while providing the computational depth of 5.4B effective active parameters.
For developers seeking a local AI model with 26.4B parameters in 2026, Inkling 2 Mini addresses a critical gap in local deployment: frontier-grade agentic reasoning that comfortably fits inside consumer-tier memory budgets. It combines native vision processing, a 1,048,576-token context window, and out-of-the-box support for modern quantizations like NVFP4 and MXFP8. It is engineered specifically for local agentic workflows, long-context document analysis, and autonomous coding without reliance on cloud APIs.
1+-------------------------------------------------------------------+2| Inkling 2 Mini |3| 26.4B Total Parameters | 3.4B Active (5.4B Effective) MoE |4+-------------------------------------------------------------------+5| [Input: Text + Vision (40px - 4096px)] |6| │ |7| [ViT Vision Encoder] ──> Joint Hidden Space Projection |8| │ |9| [24-Layer Decoder-Only Transformer] |10| ├── Layers 1–6: Global/Local Hybrid Attention |11| ├── Layers 7–18: Looped MoE Block (3 passes / token) |12| │ └─ routes 8 of 128 experts per layer |13| └── Layers 19–24: Final Decoder Layers |14| │ |15| [Output: Autoregressive Text | 1,048,576 Context Window] |16+-------------------------------------------------------------------+
The core architecture of Inkling 2 Mini is a 24-layer decoder-only looped transformer. Unlike conventional transformers that pass tokens through a static sequence of layers, Inkling 2 Mini executes its middle 12 layers three consecutive times using shared weights. This design results in 48 layer executions per token from just 24 layers of stored weights. Each recurrent pass through these middle layers adds roughly 1.0B effective active parameters of computational depth, elevating its 3.4B physical active footprint to 5.4B effective active parameters per token.
Its feed-forward blocks utilize a sparse Mixture of Experts layout consisting of 128 total experts per layer, with tokens routed dynamically to 8 experts. This high expert count ensures high specialization across coding, mathematical reasoning, and natural language domains without bloating memory transfer during decoding. The attention mechanism blends local and global attention layers to keep memory overhead manageable across extended contexts.
1 Token Input Vector2 │3 ┌─────────▼─────────┐4 │ Router Network │ (Select top 8 of 128 experts)5 └─────────┬─────────┘6 │7 ┌────┬────┼────┬────┐8 ▼ ▼ ▼ ▼ ▼9 [E12][E34][E55][E89][E102] ... (Top-8 Activated Experts)10 │ │ │ │ │11 └────┴────┼────┴────┘12 │13 ▼14 Accumulated MoE Output
The model is natively multimodal. Rather than using an external adapter bolted onto an existing text checkpoint, Inkling 2 Mini integrates a Vision Transformer (ViT) that encodes visual inputs directly into the joint embedding space. It accepts pixel inputs ranging from 40x40 to 4096x4096 pixels, processing text and image tokens concurrently through the same decoder stack.
The native checkpoint is distributed as a compact ~20GB file using mixed NVFP4 and MXFP8 precisions. This low-precision baseline allows developers to bypass lossy post-training quantizations if their hardware supports modern FP4/FP8 compute kernels.
| Architectural Parameter | Specification |
|---|---|
| Total Parameters | 26.4B |
| Physical Active Parameters | 3.4B |
| Effective Active Parameters | 5.4B (via 3-pass layer looping) |
| Total / Routed Experts | 128 total, top-8 routed per layer |
| Layer Structure | 24 layers total (middle 12 looped 3x = 48 layer passes) |
| Context Length | 1,048,576 tokens |
| Native Quantization | NVFP4 / MXFP8 (~20GB checkpoint) |
| Supported Modalities | Text input, Image input, Text output |
Inkling 2 Mini targets tasks where structured reasoning and agentic execution matter more than raw parameter bloat. The looped layer design provides extended internal deliberation before emitting tokens, significantly reducing runaway chain-of-thought token generation while boosting accuracy on technical benchmarks.
Inkling 2 Mini leads open-weight models under 100B parameters on rigorous coding benchmarks, achieving 48.2% on DeepSWE v1.1. When deployed as a local coding engine in agent frameworks like Aider, Cline, or OpenHands, it reads entire repositories within its context, constructs multi-file diffs, and follows strict tool-calling conventions without drifting off-task.
Because vision is native to the transformer stack, the model excels at transcribing complex architectural diagrams, parsing multi-column PDF layouts, and translating UI wireframes directly into functional React, Vue, or Tailwind code. Image dimensions up to 4096px ensure detailed schematics retain legibility during tokenization.
Inkling 2 Mini is fine-tuned for high-reliability structured outputs (JSON, YAML, and executable bash scripts). In local agent systems requiring multi-step tool execution, it systematically parses schema definitions, calls external APIs, and processes the returned data without dropping arguments.
The 1,048,576-token context window allows developers to ingest large legal binders, technical documentation sets, or complete codebases in a single prompt. Because its active parameter count is low, attention caching and memory ingestion remain stable during extended context processing compared to 30B+ dense alternatives.
Running a 26.4B parameter model locally typically presents challenges for consumer GPUs. However, Inkling 2 Mini MoE efficiency changes the hardware calculation. Because only 3.4B parameters are actively computing during each forward step, memory bandwidth bottlenecks are minimized, while the entire 26.4B parameter set fits inside a single 24GB or 32GB memory tier.
1Memory Footprint by Hardware Tier:2324GB VRAM (RTX 3090 / 4090)4┌───────────────────────────────┬──────────┐5│ Q4_K_M Model Weights (~16GB) │ KV Cache │ (Up to 32k context)6└───────────────────────────────┴──────────┘7832GB Unified Memory (Apple M-Series / DGX Spark)9┌───────────────────────────────┬─────────────────────┐10│ NVFP4 / MXFP8 (~20GB) │ KV Cache (64k–128k) │11└───────────────────────────────┴─────────────────────┘121364GB+ Workstation / Unified RAM14┌───────────────────────────────┬──────────────────────────────────────┐15│ Unquantized / FP8 (~22GB) │ Deep KV Cache (Full 1M Context Tier) │16└───────────────────────────────┴──────────────────────────────────────┘
To calculate your required memory, factor in both the model weight footprint and the KV cache required for your target context length.
| Precision / Quantization | Model Size | Minimum VRAM (8k Context) | Recommended VRAM (32k+ Context) |
|---|---|---|---|
| NVFP4 / Q4_K_M | ~16 GB | 20 GB | 24 GB |
| MXFP8 / Q8_0 | ~22 GB | 26 GB | 32 GB |
| BF16 / FP16 | ~52 GB | 58 GB | 64 GB+ (Dual GPU / Mac) |
For general local use, Q4_K_M via llama.cpp or GGUF provides the ideal balance between output fidelity and memory headroom on 24GB GPUs. If you are operating modern Ada Lovelace (RTX 40-series) or Blackwell hardware, running the native NVFP4 or MXFP8 checkpoints via SGLang or vLLM yields superior token generation rates and avoids standard quantization degradation.
Inference speed is influenced by the looped middle layers. Even though active memory reads correspond to 3.4B parameters, the arithmetic operations equal a 5.4B parameter forward pass per token.
The fastest path to running Inkling 2 Mini locally on macOS, Linux, or Windows is through Ollama:
1# Pull and run the standard 4-bit quantized build2ollama run inkling-2-mini34# Run with an image input directly via CLI5ollama run inkling-2-mini "Describe this UI schematic and write the HTML/Tailwind skeleton for it" -i ./mockup.png
For production agent setups, deploy the model through vLLM or SGLang for optimal batched throughput:
1# Launching via SGLang with native FP8 support2python3 -m sglang.launch_server \3 --model-path thinkingmachines/Inkling-2-Mini \4 --port 30000 \5 --mem-fraction-static 0.85
Evaluating Inkling 2 Mini against existing models in the 20B to 35B parameter class clarifies where its strengths and architectural tradeoffs lie.
1 Reasoning vs Memory Tradeoff2 High ▲3 │ [Inkling 2 Mini]4 │ (26.4B MoE / 3.4B active)5R │ High reasoning depth via loops6E │ Fits in 24GB VRAM7A │8S │ [Qwen2.5-Coder-32B]9O │ (Dense 32.5B)10N │ High raw knowledge, needs 32GB+ VRAM11I │12N │ [Mixtral 8x7B]13G │ (46.7B MoE / 12.9B active)14 │ High VRAM demand (~32GB+), older architecture15 Low └──────────────────────────────────────────────►16 Low VRAM Requirement High
Qwen2.5-Coder-32B is one of the premier open-weights coding engines, but as a dense 32.5B parameter network, it presents steep hardware hurdles. Qwen requires roughly 20GB of VRAM just to store 4-bit weights, leaving almost no buffer for context or KV caching on a standard 24GB RTX 4090. Inkling 2 Mini activates only 3.4B parameters per token from a 26.4B pool, executing faster during generation while matching or exceeding Qwen on multi-step reasoning benchmarks. Furthermore, Inkling 2 Mini features native vision, which Qwen2.5-Coder lacks without pairing it with an external visual encoder.
Mixtral 8x7B popularized open-source sparse MoE designs, routing tokens across 8 experts for an active parameter footprint of 12.9B out of 46.7B total. However, hosting Mixtral requires keeping nearly 47B parameters in memory, demanding at least 32GB of VRAM for 4-bit quantization and pushing it out of reach for single consumer GPUs. Inkling 2 Mini cuts total memory consumption in half (26.4B total) and requires only 3.4B active weights per pass. Its recurrent looping allows it to outperform Mixtral on programming tasks and agentic benchmarks while operating comfortably within a single 24GB GPU or 32GB Apple Silicon setup.
Choose Inkling 2 Mini over dense 30B models when you need local vision capabilities, low memory footprint, agentic function-calling, and reasoning depth that fits within a single consumer graphics card.

Explore the Provider
Aggregate stats, leaderboard, release timeline, and benchmark coverage across every Thinking Machines Lab model we track.

Check which of your devices can run this model and how fast.

Compare hosted per-token prices before you commit to local hardware.

Break-even math for self-hosting, renting a GPU, or paying per token.
Live status for the hosted APIs you might use instead of running this locally.