Mistral Medium 3.5

Our Take

A workable 128B-parameter dense language model from Mistral AI. Pulls ahead on real GitHub issue solving (SWE-Verified) (78/100), so reach for it when that's the dimension that matters. Newly released, so production-readiness is still being shaken out.

Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.

Mistral Medium 3.5 uses a dense transformer architecture with 128B parameters. Unlike mixture-of-experts (MoE) designs where only a subset of parameters activate per token, this dense model uses all parameters for every forward pass. That means inference speed and VRAM consumption are predictable: you need enough GPU memory to load the full model at a given quantization, and tokens-per-second scales with compute bandwidth.

The context window is 262,144 tokens (effectively 256k). This enables processing entire codebases, full-length books, or multi-turn agentic sessions without truncation. The model also includes a dedicated vision encoder trained from scratch to handle variable image sizes and aspect ratios, accepting images alongside text input.

The architecture supports:

• Configurable reasoning effort via a reasoning_effort parameter ("none" or "high"). When set to "high", the model applies additional test-time compute, boosting performance on math, logic, and agentic tasks.
• Native function calling and JSON output, essential for agentic workflows.
• Multilingual support across 20+ languages including English, French, German, Spanish, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic, Hindi, Bengali, and others.

Recommended sampling parameters from Mistral: temperature 0.7 with reasoning_effort="high", or 0.0–0.7 with reasoning_effort="none"; top_p 0.95 for reasoning, leave as 1.0 otherwise.

Mistral Medium 3.5 excels in four primary areas:

Coding & Agentic Tasks – With a SWE-Bench Verified score of 77.6% (close to Claude Sonnet 4.6’s 79.6%), it can resolve real-world GitHub issues, generate pull requests, and refactor codebases. Its strong function-calling and JSON output support make it suitable for building autonomous coding agents that interact with tools, APIs, and repositories.

Reasoning & Math – Configurable reasoning effort lets you push the model on multi-step problems. For complex agentic runs or math puzzles, setting reasoning_effort="high"" yields better results at the cost of slower generation.

Vision & Multimodal Understanding – The model can analyze images, extract information from diagrams, screenshots, or documents, and answer questions about visual content. This is useful for tasks like UI-to-code conversion, document parsing, or captioning.

General Instruction-Following & Chat – As a unified flagship, it handles open-ended chat, system prompts, and structured instructions reliably. The large context window supports extended conversations and document-grounded Q&A.

Multilingual support covers major European, Asian, and Middle Eastern languages, making it viable for international deployments.

A 128B dense model is demanding — expect substantial hardware requirements. Here’s what you need to know for local inference.

Consumer hardware: The most realistic path is using 4-bit quantization (Q4_K_M) on a pair of RTX 4090 GPUs (48GB total) or a single RTX 6000 Ada (48GB). For a single RTX 4090 (24GB), you’ll need 3-bit quantization and layer offloading to CPU RAM, which significantly slows tokens per second. Apple Silicon users with M4 Max or M4 Ultra (64GB+ unified memory) can run Q4_K_M entirely in memory, but expect lower throughput than NVIDIA.

• RTX 4090 (2x, Q4_K_M): ~4–8 tokens per second, depending on prompt length and context.
• H100 (80GB, Q4_K_M): ~15–25 tokens per second.
• Apple M4 Ultra (192GB, Q4_K_M): ~2–5 tokens per second.

For real-time interaction, dual consumer GPUs or a single H100-class card are recommended. Inference engines: Ollama (via llama.cpp), vLLM, and SGLang all support Mistral Medium 3.5. Ollama is the quickest way to get started — run ollama run mistral-medium-3.5 after downloading the Q4_K_M GGUF.

Quantization degrades raw capability, especially on reasoning and long-context tasks. If you need peak performance, FP16 on multiple H100s is ideal. For most local practitioners, Q4_K_M offers the best balance of quality and feasibility.

vs. Llama 3.1 123B (dense, Meta): Llama 3.1 is a strong general-purpose model with excellent multilingual support, but lacks vision capabilities and native function calling out of the box. Mistral Medium 3.5 matches or exceeds Llama 3.1 on coding benchmarks and provides vision and configurable reasoning. If you need vision or agentic tool use, Mistral is the better choice. If you prioritize pure text-generation quality and have a strong preference for Meta’s ecosystem, Llama 3.1 remains competitive.

vs. Qwen 2.5 72B (dense, Alibaba): Qwen 2.5 72B is smaller (72B vs 128B) and much easier to run on consumer hardware (single 24GB GPU at 4-bit). It scores ~72% on SWE-Bench Verified — lower but still strong. It also supports vision and function calling. Mistral Medium 3.5 outperforms it on coding and reasoning at the cost of requiring significantly more VRAM. Choose Mistral if you have the hardware and need the extra performance; choose Qwen if you need to stay on a single consumer GPU.

vs. Mixtral 8x22B (MoE, Mistral): Mixtral 8x22B uses ~45B active parameters per token, so it runs faster on modest hardware (~2–3 tokens/s on a single 4090 at Q4). However, Mistral Medium 3.5’s dense 128B architecture delivers higher benchmark scores and native vision, making it the more capable model when you have the GPU budget. Mixtral is a pragmatic fallback for limited setups.