Nemotron 3 Ultra

Nemotron 3 Ultra is NVIDIA’s largest open-weight language model to date — a 550B-parameter Mixture-of-Experts (MoE) design that activates only 55B parameters per token. It is the final and most capable model in the Nemotron 3 family, built for practitioners who need frontier-level reasoning, long-context analysis, and agentic workflows without relying on cloud APIs.

What sets Nemotron 3 Ultra apart is its hybrid architecture: it interleaves Mamba-2 state-space layers, MoE feed-forward blocks, and selective attention layers. This combination reduces the KV cache footprint and attention cost, yielding up to 5.9× higher inference throughput than comparable open models like GLM-5.1-754B-A40B (8K input / 64K output setting). The model was pre-trained on 20 trillion tokens (cutoff September 2025) and post-trained through May 2026 using SFT, reinforcement learning, and Multi-teacher On-Policy Distillation (MOPD).

Nemotron 3 Ultra competes directly with other large MoE models such as Qwen-3.5-397B-A17B and Kimi-K2.6-1T-A32B. It scores 86.8 on MMLU-Pro, 70.7 on SWE-Bench Verified, 89.0 on LiveCodeBench v6, and 94.7 on RULER at 1M context — numbers that put it at the top of the open-weight leaderboard for both reasoning and long-context tasks. Its Artificial Analysis Intelligence Index of 47.7 reflects strong overall capability across diverse benchmarks.

Nemotron 3 Ultra uses a LatentMoE design — a variant of Mixture-of-Experts where the router learns a compressed latent representation before selecting experts. This improves accuracy per active parameter compared to standard MoE. The hybrid backbone mixes:

• Mamba-2 layers – state-space models that scale linearly with sequence length, reducing memory and compute for long contexts.
• Standard attention layers – retained where full quadratic attention is beneficial for tasks requiring precise token interactions.
• MoE feed-forward blocks – only 55B of the 550B total parameters are active per token, meaning inference speed is closer to a 55B dense model than a 550B one.

The model also includes Multi-Token Prediction (MTP) layers, which act as native speculative decoding by predicting multiple future tokens in parallel. This directly boosts tokens-per-second during autoregressive generation without requiring a separate draft model.

Context length is 1,000,000 tokens — achieved through long-context extension after pre-training. At 1M tokens, Nemotron 3 Ultra outperforms all other open LLMs on the RULER benchmark, making it viable for full-document analysis, massive codebases, or long-running agentic memory.

Precision: The model was pre-trained in NVFP4 (NVIDIA’s 4-bit floating point), and the released checkpoints include BF16 and NVFP4 quantized versions. NVFP4 quantized weights reduce memory footprint by roughly 4× versus BF16 while maintaining near-lossless accuracy on most benchmarks.

Nemotron 3 Ultra is a text-only model with a broad skill set: chat, code generation, reasoning, function-calling, multilingual support, math, and instruction-following. Its strengths align with the following real-world workloads:

• Agentic coding and software engineering – SWE-Bench Verified 70.7 indicates strong ability to resolve real GitHub issues autonomously. The model can write, debug, and refactor code across languages, with a dedicated code pre-training dataset of 173B fresh GitHub tokens (cutoff September 2025).
• Complex multi-step reasoning – MMLU-Pro 86.8 and LiveCodeBench v6 89.0 reflect robust logical deduction and competitive programming. The reasoning mode can be toggled on/off via chat template (enable_thinking=True/False), allowing you to control inference-time compute budget.
• Long-context RAG and document analysis – With 1M token context and high RULER scores, the model can ingest entire technical manuals, legal contracts, or research papers in one pass, then answer questions or extract structured data.
• Tool use and function-calling – Designed for agentic workflows, it can call external APIs, execute code, and manage multi-turn interactions with structured outputs.
• Multilingual reasoning – Supports English, French, Spanish, Italian, German, Japanese, Korean, Hindi, Brazilian Portuguese, Chinese, and Arabic. Performance is strongest in English but holds up in the other languages for reasoning and instruction-following tasks.

This is not a model you run on a single consumer GPU. With 550B total parameters and 55B active, even the NVFP4 quantized version requires substantial hardware.

For practitioners without access to datacenter GPUs, the realistic path is renting multi-GPU instances or using a local cluster. Consumer cards like the RTX 4090 (24GB) cannot hold the model even with aggressive quantization — you would need at least 8× RTX 4090s in a multi-GPU setup to run NVFP4. Apple Silicon users with M4 Max (128GB unified memory) may be able to run the NVFP4 checkpoint using llama.cpp with Metal acceleration, but expect single-digit tokens per second.

The NVFP4 checkpoint is the most practical for local deployment. It requires roughly 140GB of GPU memory (model weights) plus overhead for KV cache and activations. For multi-GPU setups, use tensor parallelism across 4-8 GPUs.

If you need to run on less memory, community GGUF quantizations (Q4_K_M, Q5_K_M) are likely to appear, but as of release only the official NVFP4 and BF16 checkpoints are available. The NVFP4 version offers the best memory-accuracy tradeoff — NVIDIA reports negligible accuracy loss on most benchmarks.

On an 8× H100 (80GB) setup with NVFP4 and tensor parallelism, expect 40-60 tokens per second for short outputs (1K tokens) and 20-30 tokens per second for long generations (64K tokens). The MTP speculative decoding adds a 1.2-1.5× speedup over standard autoregressive decoding.

Ollama is not yet supported for this model size, but the recommended path is using NVIDIA’s Nemotron inference server or vLLM with the official checkpoints. For a quick local test, use the Hugging Face Transformers integration with device_map="auto" and load_in_4bit=True (via bitsandbytes) on a single node with 4-8 GPUs.

Nemotron 3 Ultra sits in a small class of frontier open-weight MoE models. The most relevant comparisons are:

vs Qwen-3.5-397B-A17B (397B total, 17B active):

• Qwen has fewer active parameters (17B vs 55B), so it runs faster on the same hardware but scores lower on reasoning benchmarks (MMLU-Pro ~80 vs 86.8).
• Nemotron’s 1M context window far exceeds Qwen’s 128K, making it the clear choice for long-document tasks.
• Qwen is more widely supported in consumer quantization (GGUF) and can run on a single 48GB GPU (e.g., RTX 6000 Ada) with Q4, whereas Nemotron requires multi-GPU.

vs GLM-5.1-754B-A40B (754B total, 40B active):

• GLM has more total parameters but fewer active per token (40B) and a smaller context (128K). Nemotron achieves 5.9× higher throughput on long outputs.
• Nemotron’s SWE-Bench score (70.7) is significantly higher than GLM’s reported ~60, making it better for autonomous coding agents.
• GLM is Chinese-centric; Nemotron has stronger multilingual support across Western and Asian languages.

When to choose Nemotron 3 Ultra: You need the highest open-weight accuracy for reasoning, coding, and long-context tasks, and you have the hardware to run it (multi-GPU workstation or cluster). It is the best option for agentic systems that must process hundreds of thousands of tokens without cloud round-trips.

When to choose an alternative: If you are limited to a single consumer GPU (24GB VRAM) or need quick deployment with Ollama, look at smaller models like Qwen-3.5-32B or Llama 4 Scout. For production inference at scale, Nemotron’s throughput advantage makes it compelling even on rented GPU instances.