Nemotron 3.5 Super VL 120B-A12B

NVIDIA released Nemotron 3.5 Super VL 120B-A12B on October 15, 2026, delivering an open multimodal Mixture-of-Experts (MoE) model engineered for autonomous agents, vision-language reasoning, and deep document analysis. By pairing a 120-billion-parameter total capacity with a sparse routing mechanism that activates just 12 billion parameters per token, NVIDIA has created an architecture designed to solve the throughput bottlenecks typical of large vision-language models. It operates with the generation speed of a compact 12B model while maintaining the vast world knowledge and contextual comprehension of a triple-digit frontier system.

Targeted at developers building private pipelines and agentic frameworks, the model handles text, high-resolution document imagery, and sampled video frames across a massive 1,048,576-token context window. Licensed under OpenMDW-1.1 for both commercial and research use, it enters the ecosystem as a premier local AI model 120B parameters 2026 release. It directly challenges dense multimodal flagships like Qwen2.5-VL 72B and Llama 3.2 90B Vision by trading higher overall VRAM capacity demands for significantly lower compute latency per generated token.

For practitioners looking to run Nemotron 3.5 Super VL 120B-A12B locally, the model presents a distinct hardware equation. You must provide enough memory to seat the full 120B weight matrix, but your compute cores only process 12B active parameters during forward passes. Once the static VRAM requirement is met, the generation phase runs at exceptional speeds on workstation hardware.

Nemotron 3.5 Super VL 120B-A12B departs from standard transformer designs by implementing a hybrid backbone. It combines interleaved Mamba-2 state space layers, standard multi-head attention layers, and an ultra-fine MoE structure featuring 512 routed experts per MoE layer. Visual processing is handled by a dedicated C-RADIO vision encoder linked to the hybrid core via a vision-language connector, mapping text strings, images, and temporal video frames into a shared token space.

The architectural mechanics deliver three key deployment characteristics:

• Sparse Execution Dynamics: The 12B active parameter budget means that memory bandwidth and FLOP demands during autoregressive decoding scale like a small model. This Nemotron 3.5 Super VL 120B-A12B MoE efficiency allows multi-GPU rigs to achieve token production rates that dense 70B or 90B models cannot match.
• Linear State Scaling via Mamba-2: The inclusion of Mamba-2 layers mitigates the exponential KV cache growth typically seen when scaling to long contexts. While pure attention mechanisms buckle under 1M tokens, the hybrid layers keep memory overhead predictable over long document and video processing sessions.
• Native Multi-Token Prediction (MTP): The base architecture natively integrates speculative decoding heads. Rather than predicting a single token per forward step, the network forecasts multiple sequential tokens simultaneously, boosting inference throughput out of the box without requiring a secondary draft model.

NVIDIA packages the model natively in BF16 (231.84 GB) and a distilled NVFP4 format (69.91 GB). The NVFP4 release relies on quantization-aware distillation from the BF16 checkpoint to preserve complex vision and reasoning performance at quarter-precision.

Nemotron 3.5 Super VL is built for tool-assisted agentic workloads. Its pretraining and post-training pipelines combine supervised fine-tuning, reinforcement learning with verifiable rewards (RLVR), and multi-teacher on-policy distillation.

• Multimodal Agentic Workflows and Code: Nemotron 3.5 Super VL 120B-A12B for coding excels at inspecting complex software interfaces, interpreting visual architecture schemas, reading UI mockups, and writing functioning code to match. It reliably generates structured API calls and executes multi-step tool loops across interactive terminal environments.
• Configurable Reasoning Modes: The chat template exposes a native enable_thinking=True/False switch. Enabling thinking engages deep chain-of-thought verification, allowing the model to self-correct during complex math, logic, and visual reasoning tasks before emitting a final answer. This behavior gives it an edge across every core Nemotron 3.5 Super VL 120B-A12B reasoning benchmark.
• Million-Token Document and Video Ingestion: With 1,048,576 tokens of context, the model can ingest hundreds of pages of PDF documentation as native images alongside extracted text. For video tasks, it processes sampled frames across hour-long recordings to pinpoint precise timestamps, summarize events, and follow visual instructions across extended sequences.

Deploying this model on private hardware requires planning for weight footprint first and compute requirements second. The primary constraint is fitting 120B parameters into addressable memory.

• BF16 (16-bit): Demands ~240 GB of VRAM. Requires enterprise infrastructure such as 4x 80GB GPUs (A100/H100) or 2x H200/B200 accelerators.
• NVFP4 (4-bit native): Demands ~72 GB to 80 GB of VRAM. Fits cleanly onto a single 80GB H100 NVL, H200, B200, or a DGX Station workstation.
• Q4_K_M (4-bit GGUF): Demands ~70 GB to 76 GB of VRAM for the base weights. To allow room for a functional context window, plan for an 80 GB to 96 GB VRAM pool.
• Q8_0 (8-bit GGUF): Demands ~130 GB to 140 GB of VRAM, requiring at least two 80 GB or 96 GB accelerators.

The best quantization for Nemotron 3.5 Super VL 120B-A12B for non-enterprise users is Q4_K_M (or EXL2 at 4.0 bpw). This maintains near-BF16 reasoning and visual comprehension while keeping the weight footprint within range of multi-GPU consumer configurations.

If you are considering how to run 120B model on consumer GPU hardware, a single card will not suffice. A standard 24GB card like an RTX 4090 lacks the capacity to hold the weights alone.

• Multi-GPU Consumer Setup: A rig containing three or four 24GB GPUs (such as four RTX 3090s or RTX 4090s) yields 72 GB to 96 GB of pooled VRAM. This is the most practical desktop hardware configuration for running the Q4_K_M quantization at full speed.
• Unified Memory Workstations: Apple Silicon workstations offer an accessible single-box solution. A Mac Studio or Mac Pro equipped with an M2, M3, or M4 Max/Ultra chip and 96 GB, 128 GB, or 192 GB of unified memory can run the model entirely in memory without offloading penalties.
• Prosumer Workstation GPUs: Dual 48GB cards, such as two RTX 6000 Ada or two RTX A6000 GPUs, provide 96 GB of aggregate VRAM over PCIe, representing the best GPU for Nemotron 3.5 Super VL 120B-A12B deployments requiring strict local workstation reliability.

Thanks to the 12B active parameter design, Nemotron 3.5 Super VL 120B-A12B tokens per second rates are high once the model is loaded:

• Quad RTX 4090 Rig (vLLM / SGLang, Q4/AWQ): Expect between 35 and 55 tokens per second during generation.
• Apple Silicon (M-series Ultra, 128GB+ unified memory, llama.cpp): Expect between 18 and 28 tokens per second.
• NVIDIA H100/H200 (TensorRT-LLM, NVFP4): Expect upwards of 70 to 100+ tokens per second, aided by native MTP speculative decoding.

To get started quickly, Ollama provides the simplest path for desktop setups once community GGUF weights are pulled:

1# Pull and run the model via Ollama (requires configured multi-GPU or unified memory)
2ollama run nemotron-3.5-super-vl:120b-q4_k_m

For high-concurrency production serving, deploy via vLLM or SGLang using tensor parallelism across your GPU array:

1vllm serve nvidia/nemotron-3.5-super-vl-120b-a12b \
2    --tensor-parallel-size 4 \
3    --max-model-len 32768 \
4    --trust-remote-code

Evaluating Nemotron 3.5 Super VL 120B-A12B vs leading alternatives highlights clear trade-offs in memory versus generation speed.

• Nemotron 3.5 Super VL 120B-A12B vs Qwen2.5-VL 72B: Qwen2.5-VL 72B is a dense model. At 4-bit precision, Qwen fits into roughly 48 GB of VRAM, making it easier to host on dual 24GB GPUs. However, because Qwen activates all 72 billion parameters on every token, Nemotron generates text significantly faster once seated in memory. Nemotron also offers a native 1M context window, outclassing Qwen's standard 128k boundary for massive video and document corpora.
• Nemotron 3.5 Super VL 120B-A12B vs Llama 3.2 90B Vision: Llama 3.2 90B Vision requires approximately 55 GB to 60 GB of VRAM at 4-bit precision. While Llama is simpler to deploy due to its conventional dense architecture, Nemotron delivers sharper agentic tool use, native multi-token speculation, and lower latency per token due to its 12B active routing path.

Choose Nemotron 3.5 Super VL 120B-A12B if you have the hardware to handle an 80GB+ VRAM footprint and need high-throughput agent execution, extended video parsing, or million-token context exploration. Choose dense 70B alternatives if your hardware is strictly limited to dual 24GB cards and you cannot accommodate the memory footprint of a 120B model.