NVIDIA Vera Rubin NVL72

NVIDIA Vera Rubin NVL72 is a rack-scale AI supercomputer built from 72 Rubin GPUs and 36 Vera CPUs, wired into a single 260 TB/s NVLink domain. It is the successor to the GB200 and GB300 NVL72 systems, and it is purpose-built for the two workloads that broke previous generations' economics: trillion-parameter mixture-of-experts models and long-context agentic inference where a single request can burn tens of thousands of tokens on reasoning and tool calls.

This is not a workstation and not a desktop product. It is a 100% liquid-cooled, third-generation MGX NVL72 rack that needs real facility power, liquid cooling, and rack-scale deployment logistics. The directory entry currently carries no confirmed manufacturer record — if you landed here looking for Unknown hardware for AI development, the platform vendor is NVIDIA. Availability is listed as announced; no MSRP is published. Rack-scale NVL72 systems have historically transacted in the millions of dollars per rack, so plan on rental capacity from a neocloud or hyperscaler as the realistic first touch.

The reason practitioners care about NVIDIA Vera Rubin NVL72 for AI is the efficiency claim. NVIDIA states one-tenth the cost per million tokens and one-fourth the GPUs for MoE training versus GB200 NVL72, plus up to 10x more tokens per megawatt. Those numbers are vendor benchmarks (the cost-per-token figure is measured on Kimi-K2-Thinking at 32K/8K input/output sequence lengths), but they define the tier: this is the top of the market, competing with prior-generation NVL72 racks from NVIDIA itself and non-NVIDIA rack-scale accelerators.

• GPUs: 72x NVIDIA Rubin GPUs
• CPUs: 36x NVIDIA Vera CPUs
• HBM4: 20.7 TB total (~287 GB per GPU)
• LPDDR5X: 54 TB (CPU-attached, extends the KV cache tier)
• Total fast-access memory: 75 TB
• NVLink domain bandwidth: 260 TB/s aggregate
• NVFP4 inference: 3,600 PFLOPS
• NVFP4 training: 2,520 PFLOPS
• FP8/FP6 training: 1,260 PFLOPS
• Networking: ConnectX-9 SuperNICs, BlueField-4 DPUs, Quantum-X800 InfiniBand, Spectrum-X Ethernet
• Max model size: trillion-parameter class

All PFLOPS figures are rack-aggregate. Divide by 72 and you get roughly 50 PFLOPS NVFP4 inference and 35 PFLOPS NVFP4 training per Rubin GPU, with 17.5 PFLOPS per GPU at FP8/FP6 for training. NVFP4 is the headline format here — it's what unlocks both the training and inference numbers, and it's why the efficiency claims land. Video generation, graph neural networks, and physical AI simulation pipelines are compute-bound, not bandwidth-bound, so the raw NVFP4 throughput is what moves those workloads.

The 20.7 TB of HBM4 is not there to hold model weights — it's there to hold KV cache at scale. Weights for even a 1T-parameter model at FP4 land around 500–600 GB, meaning the rack can hold dozens of copies. Run the KV math instead: a 128-layer GQA model with 8 KV heads at 128 head dimension, cached at FP8, costs about 256 KiB per token. A 1M-token context therefore consumes roughly 256 GB. That caps you around 80 concurrent million-token sessions in HBM4 alone, with the 54 TB LPDDR5X tier adding a couple hundred more at a latency penalty when you offload there. Models using MLA-style attention (DeepSeek-R1, DeepSeek-V3) compress the cache far harder and push concurrency much higher.

The 260 TB/s NVLink domain is what allows expert-parallel MoE inference without the all-to-all becoming the bottleneck. Scale-out runs over Quantum-X800 InfiniBand and Spectrum-X Ethernet, with ConnectX-9 SuperNICs and BlueField-4 DPUs handling the data path and offload. For multi-rack DGX SuperPOD deployments, this is the fabric that determines whether a 288-GPU job scales linearly or falls off a cliff.

Model fit is essentially a non-issue. At 20.7 TB of HBM4, the rack holds multiple full copies of any open-weight model in existence. The relevant question is throughput and concurrency, not whether it fits.

• 70B-class dense models (Llama 3.1 70B, Qwen 2.5 72B): run at BF16 or FP8 with enormous headroom. Batch sizes in the thousands per rack are practical.
• Frontier MoE models (DeepSeek-R1/V3 class, Kimi-K2, Qwen3 MoE, Llama 4): the actual target. NVFP4 expert weights are the sweet spot — MoE expert layers quantize cleanly, and the active-parameter count per token keeps decode cheap relative to the batch size you can sustain.
• Trillion-parameter models at long context: the reason this rack exists. 1M-token contexts with reasoning traces are memory-capacity problems, and 75 TB of fast-access memory is the answer.

Quantization guidance. For serving, NVFP4 is the best quality-to-speed tradeoff on this hardware — the tensor cores are designed around it, and the throughput delta versus FP8 is large. Reserve FP8/FP6 for fine-tuning runs where small quality deltas matter, and BF16 for small-model reference evaluations where you're not throughput-bound.

Tokens per second. Per-stream decode is bandwidth-bound, not compute-bound; a single user on a trillion-parameter MoE will see tens-to-low-hundreds of tokens/sec depending on active parameters. The rack's value is aggregate: thousands of concurrent streams at the same latency budget, which is exactly the shape of agentic traffic. NVIDIA's MLPerf Inference v6.1 debut submission (a preview entry) shows leading system performance at rack scale — check that submission for the workloads closest to yours before sizing.

Multimodal and long-context. Video generation, high-fidelity simulation, and multimodal front-ends all fit; the compute headroom handles the encoder/decoder side without starving the LLM serving path.

• AI factories and token providers: the cost-per-million-tokens argument is the whole pitch. If you sell inference, this is the unit-economics lever.
• Frontier labs and large enterprises: pretraining, post-training, and inference-time scaling on the same hardware, which keeps utilization high between training runs.
• Agentic platform teams: long-context multi-step agents with tool use are the workload this rack was designed around. Anyone building best-in-class local AI agents in 2026 who controls their own data center should be evaluating this tier.
• Not for hobbyists or single-workstation deployments. If you want the best hardware for running AI models locally on your desk, this is the wrong product — look at RTX 6000-class cards or a DGX Station instead. Rack-scale NVL72 systems are the answer for local in the sense of "on your own floor," not "under your desk."
• Training vs. inference: 2,520 PFLOPS NVFP4 training is credible frontier-scale capacity, but most buyers will run inference and rely on the platform's fungibility to absorb training bursts.

vs. NVIDIA GB300 / GB200 NVL72. Same rack footprint, same third-generation MGX design, same NVIDIA software stack — which makes Vera Rubin an in-place upgrade rather than a re-architecture. NVIDIA's own comparison is the strongest case: one-fourth the GPUs for MoE training and one-tenth the cost per million tokens versus GB200 NVL72. If you already operate NVL72 racks, the migration risk is low and the efficiency delta is large. If you're buying your first rack-scale system, there's little reason to buy the outgoing generation.

vs. AMD Helios-class rack systems. The competing rack-scale accelerator approach, typically positioned on power-per-dollar and open rack designs. NVIDIA's durable advantages are the 260 TB/s single NVLink domain and the maturity of the CUDA serving stack — both matter more at 72-GPU scale than on a single card. AMD wins where pricing, supply, or vendor-diversity requirements dominate. Verify current Helios specifications independently; treat vendor-published comparisons from either side as directional.

When to pick the NVL72: you're serving trillion-parameter MoE models at long context, you need expert-parallel all-to-all that doesn't bottleneck, and you can support a liquid-cooled rack. When to look elsewhere: your models fit in a few hundred gigabytes, your traffic is bursty and low-volume, or you'd rather rent capacity than own the cooling loop.