Every GPU and Mac in our directory, ranked by how well it runs Apertus 70B at 4-bit. Each grade weighs whether the model fits in memory and how fast it runs, so you can see what to buy at a glance.
At 4-bit, Apertus 70B needs about 43 GB of VRAM. The smallest card in our directory that runs it well is the LattePanda Sigma 2 (59 GB), and the GIGABYTE NVIDIA Vera Rubin NVL72 GIGAPOD AI DLC Rack runs it fastest at roughly 25971 tok/s.
76 of 127 devices can run Apertus 70B at a usable speed.
See full Apertus 70B details, specs, and benchmarksThe top devices for this model at 4-bit, ranked by fit and speed.
| Device | Grade | Speed | VRAM |
|---|---|---|---|
| NVIDIA H100 SXM5 80GBNVIDIA | SS | 62.1 tok/s | 43.4 GB |
| Google Cloud TPU v5pGoogle | SS | 51.3 tok/s | 43.4 GB |
| Intel Gaudi 2 AI AcceleratorIntel | SS | 45.5 tok/s | 43.4 GB |
| NVIDIA A100 SXM4 80GBNVIDIA | SS | 37.8 tok/s | 43.4 GB |
| Intel Gaudi 3 AI AcceleratorIntel | SS | 68.6 tok/s | 43.4 GB |
| NVIDIA H200 SXM 141GBNVIDIA | SS | 89.0 tok/s | 43.4 GB |
| AMD Instinct MI350PAMD | SS | 74.2 tok/s | 43.4 GB |
| AMD Instinct MI300XAMD | SS | 98.3 tok/s | 43.4 GB |
| Google TPU v7 (Ironwood)Google | SS | 136.9 tok/s | 43.4 GB |
| NVIDIA B200 GPUNVIDIA | SS | 148.4 tok/s | 43.4 GB |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionNVIDIA | SS | 33.4 tok/s | 43.4 GB |
| AMD Instinct MI325XAMD | SS | 111.3 tok/s | 43.4 GB |
NVIDIA H100 SXM5 80GBNVIDIA | SS | 62.1 tok/s | 43.4 GB | |
Google Cloud TPU v5pGoogle | SS | 51.3 tok/s | 43.4 GB | |
| SS | 45.5 tok/s | 43.4 GB | ||
NVIDIA A100 SXM4 80GBNVIDIA | SS | 37.8 tok/s | 43.4 GB | |
| SS | 68.6 tok/s | 43.4 GB | ||
NVIDIA H200 SXM 141GBNVIDIA | SS | 89.0 tok/s | 43.4 GB | |
| SS | 74.2 tok/s | 43.4 GB | ||
| SS | 98.3 tok/s | 43.4 GB | ||
| Ad | ||||
Google TPU v7 (Ironwood)Google | SS | 136.9 tok/s | 43.4 GB | |
NVIDIA B200 GPUNVIDIA | SS | 148.4 tok/s | 43.4 GB | |
| SS | 33.4 tok/s | 43.4 GB | ||
| SS | 111.3 tok/s | 43.4 GB | ||
| SS | 131.7 tok/s | 43.4 GB | ||
| SS | 148.4 tok/s | 43.4 GB | ||
NVIDIA Rubin GPUNVIDIA | SS | 3265.0 tok/s | 43.4 GB | |
Rubin R100 UltraNVIDIA | SS | 408.1 tok/s | 43.4 GB | |
| Ad | ||||
TPU 8i MaxGoogle | SS | 319.1 tok/s | 43.4 GB | |
| SS | 131.7 tok/s | 43.4 GB | ||
| SS | 131.7 tok/s | 43.4 GB | ||
| SS | 131.7 tok/s | 43.4 GB | ||
| SS | 131.7 tok/s | 43.4 GB | ||
SuperMicro Super AI StationSuperMicro | SS | 131.7 tok/s | 43.4 GB | |
Gigabyte W775-V10-L01Gigabyte | SS | 131.7 tok/s | 43.4 GB | |
| SS | 25971.5 tok/s | 43.4 GB | ||
| Ad | ||||
Intel GPU D270Intel | AA | 27.8 tok/s | 43.4 GB | |

Hardware Compatibility
Enter your hardware and get a ranked list of every model it can run, with speed and fit scores.

Run It Locally
Compact machines that run this class of model on your desk, without a monthly cloud bill.
Straight answers to the questions we hear most often.
Can't find what you're looking for? Book a Discovery Call
The LattePanda Sigma 2 with 59 GB of VRAM is the smallest device in our directory that runs Apertus 70B at 4-bit, at roughly 3 tok/s. Any card with at least 43 GB will fit it.
We size the model at 4-bit (Q4_K_M), add the KV cache for its context length and a fixed runtime overhead, then compare that to each device VRAM. A device is graded on how comfortably the model fits and how fast it should run, based on the card memory bandwidth.
It is an estimate of decode speed in tokens per second at 4-bit, derived from the device memory bandwidth and the model size. Real speed varies with the runtime, batch size, and context length, so treat it as a ballpark for comparing devices, not a guarantee.
4-bit (Q4_K_M) is what most people actually run locally. It roughly quarters the memory a model needs versus 16-bit, with a small quality trade-off, so far more hardware can run the model. Each model page has a full quantization breakdown if you want other levels.
Higher-quality quants like Q5, Q6, or Q8 need more VRAM, so a device that is a tight fit at 4-bit may not fit at all. The model detail page shows the VRAM required at every quantization level so you can plan for the exact one you want.