Edited by 
Short answer
GB200 NVL72 is NVIDIA's rack-scale AI system: 72 Blackwell GPUs and 36 Grace CPUs in one liquid-cooled rack, all connected by NVLink into a single domain with 130 TB/s of total GPU-to-GPU bandwidth. Together they hold 13.4 TB of HBM3e at 576 TB/s and deliver 720 PFLOPS of dense FP4 (1,440 with sparsity) or 180 PFLOPS of dense FP16. Because every GPU can talk to every other at full NVLink speed, the rack behaves almost like one giant GPU, which is ideal for serving trillion-parameter mixture-of-experts models.
GB200 NVL72 specifications
| Specification | Whole rack | Per GPU |
|---|---|---|
| Configuration | 36 Grace CPUs, 72 Blackwell GPUs | – |
| FP4 Tensor (dense / sparse) | 720 / 1,440 PFLOPS | 10 / 20 PFLOPS |
| FP8 Tensor (dense) | 360 PFLOPS | 5 PFLOPS |
| FP16 / BF16 Tensor (dense) | 180 PFLOPS | 2.5 PFLOPS |
| FP64 | 2,880 TFLOPS | 40 TFLOPS |
| GPU memory | 13.4 TB HBM3e | about 186 GB |
| GPU memory bandwidth | 576 TB/s | 8 TB/s |
| NVLink bandwidth | 130 TB/s | 1.8 TB/s |
| CPU cores and memory | 2,592 Arm Neoverse V2 cores, 17 TB LPDDR5X | – |
| Cooling | Liquid-cooled | – |
What is inside the rack
- 18 compute trays, each holding two GB200 superchips, which means 4 Blackwell GPUs and 2 Grace CPUs per tray.
- 9 NVLink switch trays containing the NVLink switch chips that connect all 72 GPUs to each other.
- A copper NVLink spine at the back of the rack, carrying the GPU-to-GPU links without optical transceivers, which saves power and cost.
- Liquid cooling for the GPUs and CPUs, needed at this power density.
The GB200 superchip
The building block is the GB200 Grace Blackwell Superchip: one 72-core Grace CPU and two Blackwell GPUs, connected by 900 GB/s NVLink-C2C so the GPUs can use the CPU's memory coherently. Each superchip offers 372 GB of HBM3e at 16 TB/s and 20 PFLOPS of dense FP4. Its desktop cousin, the GB10 chip in DGX Spark, uses the same idea at a much smaller scale.
Why one 72-GPU NVLink domain matters
In a standard 8-GPU server, GPUs talk to each other over NVLink at 1.8 TB/s, but traffic to any other server drops to network speed, typically 50–100 GB/s per GPU. Large mixture-of-experts models spread their experts across many GPUs and exchange tokens between them on every layer. In NVL72, all 72 GPUs share NVLink speed, so models with hundreds of experts and trillions of parameters can be split across the rack without hitting the network bottleneck. The combined 13.4 TB of memory also holds very large models and long KV caches.
Performance claims
NVIDIA claims GB200 NVL72 delivers 30 times faster real-time inference on trillion-parameter language models than the same number of H100 GPUs (at 50 ms per token, with 32,768 input tokens and 1,024 output tokens), 4 times faster training, and 25 times more performance at the same power as air-cooled H100 infrastructure. These comparisons combine FP4, the larger NVLink domain and Blackwell's faster chips, so gains on smaller models are much smaller. Per GPU, the raw step from H100 is about 2.5 times in dense FP16; see B200 vs H100.
NVL72, HGX and GB300
| System | GPUs in one NVLink domain | Form |
|---|---|---|
| HGX B200 / B300 | 8 | Server board, often air-cooled |
| GB200 NVL72 | 72 Blackwell | Liquid-cooled rack |
| GB300 NVL72 | 72 Blackwell Ultra (288 GB each) | Liquid-cooled rack |
Frequently asked questions
What does NVL72 mean?
"NVL" refers to NVLink and "72" to the number of GPUs connected in one NVLink domain.
How is GB200 different from B200?
B200 is a single Blackwell GPU, usually sold in 8-GPU HGX servers with x86 CPUs. GB200 is a superchip that pairs two Blackwell GPUs with a Grace CPU, and it runs at a higher power limit, giving 10 rather than 9 PFLOPS of dense FP4 per GPU.
Can NVL72 racks be connected together?
Yes. Racks connect through InfiniBand or Ethernet networking to form clusters of tens of thousands of GPUs; each rack remains its own 72-GPU NVLink domain.
Leave a Reply