Edited by 
Short answer
Per GPU, the B200 delivers about 2.3 times the dense FP16 and FP8 Tensor Core throughput of the H100 SXM (2,250 vs 989 TFLOPS FP16), adds FP4 at 9 PFLOPS dense, and has 2.25 times the memory (180 vs 80 GB) at 2.4 times the bandwidth (8 vs 3.35 TB/s). It also doubles NVLink to 1.8 TB/s. The trade-offs are power, 1,000 W against 700 W, and lower FP64 throughput for scientific computing.
B200 vs H100 specifications
| Specification | H100 SXM | B200 (HGX) | B200 vs H100 |
|---|---|---|---|
| Architecture | Hopper (2022) | Blackwell (2024) | – |
| Design | 1 die, 80 billion transistors | 2 dies joined by a 10 TB/s link, 208 billion transistors | – |
| FP16 / BF16 Tensor (dense) | 989 TFLOPS | 2,250 TFLOPS | 2.3× |
| FP8 Tensor (dense) | 1,979 TFLOPS | 4,500 TFLOPS | 2.3× |
| FP4 Tensor (dense) | not supported | 9,000 TFLOPS | new |
| FP64 Tensor | 67 TFLOPS | 37 TFLOPS | 0.55× |
| Memory | 80 GB HBM3 | 180 GB HBM3e | 2.25× |
| Memory bandwidth | 3.35 TB/s | 8 TB/s | 2.4× |
| NVLink per GPU | 900 GB/s | 1.8 TB/s | 2× |
| Max power | 700 W | 1,000 W | 1.43× |
What Blackwell changed
- Two dies, one GPU. Blackwell joins two reticle-sized dies with a 10 TB/s chip-to-chip link so software sees a single GPU, the way around the size limit of one chip.
- FP4 and FP6 with microscaling. Fifth-generation Tensor Cores add block-scaled 4-bit and 6-bit formats, including NVFP4, doubling inference throughput again over FP8.
- Much more memory. Eight stacks of HBM3e give 8 TB/s, so a 70B model in FP8 fits on one GPU with plenty of room for cache.
- Fifth-generation NVLink. 1.8 TB/s per GPU and rack-scale systems such as GB200 NVL72.
Real-world performance
NVIDIA's own system comparisons claim about 3 times faster training and up to 15 times faster inference for DGX B200 versus DGX H100, inference figures that rely on FP4 and large models. Based on the specs, a fair expectation is about 2–2.5 times faster for BF16 or FP8 work, up to about 2.4 times faster single-user generation thanks to bandwidth, and larger gains when a model can run in FP4. Per watt, dense FP16 efficiency rises from about 1.41 to 2.25 TFLOPS per watt. Estimate a run in the training time and cost calculator.
Which should you choose?
- B200 for new large-scale training and inference, especially serving big models in FP8 or FP4, where cost per token falls sharply.
- H100 when you need FP64 for scientific work, have existing air-cooled 700 W infrastructure, or find H100 capacity cheaper and sooner.
- The next step up is B300 (Blackwell Ultra) with 288 GB per GPU and more FP4 throughput; compare them in the specs database.
Frequently asked questions
How much faster is B200 than H100?
About 2.3 times in dense FP16 and FP8 Tensor Core throughput and 2.4 times in memory bandwidth, per GPU. With FP4, inference throughput can be several times higher than H100 running FP8.
Why is B200 slower at FP64?
NVIDIA spent Blackwell's silicon on low-precision AI formats. B200 delivers 37 TFLOPS of FP64 against 67 on H100, and Blackwell Ultra (B300) cuts FP64 much further, so FP64-heavy science often favours Hopper.
Does B200 need liquid cooling?
At 1,000 W per GPU, many B200 systems are liquid-cooled, although air-cooled 8-GPU HGX B200 servers exist. Rack-scale GB200 NVL72 systems are liquid-cooled.
Leave a Reply