Edited by 
Short answer
Blackwell (B200, B300) roughly doubles Hopper's dense FP8 and FP16 Tensor Core throughput, adds native FP4 for another 2× or more, more than doubles memory bandwidth to about 8 TB/s and doubles NVLink to 1.8 TB/s. The biggest gains are in inference, where FP4 and memory bandwidth matter most.
| Per GPU (SXM) | H100 | H200 | B200 | B300 |
|---|---|---|---|---|
| Architecture | Hopper | Hopper | Blackwell | Blackwell Ultra |
| Memory | 80 GB HBM3 | 141 GB HBM3e | 180 GB HBM3e | 288 GB HBM3e |
| Memory bandwidth | 3.35 TB/s | 4.8 TB/s | ≈ 8 TB/s | ≈ 8 TB/s |
| Dense FP16 / BF16 | 989 TFLOPS | 989 TFLOPS | 2.25 PFLOPS | 2.25 PFLOPS |
| Dense FP8 | 1.98 PFLOPS | 1.98 PFLOPS | 4.5 PFLOPS | 4.5 PFLOPS |
| Dense FP4 | Not supported | Not supported | 9 PFLOPS | 13.5 PFLOPS |
| NVLink per GPU | 900 GB/s | 900 GB/s | 1.8 TB/s | 1.8 TB/s |
What changed in the architecture
- Two dies, one GPU. Blackwell joins two reticle-limited dies with a 10 TB/s link so software sees a single GPU with 208 billion transistors, against 80 billion in H100.
- FP4 and FP6 Tensor Cores. Fifth-generation Tensor Cores add 4- and 6-bit floating point with hardware block scaling (MXFP4, MXFP6 and NVIDIA's NVFP4).
- Second-generation Transformer Engine. It manages finer-grained, per-block scaling factors so these tiny formats keep accuracy.
- Tensor Memory and tcgen05. Accumulators move out of registers into a dedicated on-chip Tensor Memory, and a single thread can launch large matrix operations.
- NVLink 5. 1.8 TB/s per GPU, and rack-scale NVL72 systems that put 72 GPUs in one NVLink domain.
- Blackwell Ultra (B300) raises memory to 288 GB, lifts dense FP4 to 13.5 PFLOPS and doubles the speed of the exponent instructions used in attention.
What it means for training
Most large models train in BF16 with FP8 for the matrix multiplications. In both formats, B200 offers about 2.3× H100's dense peak. Real-world gains also depend on memory, NVLink and software maturity. The higher NVLink bandwidth matters for the tensor and expert parallelism used to train mixture-of-experts models, where GPUs exchange activations constantly.
Training in FP4 is an active research area. NVIDIA reports promising results with NVFP4, but BF16 and FP8 remain the standard for production training today.
What it means for inference
Inference is where Blackwell pulls furthest ahead, for two reasons:
- Memory bandwidth. Generating text for one user reads every active weight once per token, so speed scales with bandwidth: about 8 TB/s against 3.35 TB/s, or 2.4× H100.
- FP4. Quantizing weights to NVFP4 halves the bytes read per token compared with FP8 and doubles peak math, which helps both single-user speed and large-batch throughput.
Combined, a well-optimised FP4 deployment on B200 can serve several times more tokens per second than the same model in FP8 on H100. The AI Model Explorer lets you compare H100, H200, B200 and B300 on models such as Llama, DeepSeek and gpt-oss.
Memory capacity and model size
Memory decides how many GPUs a model needs before speed even enters the picture. A 405-billion-parameter model in FP8 needs about 405 GB for its weights. That is 6 H100s at minimum, 3 H200s or 3 B200s, or 2 B300s, before adding room for the KV cache. In FP4 it shrinks to about 203 GB plus scale factors, small enough for a single B300 to hold the weights.
How to choose
This is general information, not purchasing advice. In practice the decision usually comes down to:
- Availability and price per hour from your cloud provider. Hopper capacity is plentiful and cheaper; Blackwell offers more work per GPU.
- Software support. FP4 needs recent versions of TensorRT-LLM, vLLM or SGLang and a model quantized for it.
- Model size. Large models and long contexts favour the 180 to 288 GB of B200 and B300.
- Workload. Memory-bound serving gains the most from Blackwell; compute-bound FP8 training gains about 2×.
Looking ahead to Rubin
NVIDIA's next platform, Rubin, entered full production in early 2026 with systems due in the second half of the year. Each Rubin GPU carries 288 GB of HBM4, and NVIDIA claims 50 PFLOPS of NVFP4 for inference. Read more in Tensor Core generations.
Leave a Reply