Edited by 
At a glance
Five generations of Tensor Cores have shipped since 2017, and a sixth platform, Rubin, arrives in 2026. Each one added smaller number formats or a new way to feed data. Dense FP16 throughput grew from 125 TFLOPS on V100 to 2,250 TFLOPS on B200, and Blackwell's FP4 reaches 9,000 TFLOPS dense per GPU.
| Gen | Architecture | Year | Data-center GPU | New on Tensor Cores | Dense FP16 peak |
|---|---|---|---|---|---|
| 1 | Volta | 2017 | V100 | FP16 MMA with FP32 accumulate | 125 TFLOPS |
| 2 | Turing | 2018 | T4 | INT8, INT4 | 65 TFLOPS |
| 3 | Ampere | 2020 | A100 | TF32, BF16, FP64, 2:4 sparsity | 312 TFLOPS |
| 4 | Hopper | 2022 | H100 / H200 | FP8, Transformer Engine | 989 TFLOPS |
| 5 | Blackwell | 2024 | B200 / B300 | FP6, FP4, microscaling | 2,250 TFLOPS |
| Next | Rubin | 2026 | Rubin | HBM4, NVFP4 focus | Not yet published |
1st generation: Volta (2017)
The Tesla V100 introduced Tensor Cores. Each of its 80 SMs held 8 Tensor Cores, 640 in total. Every Tensor Core multiplied two 4×4 FP16 matrices and added the result to a 4×4 FP16 or FP32 accumulator each clock, which is 64 fused multiply-adds. At its boost clock that gave V100 125 TFLOPS of mixed-precision Tensor throughput, roughly 8 times its FP32 rate.
Volta also brought the CUDA WMMA API, so developers and libraries could program the new units. Mixed-precision training, with FP16 inputs and FP32 master weights and accumulation, became standard practice because of it.
2nd generation: Turing (2018)
Turing brought Tensor Cores to GeForce RTX 20 graphics cards and to the T4 data-center card. The big addition was integer math: INT8 and INT4 modes for quantized inference, plus an experimental 1-bit mode. The 70 W T4 delivered 65 TFLOPS FP16 and 130 TOPS INT8, which made it a popular inference card in cloud servers. On the gaming side, Tensor Cores powered the first version of DLSS.
3rd generation: Ampere (2020)
The A100 rebuilt the Tensor Core for training at scale. It has 108 SMs with 4 larger Tensor Cores each, 432 in total, for 312 TFLOPS dense FP16 or BF16. Ampere added four important features:
- TF32, a 19-bit format with FP32's range and FP16's precision. Code written for FP32 runs on Tensor Cores automatically, at 156 TFLOPS dense instead of 19.5 TFLOPS.
- BF16, which keeps FP32's exponent range and largely removed the need for loss scaling in training.
- FP64 Tensor Cores, giving 19.5 TFLOPS of double precision for scientific computing.
- 2:4 structured sparsity: if two of every four weights are zero, the Tensor Core skips them and doubles its peak throughput.
Ampere also added asynchronous copies from global memory into shared memory, the first step toward hardware that keeps Tensor Cores fed. Consumer RTX 30 cards use third-generation Tensor Cores.
4th generation: Hopper (2022)
The H100 was designed for transformers. The SXM version has 132 SMs and 528 Tensor Cores, reaching 989 TFLOPS dense FP16 or BF16 and 1,979 TFLOPS dense in the new FP8 format. FP8 comes in two flavours: E4M3, with more precision, and E5M2, with more range.
The Transformer Engine combines Hopper hardware and NVIDIA software to choose between FP8 and 16-bit precision layer by layer and to manage scaling factors so accuracy holds. Hopper also introduced the Tensor Memory Accelerator (TMA), which copies whole tiles between memory levels without tying up threads, thread block clusters that let neighbouring SMs share data, and asynchronous warp-group MMA instructions (wgmma).
The H200 kept the same compute but raised memory to 141 GB of HBM3e at 4.8 TB/s, up from 80 GB at 3.35 TB/s. Ada Lovelace RTX 40 graphics cards also use fourth-generation Tensor Cores with FP8.
5th generation: Blackwell (2024)
Blackwell joins two reticle-sized dies into one GPU with 208 billion transistors, linked at 10 TB/s. Its Tensor Cores add FP6 and FP4, used with microscaling: small blocks of values share a scale factor so tiny formats can still represent a wide range. NVIDIA's own NVFP4 uses blocks of 16 values with an FP8 scale, and the second-generation Transformer Engine manages these scales.
| Per GPU, dense | B200 | B300 (Blackwell Ultra) |
|---|---|---|
| FP4 | 9 PFLOPS | 13.5 PFLOPS |
| FP8 / FP6 | 4.5 PFLOPS | 4.5 PFLOPS |
| FP16 / BF16 | 2.25 PFLOPS | 2.25 PFLOPS |
| Memory | 180 GB HBM3e | 288 GB HBM3e |
| Memory bandwidth | about 8 TB/s | about 8 TB/s |
| NVLink | 1.8 TB/s | 1.8 TB/s |
Data-center Blackwell also changed how Tensor Cores are programmed. New tcgen05 instructions are issued by a single thread and keep accumulators in a dedicated on-chip Tensor Memory, freeing registers for other work. GeForce RTX 50 cards use fifth-generation Tensor Cores with FP4 as well. For a deeper comparison, read Hopper vs Blackwell.
Rubin: what comes next (2026)
NVIDIA announced at CES 2026 that the Vera Rubin platform is in full production, with systems arriving in the second half of 2026. Each Rubin GPU pairs two dies with 288 GB of HBM4. NVIDIA claims 50 PFLOPS of NVFP4 for inference and 35 PFLOPS for training per GPU, which it describes as 5× and 3.5× Blackwell. Treat these as vendor figures until independent benchmarks are published; we will update this guide when they are.
The pattern across generations
- Smaller formats, every time. FP16 → INT8 → TF32/BF16 → FP8 → FP4. Each halving of bits roughly doubles peak math.
- Bigger tiles per instruction. From 4×4×4 per clock on Volta to warp-group and single-thread instructions that cover far larger tiles.
- Feeding the beast. Asynchronous copies (Ampere), TMA (Hopper) and Tensor Memory (Blackwell) all exist to keep Tensor Cores busy.
- Memory grows alongside. From 16 to 32 GB of HBM2 on V100 to 288 GB of HBM4 on Rubin.
Want to check your understanding? The TensorCore.si Challenge includes problems on peak throughput, FP8 rates and sparsity, and the AI Model Explorer compares H100, H200, B200 and B300 on real models.
Leave a Reply