Reference · Last reviewed November 2026
Tensor Core facts and statistics
25 verified facts about NVIDIA Tensor Cores and the GPUs around them: what they compute, how fast they are, which number formats each generation added, and the memory and interconnect that feed them. Every fact links to its primary source and has its own permanent link.
- A Tensor Core is a matrix-math unit that computes D = A × B + C on small tiles in a single operation.
- Tensor Cores debuted in the Volta V100 in 2017 at 125 TFLOPS; a Blackwell B200 delivers 2,250 dense FP16 TFLOPS and 9 PFLOPS of dense FP4.
- Each generation added smaller number formats: FP16, then INT8/INT4, TF32/BF16, FP8 and finally FP4.
What Tensor Cores are
A Tensor Core performs a matrix multiply-accumulate, D = A × B + C, on small matrix tiles as a single hardware operation, instead of one multiply and add at a time.
Tensor Cores first shipped in the NVIDIA Tesla V100 (Volta), announced in May 2017, which has 640 Tensor Cores across 80 streaming multiprocessors.
Each Volta Tensor Core performs 64 fused multiply-add operations per clock: a 4 × 4 × 4 FP16 matrix multiply with FP16 or FP32 accumulation.
From Ampere onward, NVIDIA data-center GPUs have 4 Tensor Cores per streaming multiprocessor: 432 on the A100 (108 SMs) and 528 on the H100 SXM (132 SMs).
Peak Tensor Core throughput is SMs × Tensor Cores per SM × FLOPs per Tensor Core per clock × clock speed. For the A100: 108 × 4 × 512 FLOPs × 1.41 GHz ≈ 312 TFLOPS of dense FP16.
Performance
The H100 SXM delivers 989 TFLOPS of dense FP16/BF16 and 1,979 TFLOPS of dense FP8 Tensor Core throughput. "With sparsity" figures are double.
On the H100 SXM, Tensor Core FP16 throughput (989 TFLOPS) is about 15 times its general-purpose FP32 CUDA core throughput (67 TFLOPS).
A Blackwell B200 in an HGX system delivers 9 PFLOPS of dense FP4 and 2.25 PFLOPS of dense FP16/BF16 per GPU.
A Blackwell Ultra B300 in an HGX system delivers about 13.5 PFLOPS of dense FP4 per GPU (108 PFLOPS for eight GPUs).
Dense FP16 Tensor Core throughput grew 18-fold in seven years, from 125 TFLOPS on V100 (2017) to 2,250 TFLOPS on B200 (2024).
"With sparsity" throughput relies on 2:4 structured sparsity, introduced with Ampere in 2020: in every group of four weights, two are zero, and the Tensor Core skips them to double throughput.
Number formats
Tensor Core formats by generation: Volta FP16; Turing added INT8, INT4 and INT1; Ampere added TF32, BF16 and FP64; Hopper added FP8; Blackwell added FP6 and FP4 with microscaling.
TF32 has FP32's 8 exponent bits and FP16's 10 mantissa bits (19 bits in total), so FP32 code can run on Tensor Cores with no change to its range.
FP8 comes in two variants: E4M3, with a maximum of 448, and E5M2, with a maximum of 57,344.
FP16 overflows above 65,504, while BF16 keeps FP32's range of about 3.4 × 10³⁸ by using 8 exponent bits and only 7 mantissa bits.
An FP4 E2M1 element can represent only eight magnitudes: 0, 0.5, 1, 1.5, 2, 3, 4 and 6. Block scaling makes it usable: OCP MXFP4 shares one scale per 32 elements, NVIDIA NVFP4 one FP8 scale per 16.
In PyTorch, TF32 for matrix multiplies has been off by default since version 1.12; it must be enabled to let FP32 matmuls use Tensor Cores.
Memory and interconnect
The H100 SXM has 80 GB of HBM3 at 3.35 TB/s; the H200 has 141 GB of HBM3e at 4.8 TB/s with the same compute.
To keep its FP16 Tensor Cores busy, an H100 SXM needs about 295 FLOPs per byte read from memory (989 TFLOPS ÷ 3.35 TB/s). Kernels below that are memory-bound.
Generating one token at batch size 1 reads every weight once, so a 70-billion-parameter model in FP16 must move about 140 GB per token; that is why memory bandwidth limits chatbot speed.
NVLink bandwidth per GPU has grown each generation: 300 GB/s (V100), 600 GB/s (A100), 900 GB/s (H100) and 1.8 TB/s (Blackwell).
GB200 NVL72 connects 72 Blackwell GPUs into a single NVLink domain, so they can exchange data as if they were one very large GPU.
Fourth-generation NVLink's 900 GB/s per GPU is about 7 times the bandwidth of a PCIe Gen5 x16 slot.
Beyond NVIDIA
AMD's equivalents are called Matrix Cores; the Instinct MI355X delivers about 2.5 PFLOPS of dense FP16 and about 10 PFLOPS of dense FP4 with 288 GB of HBM3e.
Google's TPUs use large systolic-array matrix units; the Ironwood TPU (TPU7x) delivers 4,614 TFLOPS of FP8 with 192 GiB of HBM at 7.38 TB/s per chip.
How to cite these facts
You are welcome to quote any fact on this page. Please link to the fact's own anchor (use "Copy link") or to this page, and credit TensorCore.si.
TensorCore.si Editorial. (2026, November). Tensor Core facts and statistics. TensorCore.si. https://tensorcore.si/tensor-core-facts/