Edited by 
Short answer
TFLOPS counts trillions of floating-point operations per second; TOPS counts trillions of operations of any kind, usually integer (INT8) or, in NVIDIA's "AI TOPS", 4-bit floating point. Both count one multiply-add as two operations. The numbers only compare fairly at the same precision and both dense or both sparse. Halve a figure to remove 2:4 sparsity, and halve again for each step up in precision, FP4 to FP8 to FP16. That turns the RTX 5090's 3,352 AI TOPS into 419 dense FP16 TFLOPS. For running language models, memory bandwidth often matters more than either number.
What TOPS and TFLOPS mean
| Term | Meaning | Typical use |
|---|---|---|
| FLOPS | Floating-point operations per second | GPUs, TPUs, scientific computing |
| TFLOPS | 1012 FLOPS | GPU specs, such as 989 dense FP16 TFLOPS on H100 |
| PFLOPS | 1015 FLOPS, or 1,000 TFLOPS | Newer data-center GPUs and racks |
| TOPS | 1012 operations per second, any number format | NPUs (usually INT8), edge AI chips |
| AI TOPS | NVIDIA's GeForce and RTX rating | FP4 with sparsity on Blackwell cards |
In every case one fused multiply-add (a × b + c) counts as two operations. A Tensor Core doing one 4 × 4 matrix multiply-accumulate performs 64 multiply-adds, or 128 operations.
Four things that change the headline number
1. Precision
Smaller numbers are cheaper to multiply, so the same Tensor Cores run FP8 at twice the FP16 rate and FP4 at twice the FP8 rate. A chip's FP4 figure is typically four times its FP16 figure. Read FP16, BF16, FP8 and FP4 explained for the formats themselves.
2. Sparsity
From Ampere onward, NVIDIA Tensor Cores can skip zeros in a 2:4 pattern, two zeros in every four values, which doubles peak throughput. Most spec sheets headline the sparse figure. Real models rarely run in that pattern, so the dense figure, half the sparse one, is the fair number.
3. Accumulate precision
Data-center GPUs add up products in FP32 at full speed. GeForce cards run FP16 with FP32 accumulate at half the rate of FP16 accumulate, so a consumer card has two FP16 numbers. Training normally uses FP32 accumulate.
4. Clock speed
Peak figures use the boost clock. Under sustained load, clocks and therefore throughput can be lower, especially in laptops and power-capped servers.
Converting headline numbers to dense FP16
| Chip | Headline figure | Remove sparsity | Step to FP8 | Dense FP16 |
|---|---|---|---|---|
| GeForce RTX 5090 | 3,352 AI TOPS (FP4, sparse) | 1,676 FP4 | 838 FP8 | 419 TFLOPS (209.5 with FP32 accumulate) |
| NVIDIA H100 SXM | 3,958 FP8 TFLOPS (sparse) | 1,979 FP8 | already FP8 | 989 TFLOPS |
| NVIDIA B200 (HGX) | 18 FP4 PFLOPS (sparse) | 9 PFLOPS FP4 | 4.5 PFLOPS FP8 | 2,250 TFLOPS |
| Laptop NPU | 40–80 TOPS (INT8, dense) | already dense | n/a | Rarely published |
Once converted, the comparison is fair. The RTX 5090 has about 42% of an H100's dense FP16 throughput with FP16 accumulate, not the 85% that 3,352 against 3,958 suggests. Every figure in our GPU specs database is already dense and per chip.
How peak throughput is calculated
Peak throughput is units × operations per clock × clock speed. The original Tensor Core GPU, the V100, has 640 Tensor Cores. Each does 64 FP16 multiply-adds per clock, and the boost clock is 1.53 GHz:
640 × 64 × 2 operations × 1.53 GHz ≈ 125 TFLOPS, exactly the V100's rated FP16 Tensor Core figure.
See how that number has grown since in Tensor Core generations.
Peak is not what you get
Peak numbers are ceilings. Well-tuned large matrix multiplies reach roughly 60% to 80% of peak, and whole training runs commonly reach 30% to 50%, a measure called model FLOPs utilization (MFU). Inference for one user is usually far lower still, because it is limited by memory, not math.
Generating a token means reading the model's active weights from memory. An H100 can do about 295 FP16 operations in the time it reads one byte (989 TFLOPS ÷ 3.35 TB/s). Token-by-token generation does only about two operations per weight, so the Tensor Cores mostly wait. That is why the VRAM calculator estimates speed from bandwidth, not TOPS. Read how to read a GPU spec sheet for the full method.
A checklist for comparing AI chips
- Is the figure dense or sparse? Halve sparse figures.
- Which precision? Compare FP8 with FP8 and FP16 with FP16.
- For GeForce cards, which accumulate precision?
- Is it per chip, per card, per server or per rack?
- How much memory, and how fast? For language models, bandwidth predicts speed and capacity decides what fits.
- Will your software use the format? FP4 throughput helps only if your framework and model support FP4.
Frequently asked questions
Is 1 TOPS the same as 1 TFLOPS?
Both mean 1012 operations per second, but TOPS can count integer or very low-precision operations while TFLOPS counts floating-point ones. 1 TOPS of INT8 is not the same amount of useful work as 1 TFLOPS of FP16, so convert to the same precision before comparing.
What does a 45 TOPS NPU mean?
It means the NPU can perform about 45 trillion operations per second, usually at INT8 without sparsity. It is enough for on-device features such as live captions and small language models. See NPU vs GPU vs TPU for how NPUs compare with GPUs.
Why does NVIDIA quote AI TOPS for GeForce cards?
AI TOPS is a single headline number for marketing, measured at the fastest precision the card supports with sparsity. For Blackwell GeForce cards that is FP4 with 2:4 sparsity, the largest possible figure.
How many TOPS do I need to run an LLM locally?
Compute is rarely the limit. What matters is having enough memory to hold the model and enough bandwidth to read it quickly. Read RTX 5090 for AI and the best GPU for local LLMs for real examples.
