Data · Last updated November 2026
AI GPU and accelerator specs database
Memory, bandwidth and Tensor Core or matrix throughput for 28 AI accelerators and GPUs, from NVIDIA V100 to Rubin, AMD MI250X to MI455X, Intel Gaudi 3, AWS Trainium and Google TPUs, plus the workstation and desktop cards people use for local AI. Every number is per chip, dense (no sparsity) and taken from the vendor's own datasheets or documentation.
- The largest memory per chip is 432 GB of HBM4 on AMD Instinct MI455X. NVIDIA Rubin, B300, GB300 and AMD MI355X each carry 288 GB.
- The fastest memory is 23.3 TB/s on MI455X, followed by NVIDIA Rubin at 19.2 TB/s (NVIDIA lowered this from an earlier 22 TB/s target in September 2026). Both are about 6 to 7 times an H100's 3.35 TB/s.
- The highest dense FP4 throughput is about 40 PFLOPS on MI455X and 35 PFLOPS on Rubin, against 9 PFLOPS on B200.
- From V100 (2017) to Rubin (2026), dense FP16 Tensor Core throughput grew 32 times (125 to 4,000 TFLOPS) while memory bandwidth grew about 21 times.
- For local AI, the RTX 5090 (32 GB, 1.79 TB/s) has the most bandwidth of any desktop card, while DGX Spark trades bandwidth (0.27 TB/s) for 128 GB of unified memory.
Compare every accelerator
Filter by vendor and click any column to sort. Bars show each chip relative to the leader in the sorted column.
| V100 SXM2NVIDIAFirst Tensor Cores | 2017 | VoltaTSMC 12 nm · HBM2 | 32 | 0.90 | 125 | – | – | 7.8 | 300 | 300NVLink 2 |
| A100 SXM 80GBNVIDIATF32, BF16, 2:4 sparsity | 2020 | AmpereTSMC 7 nm · HBM2e | 80 | 2.04 | 312 | – | – | 19.5 | 400 | 600NVLink 3 |
| H100 SXMNVIDIAFP8, Transformer Engine | 2022 | HopperTSMC 4N · HBM3 | 80 | 3.35 | 989 | 1,979 | – | 67 | 700 | 900NVLink 4 |
| H200 SXMNVIDIAH100 compute, more memory | 2024 | HopperTSMC 4N · HBM3e | 141 | 4.80 | 989 | 1,979 | – | 67 | 700 | 900NVLink 4 |
| B200 (HGX)NVIDIAFP4 and FP6 | 2024 | BlackwellTSMC 4NP · HBM3e | 180 | 8.00 | 2,250 | 4,500 | 9,000 | 37 | 1,000 | 1,800NVLink 5 |
| B300 (HGX)NVIDIABlackwell Ultra | 2025 | Blackwell UltraTSMC 4NP · HBM3e | 288 | 8.00 | 2,250 | 4,500 | 13,500 | 1.25 | 1,100 | 1,800NVLink 5 |
| H100 NVLNVIDIAPCIe card, bridged in pairs | 2023 | HopperTSMC 4N · HBM3 | 94 | 3.90 | 835 | 1,671 | – | 60 | 400 | 600NVLink bridge |
| GB200 (per GPU)NVIDIAGrace Blackwell, NVL72 racks | 2024 | BlackwellTSMC 4NP · HBM3e | 186 | 8.00 | 2,500 | 5,000 | 10,000 | 40 | 1,200 | 1,800NVLink 5 |
| GB300 (per GPU)NVIDIAGrace Blackwell Ultra, NVL72 racks | 2025 | Blackwell UltraTSMC 4NP · HBM3e | 288 | 8.00 | 2,500 | 5,000 | 15,000 | 1.25 | 1,400 | 1,800NVLink 5 |
| Rubin (per GPU)NVIDIAVera Rubin NVL72, first with HBM4 | 2026 | RubinTSMC 3 nm · HBM4 | 288 | 19.20 | 4,000 | 17,500 | 35,000 | 33 | – | 3,000NVLink 6 |
| L40SNVIDIAInference and graphics card | 2023 | Ada LovelaceTSMC 4N · GDDR6 | 48 | 0.86 | 362 | 733 | – | – | 350 | 64PCIe 4.0 |
| L4NVIDIA72 W low-profile inference card | 2023 | Ada LovelaceTSMC 4N · GDDR6 | 24 | 0.30 | 121 | 242 | – | – | 72 | 64PCIe 4.0 |
| RTX PRO 6000 BlackwellNVIDIAWorkstation card, 96 GB | 2025 | BlackwellGB202 · GDDR7 | 96 | 1.79 | 500 | 1,000 | 2,000 | – | 600 | 128PCIe 5.0 |
| GeForce RTX 5090NVIDIATop consumer card for local AI | 2025 | BlackwellGB202 · GDDR7 | 32 | 1.79 | 209.5 | 838 | 1,676 | – | 575 | 128PCIe 5.0 |
| GeForce RTX 4090NVIDIAPrevious consumer flagship | 2022 | Ada LovelaceAD102 · GDDR6X | 24 | 1.01 | 165.2 | 660.6 | – | – | 450 | 64PCIe 4.0 |
| DGX Spark (GB10)NVIDIADesktop AI computer | 2025 | BlackwellGB10 · LPDDR5x unified | 128 | 0.27 | – | – | 500 | – | 240 | 25ConnectX-7 |
| Instinct MI300XAMD192 GB in one package | 2023 | CDNA 3TSMC 5/6 nm · HBM3 | 192 | 5.30 | 1,307 | 2,615 | – | 163 | 750 | 896Infinity Fabric |
| Instinct MI325XAMDMI300X with HBM3e | 2024 | CDNA 3TSMC 5/6 nm · HBM3e | 256 | 6.00 | 1,307 | 2,615 | – | 163 | 1,000 | 896Infinity Fabric |
| Instinct MI355XAMDFP4 and FP6 | 2025 | CDNA 4TSMC N3P/N6 · HBM3e | 288 | 8.00 | 2,517 | 5,033 | 10,066 | 78.6 | 1,400 | 1,075Infinity Fabric |
| Instinct MI250XAMDFrontier supercomputer GPU | 2021 | CDNA 2TSMC 6 nm · HBM2e | 128 | 3.28 | 383 | – | – | 95.7 | 560 | 800Infinity Fabric |
| Instinct MI455XAMDHelios racks, 432 GB HBM4 | 2026 | CDNA 5TSMC 2/3 nm · HBM4 | 432 | 23.30 | 5,033 | 20,133 | 40,265 | – | – | 3,600UALink scale-up |
| Gaudi 3IntelEthernet built into the chip | 2024 | Gaudi 3TSMC 5 nm · HBM2e | 128 | 3.70 | 1,678 | 1,678 | – | – | 900 | 1,20024× 200 GbE |
| Trainium2AWSTrn2 instances and UltraServers | 2024 | NeuronCore-v3HBM | 96 | 2.90 | 667 | 1,299 | – | – | – | 1,280NeuronLink |
| Trainium3AWSTrn3 UltraServers, 144 chips | 2025 | NeuronCore-v43 nm · HBM3e | 144 | 4.90 | 671 | 2,517 | 2,517 | – | – | 2,560NeuronLink-v4 |
| TPU v5pGoogleTraining pods of 8,960 chips | 2023 | TPU v5pHBM2e | 95 | 2.77 | 459 | 459 | – | – | – | 1,200ICI |
| TPU Trillium (v6e)GooglePods of 256 chips | 2024 | TPU v6eHBM | 32 | 1.64 | 918 | 918 | – | – | – | 800ICI |
| TPU Ironwood (TPU7x)GooglePods of 9,216 chips | 2025 | TPU7xHBM3e | 192 | 7.38 | 2,307 | 4,614 | – | – | – | 1,200ICI |
| TPU v5eGoogleCost-efficient training and inference | 2023 | TPU v5eHBM2 | 16 | 0.82 | 197 | – | – | – | – | 200ICI |
All throughput is dense (without 2:4 sparsity); "with sparsity" figures on spec sheets are double these. FP64 uses Tensor Core or matrix FP64 where the chip has it (V100: vector FP64). B300 power is the approximate HGX figure. TPU power is not published, and TPU memory is listed in GiB as Google reports it. Chip-to-chip bandwidth is each vendor's quoted total per chip, so compare it within a vendor rather than across vendors. Per-GPU figures for GB200, GB300 and Rubin are NVIDIA's rack totals divided by 72; Rubin FP4 is the dense training figure (NVIDIA quotes 50 PFLOPS for inference with its adaptive compression). GeForce FP16 is shown with FP32 accumulate, which NVIDIA runs at half rate on consumer cards (330 and 419 TFLOPS with FP16 accumulate on the 4090 and 5090). RTX PRO 6000 and DGX Spark figures are half of NVIDIA's sparse ratings. Power for DGX Spark is the whole system. Desktop and workstation cards connect over PCIe, shown as total two-way bandwidth. A dash means the format is not supported or not published.
Head-to-head comparison
Pick two chips to see the ratio on every spec.
How to read these numbers
Dense, not sparse
Vendors often headline "with sparsity" numbers, which assume half the weights are zero in a 2:4 pattern. Real models rarely run that way, so this table uses dense numbers.
Bandwidth often matters more
Generating tokens one at a time is memory-bound: an H100 needs about 295 FLOPs per byte to keep its Tensor Cores busy. For chatbots, memory size and bandwidth predict speed better than TFLOPS.
Peak is a ceiling
These are theoretical maximums at boost clock. Well-tuned large matrix multiplies reach roughly 60–80% of peak; whole training runs usually land lower.
Want the full method? Read how to read a GPU spec sheet, see why memory matters in what is HBM, or estimate what a model needs in the AI model GPU calculator.
Frequently asked questions
Which AI GPU has the most memory?
As of November 2026, AMD Instinct MI455X has the most memory per chip at 432 GB of HBM4. NVIDIA Rubin, B300 and GB300 and AMD MI355X each have 288 GB, AMD MI325X has 256 GB and Google Ironwood 192 GB.
How much faster is B200 than H100?
Per chip, B200 has about 2.3 times the dense FP16 and FP8 Tensor Core throughput of H100 (2,250 vs 989 TFLOPS FP16), adds FP4 at 9 PFLOPS dense, and has 2.4 times the memory bandwidth (8 vs 3.35 TB/s). Real speedups depend on the workload and software.
What is the difference between H100 and H200?
They share the same Hopper GPU and the same compute throughput. H200 replaces 80 GB of HBM3 at 3.35 TB/s with 141 GB of HBM3e at 4.8 TB/s, which helps large models and long contexts.
What are NVIDIA Rubin's specs?
Each Rubin GPU in a Vera Rubin NVL72 rack has 288 GB of HBM4 at 19.2 TB/s, 35 PFLOPS of dense NVFP4 for training (50 PFLOPS for inference), 17.5 PFLOPS of FP8, 4 PFLOPS of FP16/BF16, 33 TFLOPS of FP64 and 3 TB/s of NVLink 6. NVIDIA lowered the memory and NVLink figures from 22 TB/s and 3.6 TB/s in September 2026. Production shipments began in the second half of 2026.
Is the RTX 5090 or RTX PRO 6000 better for local AI?
Both have 1.79 TB/s of bandwidth, so they generate tokens at similar speed when a model fits. The RTX PRO 6000 has three times the memory (96 GB vs 32 GB), which lets it hold 70B-class models at 8-bit, and runs FP16 with FP32 accumulate at full rate. The RTX 5090 costs far less. Check a specific model in the VRAM calculator or read the best GPU for local LLMs.
Why are TPU FP4 and FP64 cells empty?
Google does not publish FP4 or FP64 matrix throughput for these TPUs, and TPUs are not designed for FP64 scientific computing. The table only shows figures the vendor has published.
Sources
- NVIDIA Tesla V100 datasheet
- NVIDIA A100 datasheet
- NVIDIA H100 product page and H200 product page
- NVIDIA HGX platform specifications (HGX B200 and B300)
- AMD Instinct MI300X, MI325X and MI355X product pages
- Google Cloud TPU7x (Ironwood) documentation, which also lists v5p and v6e per-chip specs
- NVIDIA Vera Rubin NVL72, GB300 NVL72 and GB200 NVL72 product pages; TechRadar on the Rubin bandwidth revision
- NVIDIA L40S, L4, RTX PRO 6000 Blackwell, GeForce RTX 5090 and DGX Spark product pages
- AMD Instinct MI400 Series and MI250X product pages
- Intel Gaudi 3 AI accelerator white paper
- AWS Neuron documentation: Trainium3 architecture, which also gives Trainium2 figures
- Google Cloud TPU v5e documentation
Cite this dataset
TensorCore.si Editorial. (2026, November). AI GPU and accelerator specs database. TensorCore.si. https://tensorcore.si/gpu-specs/