GPU Architecture
Inside the GPU: streaming multiprocessors, warps, CUDA cores, caches, HBM memory and the data paths that feed Tensor Cores.
How a GPU is built, and why memory matters as much as math
A GPU is thousands of small cores grouped into streaming multiprocessors and fed by very fast memory. This section covers how that hardware is organised, how software such as CUDA drives it, and why memory bandwidth often decides how fast AI runs.
Start with how a GPU works, then follow the path from memory and interconnects to how language models actually run on the hardware.
Start here
Tools for this topic
Edited by Sami Ullah Khan. Read our editorial policy.
All GPU Architecture articles
NVIDIA Rubin Explained: Vera Rubin NVL72 Specs, HBM4 and Rubin vs Blackwell
NVIDIA Rubin is the GPU after Blackwell: 288 GB of HBM4 at 19.2 TB/s, 35 PFLOPS of dense NVFP4 and NVLink 6, paired with…
GB200 NVL72 Explained: 72 GPUs, One NVLink Domain
GB200 NVL72 puts 72 Blackwell GPUs and 36 Grace CPUs in one liquid-cooled rack with 130 TB/s NVLink and 13.4 TB HBM3e. Specs, what…
TPU vs GPU: What Is the Difference for AI? (2026)
TPUs are Google's systolic-array AI chips; GPUs are general-purpose with Tensor Cores. Architecture, Ironwood vs B200 specs, scaling, software and which to choose.
KV Cache Explained: Why Long Context Uses So Much Memory
The KV cache stores keys and values for every past token. The formula, sizes for Llama, Qwen and DeepSeek, GQA vs MLA, and how…
NVLink and NVSwitch Explained: How AI GPUs Talk to Each Other
NVLink connects GPUs at up to 1.8 TB/s and NVSwitch lets 8 to 72 GPUs talk at full speed. Generations, NVL72, all-reduce math and…
What Is HBM? High Bandwidth Memory for AI GPUs Explained
HBM stacks DRAM dies beside the GPU on a 1,024-bit bus per stack. Learn how it works, every generation from HBM2 to HBM4, and…
Tensor Cores vs CUDA Cores: What's the Difference?
CUDA cores do one multiply-add on single numbers; Tensor Cores do a whole matrix tile at once. Here is how they differ, how much…
How a GPU Works: SMs, Warps and the Memory Hierarchy
A tour of a modern NVIDIA GPU from the top down: streaming multiprocessors, warps of 32 threads, registers, shared memory, L2 cache and HBM,…