Edited by 
Short answer
GDDR7 and HBM (high bandwidth memory) are the two main types of graphics memory. GDDR7 chips sit on the circuit board around the GPU and use a fast, narrow interface; it powers consumer cards such as the RTX 50 series, with up to about 1.8 TB/s on an RTX 5090. HBM stacks DRAM dies vertically next to the GPU on the same package and uses a very wide interface; data center GPUs with HBM3E reach around 8 TB/s, and HBM4 goes higher. HBM gives far more bandwidth and capacity per GPU but costs much more, which is why it is used for AI accelerators.
Why GPU memory matters for AI
AI workloads constantly move huge amounts of data between memory and the compute units. When a large language model generates text, it must read every weight once per token, so speed is often limited by memory bandwidth rather than compute. The amount of memory, the VRAM, decides which models fit at all. The type of memory is therefore one of the most important differences between a gaming GPU and an AI accelerator.
What is GDDR7?
GDDR stands for graphics double data rate. GDDR memory chips are separate packages soldered onto the graphics card around the GPU. Each chip connects through a 32-bit channel, and a GPU combines many chips into a wide bus, such as 256 or 512 bits.
GDDR7 is the seventh generation. JEDEC published the standard in March 2024. Its main changes:
- PAM3 signalling: each signal carries three voltage levels instead of two, sending more data per clock than GDDR6's NRZ signalling.
- Higher speeds: first products run at around 28 to 32 Gbps per pin, with the standard allowing up to 48 Gbps in future.
- Better efficiency: lower voltage and improved power management per bit transferred.
The GeForce RTX 5090 uses 32 GB of GDDR7 on a 512-bit bus at 28 Gbps, giving 1,792 GB/s (512 / 8 x 28). That is about 78% more than the RTX 4090's GDDR6X.
What is HBM?
High bandwidth memory takes a completely different approach. Several DRAM dies are stacked on top of each other and connected by thousands of vertical wires called through-silicon vias (TSVs). Each stack sits right next to the GPU die on a silicon interposer, inside the same package, using advanced packaging such as TSMC's CoWoS.
Because the connection is so short and dense, each HBM stack has a very wide interface, 1,024 bits for HBM3E, running at a moderate per-pin speed. Wide and slow beats narrow and fast for both bandwidth and energy efficiency.
HBM generations
| Generation | Interface per stack | Approx. bandwidth per stack | Example GPUs |
|---|---|---|---|
| HBM2e | 1,024-bit | about 0.4 to 0.46 TB/s | A100 |
| HBM3 | 1,024-bit | about 0.8 TB/s | H100, MI300X |
| HBM3E | 1,024-bit | about 1.2 TB/s | H200, B200, MI325X |
| HBM4 | 2,048-bit | about 2 TB/s or more | NVIDIA Rubin and other 2026 accelerators |
The H200 carries 141 GB of HBM3E at 4.8 TB/s, and the B200 has 192 GB of HBM3E at 8 TB/s. JEDEC published the HBM4 standard in April 2025, doubling the interface width to 2,048 bits per stack. See our explainer on what HBM is for more on how stacking works.
GDDR7 vs HBM3E: head-to-head
| GDDR7 | HBM3E | |
|---|---|---|
| Where it sits | Separate chips on the board | Stacks inside the GPU package |
| Bus width per device | 32-bit per chip | 1,024-bit per stack |
| Per-pin speed | 28 to 32 Gbps (up to 48) | about 9 to 10 Gbps |
| Typical GPU bandwidth | 0.9 to 1.8 TB/s | 4.8 to 8 TB/s |
| Typical GPU capacity | 8 to 32 GB (up to 96 GB on workstation cards) | 141 to 288 GB |
| Energy per bit | Higher | Lower |
| Cost and complexity | Lower, standard PCB | Much higher, needs advanced packaging |
| Used in | Gaming and workstation GPUs | AI and HPC accelerators |
The workstation RTX PRO 6000 Blackwell shows GDDR7's upper limit for capacity: 96 GB using higher-density chips on both sides of the board, at about 1.8 TB/s. An HBM3E accelerator like the B200 has twice the capacity and more than four times the bandwidth on a single GPU.
Why AI accelerators use HBM
- Bandwidth: LLM inference and training are bandwidth hungry, and HBM delivers several times more per GPU.
- Capacity: large models need hundreds of gigabytes close to the compute; HBM stacks keep growing taller (8, 12 and 16 dies).
- Power: moving data a few millimetres inside a package costs less energy than driving signals across a circuit board, which matters in a data center running thousands of GPUs.
The downsides are cost and supply. HBM is made by only three companies (SK hynix, Samsung and Micron), and it depends on limited advanced packaging capacity. HBM supply has been one of the main bottlenecks in AI chip production, and it is a major reason data center GPUs cost tens of thousands of dollars.
Why gaming GPUs use GDDR7
For games, GDDR7 offers plenty of bandwidth at a reasonable price, and it lets card makers use ordinary circuit boards. Graphics workloads also benefit from large on-chip caches, which reduce how often the GPU needs main memory. GDDR7 also makes it easier to offer several memory sizes on one GPU design.
For local AI on a consumer GPU, the practical lesson is that capacity matters most: a model must fit in VRAM. Bandwidth then decides how fast it generates tokens. Our guide to the best GPU for local LLMs compares cards on both.
What about LPDDR and unified memory?
A third option is LPDDR, the low-power memory used in phones and laptops. Apple silicon Macs and NVIDIA's DGX Spark use LPDDR5X as unified memory shared by CPU and GPU. It offers large capacities (up to 128 GB or more) at lower bandwidth than GDDR7 or HBM, which makes these machines good at fitting big models but slower at generating tokens. NVIDIA's Grace CPU also uses LPDDR5X alongside HBM GPUs in Grace Blackwell systems.
Frequently asked questions
Is HBM faster than GDDR7?
Per GPU, yes. An HBM3E GPU has around 4 to 8 TB/s of bandwidth versus up to about 1.8 TB/s for GDDR7 cards. Per pin, GDDR7 is faster, but HBM's interface is far wider.
Will gaming GPUs ever use HBM?
AMD used HBM on some Radeon gaming cards between 2015 and 2019, but the cost was hard to justify. For now, HBM is reserved for data center products.
What memory does the RTX 5090 use?
32 GB of GDDR7 on a 512-bit bus, with 1,792 GB/s of bandwidth.
Sources and further reading
- JEDEC, GDDR7 standard announcement (March 2024)
- JEDEC, HBM4 standard announcement (April 2025)
- NVIDIA, GeForce RTX 5090 specifications
- NVIDIA, H200 Tensor Core GPU
Leave a Reply