Skip to content
,

CoWoS Explained: Advanced Chip Packaging and the Real Bottleneck Behind AI GPUs

Why does AI chip supply depend on packaging? An in-depth guide to TSMC's CoWoS (Chip-on-Wafer-on-Substrate), silicon interposers, HBM integration, CoWoS-S vs CoWoS-L vs CoWoS-R, chiplets, the reticle limit, competing technologies like Intel EMIB and Foveros, and what comes next.

5 min read

CoWoS and Advanced Packaging — cover image from TensorCore.si

Short answer

CoWoS (Chip-on-Wafer-on-Substrate) is TSMC's advanced packaging technology that places a GPU or accelerator die side by side with stacks of HBM memory on a silicon interposer, connecting them with thousands of ultra-fine wires. It is what lets chips like NVIDIA's H100, H200 and B200 and AMD's Instinct MI300X reach several terabytes per second of memory bandwidth. Because nearly every leading AI accelerator needs it, CoWoS capacity became a major bottleneck for AI chip supply from 2023 onwards, alongside HBM production itself.

Why packaging suddenly matters

For decades, chip progress came mainly from shrinking transistors. AI accelerators now hit two walls that transistor shrinking alone cannot fix:

1. The memory wall: AI workloads need far more memory bandwidth than traditional circuit-board memory can deliver. The answer is HBM: DRAM dies stacked vertically and placed millimetres from the processor with a very wide interface (1,024 bits per stack for HBM3/HBM3E, 2,048 for HBM4). 2. The reticle limit: lithography machines can only expose an area of about 26 × 33 mm (around 858 mm²) in one shot, which caps the size of a single die. The largest AI chips are already at that limit.

Advanced packaging solves both by joining multiple dies, logic and memory, into one package that behaves like a single, giant chip.

How CoWoS works

The name describes the layers from top to bottom:

1. Chip: the logic die (GPU) and HBM stacks. 2. on Wafer: these sit on a silicon interposer, a thin piece of silicon with very dense wiring layers (redistribution layers) and through-silicon vias (TSVs). The interposer is made on a wafer, hence the name. 3. on Substrate: the interposer is mounted on an organic package substrate that connects to the circuit board.

The interposer's wiring is much finer than any circuit board, allowing thousands of connections between the GPU and each HBM stack at high speed and low power. That is how an H100 reaches about 3.35 TB/s and a B200 about 8 TB/s.

CoWoS variants

VariantInterposerKey ideaExamples
CoWoS-SFull silicon interposerFinest wiring; size limited by interposer areaNVIDIA A100, H100; AMD MI300 family
CoWoS-ROrganic (RDL) interposerCheaper and more flexible, coarser wiringNetworking and some accelerators
CoWoS-LOrganic interposer with embedded silicon bridges (local silicon interconnect)Very large packages; silicon only where dense links are neededNVIDIA Blackwell B200/GB200 and later

Why Blackwell needed CoWoS-L

NVIDIA's Blackwell B200 joins two reticle-sized GPU dies with a 10 TB/s die-to-die link and eight HBM3E stacks in a single package. A full silicon interposer that large is impractical, so CoWoS-L uses small silicon bridges embedded in a larger organic interposer to link the dies and memory where dense wiring is needed. The two dies then appear to software as one GPU. See B200 vs H100 and GB200 NVL72 explained.

Why CoWoS became a bottleneck

  • Demand exploded: the AI boom from 2023 sent orders for H100-class GPUs far beyond plans.
  • Long lead times: building packaging lines takes many months, with specialised equipment.
  • Complexity and yield: combining several expensive dies means one defect can waste the whole package, so testing before assembly ("known good die") is critical.
  • Coupled supply: each package also needs HBM stacks from SK hynix, Samsung or Micron, which were in short supply too.

TSMC expanded CoWoS capacity aggressively, and outsourced assembly and test companies (OSATs) such as ASE and Amkor added capacity for parts of the process. Packaging and test constraints have also rippled into supply of ordinary chips, as covered in our report on the 2026 chip shortage.

Other advanced packaging technologies

TechnologyCompanyApproach
InFO (Integrated Fan-Out)TSMCFan-out packaging without a silicon interposer; used in phones and some HPC parts
SoICTSMCTrue 3D stacking of dies with direct copper-to-copper hybrid bonding
EMIBIntelSmall silicon bridges embedded in the substrate to link dies (similar idea to CoWoS-L)
FoverosIntel3D stacking of logic dies
I-Cube / X-CubeSamsung2.5D interposer and 3D stacking solutions
AMD 3D V-Cache and MI300AMD with TSMCStacking cache or compute dies on base dies via hybrid bonding

AMD's Instinct MI300X is a striking example: eight compute chiplets stacked on four I/O dies, combined with eight HBM3 stacks for 192 GB of memory, using both 3D stacking and 2.5D packaging. See MI355X vs B200.

2.5D vs 3D packaging

  • 2.5D: dies sit side by side on an interposer (CoWoS, EMIB). Easier thermal management; limited by interposer area.
  • 3D: dies stacked on top of each other (HBM itself, SoIC, Foveros). Shortest connections and highest density, but harder to cool and test.

Future AI chips combine both: 3D-stacked logic and memory placed on large 2.5D interposers.

What comes next

  • Bigger interposers: packages spanning several reticle sizes with more HBM stacks, for example in NVIDIA's Rubin generation and beyond.
  • HBM4 and custom base dies: HBM4 doubles the interface width, and its base die can be customised and made on logic processes, blurring memory and compute.
  • Co-packaged optics: placing optical connections inside the package to move data between chips with less power.
  • Panel-level packaging: using large rectangular panels instead of round wafers to make bigger interposers more efficiently.
  • Glass substrates: flatter, more stable substrates for very large packages.
  • Wafer-scale integration: the extreme approach taken by Cerebras, building a whole wafer as one chip. See Cerebras WSE-3.

Why this matters for GPU buyers and renters

Packaging and HBM capacity, not just chip fabrication, determine how many data center GPUs can be built. That shapes availability and pricing for H100, H200 and B200 GPUs in the cloud; follow daily rates in our GPU cloud price tracker and read why GPU prices are rising.

Frequently asked questions

What does CoWoS stand for?

Chip-on-Wafer-on-Substrate: chips are mounted on a silicon interposer (made on a wafer), which is then mounted on a package substrate.

Do consumer GPUs use CoWoS?

Generally no. Consumer graphics cards use GDDR memory on the circuit board, which does not need an interposer. CoWoS is used for HBM-based data center accelerators.

Is CoWoS only made by TSMC?

CoWoS is TSMC's brand. Other companies offer comparable 2.5D packaging, such as Intel's EMIB and Samsung's I-Cube, and OSATs handle some steps of the process.

Why not just make one bigger chip?

The reticle limit caps single-die size, and very large dies have low yields because one defect can ruin the whole die. Splitting a design into chiplets and packaging them together is cheaper and scales further.

Sources and further reading

Think you understand it now? Prove it.

Solve real Tensor Core math in the TensorCore.si Challenge and earn a certificate with your name.

Take the challenge

Leave a Reply

Your email address will not be published. Required fields are marked *