Edited by 
Short answer
An LPU (Language Processing Unit) is Groq's AI inference chip. Instead of storing a model in large HBM memory like a GPU, it keeps the weights in fast on-chip SRAM spread across many chips, and a compiler schedules every operation in advance. That gives very high, predictable token-generation speed but little memory per chip, so large models need hundreds of chips. In December 2025 NVIDIA paid about $20 billion to license Groq's technology and hire its leaders, and in March 2026 it unveiled the Groq 3 LPX rack, built to handle the decode phase alongside Vera Rubin GPUs.
LPU vs GPU at a glance
| Groq 3 LPU (LP30) | NVIDIA Rubin GPU | |
|---|---|---|
| Memory per chip | 512 MB on-chip SRAM | 288 GB HBM4 |
| Memory bandwidth per chip | 150 TB/s | 19.2 TB/s |
| Dense FP8 compute | About 1.23 PFLOPS | 17.5 PFLOPS |
| Scheduling | Static, planned by the compiler | Dynamic, by hardware schedulers |
| Best at | Fast, low-latency token generation | Everything: training, prefill, decode, large batches |
| Weakness | Capacity: many chips per model | Decode limited by HBM bandwidth |
| Manufacturing | Samsung 4 nm | TSMC 3 nm class |
How an LPU works
A GPU keeps model weights in HBM next to the chip and streams them in for every token. During decode that stream is the bottleneck. Groq's design avoids it by keeping weights in SRAM on the chip itself, which is far faster than HBM, and by dividing a model across many chips connected directly to each other.
The second idea is determinism. Groq's compiler plans every data movement and calculation ahead of time, down to the clock cycle, so the chip needs no caches, no dynamic schedulers and no waiting on memory. Every run of the same workload takes the same time, which keeps latency low and predictable. The first-generation chip had about 230 MB of SRAM.
The trade-off: speed vs capacity
SRAM is fast but takes far more silicon per byte than HBM. A 70B model at 8-bit needs about 70 GB, which is a fraction of one Rubin GPU's memory but well over a hundred Groq 3 chips. The KV cache for long contexts and many users needs space too. LPUs therefore suit serving popular models to many users at very high speed per user, while GPUs remain more flexible and far better for training. GroqCloud reported about 840 tokens per second on Llama 3.1 8B and about 394 on Llama 3.3 70B in early 2026.
NVIDIA's Groq deal
- December 24, 2025: NVIDIA agreed to pay about $20 billion for a non-exclusive license to Groq's LPU technology, compiler and chip designs, and hired founder Jonathan Ross, president Sunny Madra and core engineers.
- Groq continues: NVIDIA did not buy the company. Groq remains independent and keeps running GroqCloud, now led by former finance chief Simon Edwards.
- March 2026: at GTC, NVIDIA unveiled the Groq 3 LPU and LPX rack, its first rack-scale product built around a non-GPU chip.
NVIDIA Groq 3 LPX
| Groq 3 LPX rack | Value |
|---|---|
| Chips | 256 Groq 3 LPUs (LP30) |
| On-chip SRAM | 128 GB total |
| Memory bandwidth | 40 PB/s total |
| FP8 compute | About 315 PFLOPS |
| Role | Decode, next to a Vera Rubin NVL72 rack that handles prefill |
| Shipping | Planned for the third quarter of 2026 |
NVIDIA's Dynamo software splits requests so that Rubin GPUs process prompts and build the KV cache, then LPUs generate the tokens. NVIDIA claims an LPX paired with Vera Rubin NVL72 delivers up to 35 times more throughput per megawatt than Blackwell NVL72 on trillion-parameter models; this is NVIDIA's figure for its chosen workloads. Future LP35 chips are planned to add NVFP4, and LP40 to follow with the Feynman generation.
Other inference chips
- Cerebras builds one chip from a whole wafer with very large SRAM, another way of keeping weights on chip.
- Google TPUs and AWS Trainium are custom accelerators with HBM, closer to GPUs in design. See TPU vs GPU.
- Compare memory and bandwidth across 28 chips in the specs database.
Frequently asked questions
Is an LPU better than a GPU?
For fast token generation on models it can hold, an LPU can be quicker per user. GPUs are far more flexible: they train models, handle long prompts and big batches efficiently, and run any framework. NVIDIA's own plan uses both together.
Did NVIDIA buy Groq?
No. NVIDIA licensed Groq's technology on a non-exclusive basis and hired its founder, president and core engineering team. Groq remains a separate company and continues to operate GroqCloud.
What does LPU stand for?
Language Processing Unit, Groq's name for its inference chip, designed around the sequential nature of generating language one token at a time.
Can I buy a Groq LPU?
Not as a card for a PC. You can use Groq chips through the GroqCloud API, and NVIDIA sells the Groq 3 LPX as a data-center rack through its partners.
