Edited by 
Short answer
An AI data center is a building, or a campus of buildings, designed to train and run AI models on tens of thousands to millions of GPUs or other AI accelerators. Compared with a traditional data center it packs far more power into each rack (around 120 kW for an NVIDIA GB200 NVL72 rack versus roughly 5 to 15 kW for typical server racks), needs liquid cooling, and links its chips with very fast networks so they can work as one machine. The largest sites under construction in 2026 draw more than a gigawatt each, about as much as a large nuclear reactor produces.
What does an AI data center do?
An AI data center does two jobs.
- Training: building a model by feeding it huge amounts of data and adjusting its billions of parameters. Training a frontier model ties up tens of thousands of GPUs for weeks or months, all working on the same job and exchanging results every fraction of a second.
- Inference: running a trained model to answer requests, such as generating a chatbot reply, an image or a line of code. Each request is small, but billions of them arrive every day, so inference now uses a large and growing share of AI computing.
NVIDIA calls these facilities "AI factories": data goes in, and what comes out is intelligence measured in tokens. The phrase captures the main shift. A traditional data center stores and serves information for many separate customers and apps. An AI data center is closer to one giant computer built for one kind of work.
AI data center vs traditional data center
| Traditional data center | AI data center | |
|---|---|---|
| Main hardware | CPU servers, storage arrays | GPUs or AI accelerators (NVIDIA, AMD, Google TPU, AWS Trainium) plus CPUs |
| Power per rack | About 5 to 15 kW | About 40 kW for air-cooled GPU servers, 120 kW and rising for liquid-cooled racks |
| Cooling | Mostly air | Direct-to-chip liquid cooling for the densest racks |
| Network | Ethernet for general traffic | A separate high-speed "back-end" GPU network (InfiniBand or AI Ethernet) as well as the normal front-end network |
| Workload | Many small, independent jobs | A few huge jobs spread across thousands of chips, plus high-volume inference |
| Site size | Tens of megawatts | Hundreds of megawatts to more than a gigawatt |
What is inside an AI data center
1. Compute
The heart of the building is rows of GPU servers. A standard building block is an eight-GPU server such as an NVIDIA HGX B200 system, or a whole rack acting as one unit, such as the GB200 NVL72, which joins 72 Blackwell GPUs and 36 Grace CPUs with NVLink. Read GB200 NVL72 explained for how a rack-scale system works. Other operators use AMD Instinct GPUs, Google TPUs or Amazon Trainium chips.
2. Networking
AI training only works if the GPUs can share data constantly. Inside a server or rack, GPUs talk over NVLink at up to 1.8 TB/s per GPU. Between racks, each GPU usually gets its own 400 or 800 Gb/s network port on a dedicated back-end network built from InfiniBand or Ethernet designed for AI, such as NVIDIA Spectrum-X. A large cluster needs tens of thousands of optical cables and multiple layers of switches. A separate front-end network connects the cluster to storage and the internet. Read what is a GPU cluster and NVLink and NVSwitch explained for more.
3. Storage
Training datasets and model checkpoints run to petabytes. Fast parallel file systems on NVMe flash feed data to the GPUs and save checkpoints regularly so a failed job can resume instead of restarting.
4. Power
Power is the biggest constraint on AI data centers today. A gigawatt campus needs its own high-voltage substation and often its own generation. Operators are signing deals for natural gas turbines, solar and battery farms, and nuclear power, and some sites run on-site gas plants because grid connections can take years.
5. Cooling
Almost all the electricity a GPU uses turns into heat. At 1,000 W or more per GPU, air cannot carry it away fast enough, so dense racks use direct-to-chip liquid cooling: cold plates sit on the chips and a coolant loop carries the heat to cooling towers or dry coolers. Efficient sites aim for a power usage effectiveness (PUE) close to 1.1, meaning only about 10% of power goes to cooling and other overhead.
How much power does an AI data center use?
A lot, and the numbers are rising quickly. According to the International Energy Agency, data centers worldwide used about 415 TWh of electricity in 2024, roughly 1.5% of global electricity, and that figure is projected to more than double to about 945 TWh by 2030, with AI the main driver.
Individual sites now plan in gigawatts. One gigawatt running all year is about 8.8 TWh, roughly the yearly electricity use of 800,000 average US homes. NVIDIA's Jensen Huang has estimated the total cost of building a 1 GW AI factory at about $50 to 60 billion, most of it in chips and networking.
Examples of large AI data centers in 2026
- OpenAI Stargate, Abilene, Texas: built by Crusoe for OpenAI and Oracle. Epoch AI estimated in April 2026 that about 0.3 GW was operating, with a planned 1.2 GW across eight buildings of NVIDIA Blackwell GPUs, around one million H100-equivalents when complete. It is one of seven US Stargate sites planned to exceed 9 GW in total by 2029.
- xAI Colossus, Memphis, Tennessee: built to train the Grok models. It went from an empty factory to 100,000 H100 GPUs in 2024 and has since been expanded, with a second Colossus site under way.
- Meta Prometheus and Hyperion: Meta's planned multi-gigawatt clusters in Ohio and Louisiana; Hyperion is designed to scale toward 5 GW.
These numbers change fast, and capacity figures are often estimates or plans. Check the sources below for the latest.
Why AI data centers matter
The size of the AI models we get, and how fast and cheaply they answer, depends directly on how much computing power exists. Training a model ten times larger needs roughly ten times more compute, and serving hundreds of millions of users needs huge inference fleets. That is why chip supply, power and cooling, not algorithms alone, now set the pace of AI progress. Read how much it costs to train an LLM for the economics, and how AI works for what all this hardware is computing.
Frequently asked questions
What is the difference between a data center and an AI data center?
A regular data center runs many independent applications on CPU servers. An AI data center is built around GPUs or AI accelerators, uses several times more power per rack, needs liquid cooling and a dedicated high-speed network, and often runs a single job across thousands of chips.
What is an AI factory?
"AI factory" is NVIDIA's term for an AI data center whose purpose is producing intelligence: training models and generating tokens at scale. It is a marketing term, but it describes the shift from storing data to manufacturing AI output.
How many GPUs are in an AI data center?
From a few thousand in a regional inference site to hundreds of thousands in the largest training campuses. Gigawatt-scale sites are designed for several hundred thousand to around a million GPUs.
Why do AI data centers use so much water?
Some cool with evaporative cooling towers, which consume water. Many new AI sites use closed-loop liquid cooling and dry coolers that use little water, at the cost of more electricity on hot days.
Sources and further reading
- International Energy Agency, Energy and AI report
- Epoch AI, OpenAI Stargate: where the US sites stand (April 2026)
- NVIDIA, GB200 NVL72
- Scientific American, AI will drive doubling of data center energy demand by 2030
Leave a Reply