The engine behind modern AI
See inside the Tensor Core, the chip that powers AI
Every chatbot reply and every AI image is billions of tiny matrix multiplications. Take a GPU apart layer by layer, watch the math run live, then prove you understand it and earn a certificate.
D = A × B + C
Since you opened this page, one H100 GPU at full speed could have done
Theoretical dense FP16 peak of one H100 SXM (989 TFLOPS). Token estimate assumes about 140 billion operations per token; real systems are slower. Where does that speed come from?
- 2017first Tensor Cores, in the Volta V100
- 64multiply-adds per clock in a first-gen Tensor Core
- 528Tensor Cores in an H100 SXM
- 989 TFLOPSdense FP16 on H100
- 9 PFLOPSdense FP4 per B200
- 288 GBof HBM3e on B300
- 8 TB/smemory bandwidth on Blackwell
- 4 bitsthe smallest Tensor Core float, FP4
Start here
Go from zero to Tensor Core expert in eight reads
A learning path in the order we recommend. Each guide builds on the one before and links to the original NVIDIA sources.
What is a Tensor Core?
What D = A × B + C means, why AI is mostly matrix math, and where Tensor Cores sit inside a GPU.
Read the guide →Tensor Cores vs CUDA Cores
One number at a time versus a whole tile at once, and why every AI model still needs both.
Read the guide →How a GPU works
Streaming multiprocessors, warps, shared memory and HBM, and why moving data is the real bottleneck.
Read the guide →FP16, BF16, FP8 and FP4
How smaller numbers make AI faster, with a live tool that rounds any value into every format.
Read the guide →Tensor Core generations
Volta, Turing, Ampere, Hopper, Blackwell and Rubin: the formats and features each one added.
Read the guide →Hopper vs Blackwell
H100, H200, B200 and B300 compared on memory, FP8 and FP4, and what it means for training and inference.
Read the guide →What is HBM?
The stacked memory beside every AI chip, why bandwidth limits chatbot speed, and HBM2 to HBM4.
Read the guide →Reading GPU spec sheets
Dense vs sparse TFLOPS, per-GPU vs per-server numbers, and how to estimate real training time.
Read the guide →From chip to chatbot
How AI actually runs on a computer
Seven diagrams that show what the processor, RAM, VRAM and GPU each do when you run an AI model, how they hand work to each other, how AI factories scale it to thousands of GPUs, and how AI agents put it all to work.
Learn AI from beginner to master →The future: superintelligence (SI) →
How AI works
A language model never sees words. It splits your text into tokens, turns each token into a list of numbers, pushes those numbers through dozens of layers of learned weights, and ends with a probability for every possible next token. It picks one, adds it to the text and runs again: one full pass for every word piece it writes.
- It's mostly matrix math. Each layer multiplies the token numbers by huge tables of weights. That is exactly the work Tensor Cores are built for.
- About 2 operations per weight per token. An 8-billion-parameter model needs roughly 16 billion math operations for every token it outputs.
- Training vs inference. Training learns the weights and takes thousands of GPUs for weeks. Inference only uses them, and small models run on one graphics card.
The processor (CPU)
The CPU is the computer's general-purpose brain: a handful of fast, flexible cores, typically 6 to 24 on a desktop, built for branching logic and step-by-step work. In AI it is the conductor, not the orchestra: it runs the operating system and the app, reads the model file, turns your text into tokens and tells the GPU what to calculate.
- Few smart cores vs thousands of simple ones. An RTX 5090 has 21,760 CUDA cores and 680 Tensor Cores working in parallel. That is why the heavy math moves to the GPU.
- Can a CPU run AI alone? Yes, small models, slowly. Its speed is capped by RAM bandwidth, around 60–100 GB/s on dual-channel DDR5, roughly 20 times less than a high-end GPU's VRAM.
- NPUs. Many new laptop processors add a small neural processing unit for low-power AI features like background blur and live captions.
RAM (system memory)
RAM is the computer's working memory. Before a model can run, its file is read from the SSD into RAM and then copied across the PCIe slot into the GPU's VRAM. While it runs, RAM holds the operating system, the app and anything that did not fit on the GPU.
- Have more RAM than the model file. A 40 GB model on a machine with 32 GB of RAM loads slowly or not at all.
- Spill-over is slow. If a model does not fit in VRAM, tools like llama.cpp and Ollama keep some layers in RAM and run them on the CPU. It works, but those layers run at RAM speed, so generation slows sharply.
- Unified memory (Apple silicon, NVIDIA DGX Spark) is one shared pool for CPU and GPU. DGX Spark has 128 GB at 273 GB/s, so big models fit but run slower than on a GDDR7 card.
VRAM (GPU memory)
VRAM is the graphics card's own memory, soldered millimetres from the GPU chip. It is the single most important spec for running AI locally: capacity decides whether a model runs at all, and bandwidth decides how fast it writes.
- What has to fit: the model weights, the KV cache (which grows with every token of context) and some overhead for the software.
- Speed rule of thumb: tokens per second ≈ bandwidth × 0.7 ÷ model size. On an RTX 5090: 1,792 GB/s × 0.7 ÷ 4.9 GB ≈ 250 tokens per second.
- HBM in data centers. Chips like the B200 stack memory on the package for 8 TB/s, and NVIDIA's Rubin reaches 19.2 TB/s with HBM4.
How they work together
Here is what happens between pressing Enter and seeing an answer on a PC with a graphics card:
- Load: the model file is read from the SSD into RAM.
- Copy: the weights cross PCIe into VRAM, once, when the model starts.
- Prepare: the CPU tokenizes your prompt and queues work for the GPU.
- Prefill: the GPU reads the whole prompt at once. This step is limited by compute, so Tensor Cores do the heavy lifting.
- Decode: for every new token, the GPU reads all the weights from VRAM again. This step is limited by memory bandwidth.
- Output: each token returns to the CPU, becomes text and appears on screen.
The AI factory
An AI factory is a data center built for one job: turning electricity and data into tokens, meaning answers, code, images and trained models. NVIDIA popularised the name. The building blocks stack up: GPU → server → rack → pod → data center.
- Scale-up inside a rack. NVLink joins GPUs so they act like one giant chip. A GB200 NVL72 rack links 72 Blackwell GPUs and 36 Grace CPUs.
- Scale-out between racks. Leaf and spine switches running InfiniBand or Ethernet, at 400 to 800 Gb/s per GPU, connect thousands of racks into one cluster.
- Power and cooling are the limit. A GB200 NVL72 rack draws about 120 kW and needs liquid cooling. The largest new sites are planned in hundreds of megawatts to gigawatts.
AI agents
An AI agent is a language model that works in a loop. Give it a goal and it plans a step, acts with a tool such as web search, code or an app, observes the result, remembers what it learned and decides what to do next, until the task is done.
- Chatbot vs agent. A chatbot answers one prompt. An agent can make dozens of model calls to finish one task.
- Why hardware matters. Each step re-reads a growing history, so agents need lots of fast VRAM for the KV cache and quick prompt processing (prefill).
- Who does what. The tools run on ordinary CPUs and servers. The thinking runs on GPUs or other AI accelerators.
Why VRAM matters: the memory speed ladder
Peak bandwidth from storage to the fastest AI chips. Generating text means reading the whole model for every token, so the further down this list your model lives, the faster it writes. Bars use a log scale: each step on the scale is about 10 times faster.
Live simulation
Watch a Tensor Core do in one step what takes 64
Real numbers, computed live in your browser. First a regular core works through D = A × B + C one multiply-add at a time. Then a Tensor Core does the whole tile at once. Then a real AI layer splits into thousands of tiles.
Simulation lab
Learn AI hardware by running it
Interactive simulators that show what happens inside a GPU and a language model, with real chip specs and every calculation visible. Use your mouse, or click a simulator and drive it with the keyboard.
Bits vs speed
Fewer bits, more AI per second
Pick a number format and a GPU. Each halving of bits roughly doubles Tensor Core throughput and halves the memory a model needs.
Format
GPU
How the bits are split
Peak figures from NVIDIA specifications. Real workloads reach a fraction of peak.
TensorCore.si Challenge
Solve the math behind AI. Earn your certificate.
17 problems in three levels, from one matrix cell to sizing a 405-billion-parameter model. Every attempt has new numbers. Score 80% and download a certificate with your name and a unique ID anyone can verify.
Foundations
Matrix multiply-accumulate, FMA counts, FP4 rounding and peak TFLOPS. LEVEL 2 · 6 PROBLEMSEngineer
FP8 throughput, 2:4 sparsity, arithmetic intensity, INT8 and attention. LEVEL 3 · 6 PROBLEMSArchitect
Training GPU-days, the roofline model, KV cache and mixture-of-experts.New · TensorCore.si Arcade
Learn GPU tech by playing
Five two-minute games that turn the hardest ideas in AI hardware into something you can feel: matrix math, number formats, memory limits, chip history and what each core does.
How it works
One instruction, a whole block of math
A regular core does one multiply-add at a time. A Tensor Core takes small matrix tiles and computes D = A × B + C in a single operation. Big AI layers are thousands of these tiles.
Load the tiles
Matrices A and B arrive in a low-precision format such as FP16, FP8 or FP4. The running total C is kept in higher precision, usually FP32.
Multiply in parallel
Every row of A meets every column of B at once. A first-generation Tensor Core did a 4×4×4 block, 64 fused multiply-adds, every clock.
Accumulate the result
The products are summed and added to C in full precision, giving D. That result feeds the next tile, layer after layer.
Generations
Smaller numbers, faster math, every two years
Each generation added lower-precision formats and smarter ways to feed data, from FP16 in 2017 to FP4 and HBM4 today.
Volta
The first Tensor Cores: 640 in the V100, each doing 64 multiply-adds per clock for 125 TFLOPS.
Turing
Integer formats for fast inference, and Tensor Cores in GeForce RTX cards for DLSS.
Ampere
TF32, BF16, FP64 Tensor Cores and 2:4 structured sparsity. A100 reaches 312 TFLOPS dense.
Hopper
FP8 and the Transformer Engine. H100 hits 989 TFLOPS dense FP16 and 1,979 in FP8.
Blackwell
Two dies, native FP4 with block scaling and 9 PFLOPS dense FP4 per B200.
Rubin
288 GB of HBM4 per GPU. NVIDIA claims 50 PFLOPS of NVFP4 inference. Shipping in the second half of 2026.
AI Model Explorer
What does it take to run Llama, DeepSeek or gpt-oss?
Pick a model and a GPU to see the memory, the Tensor Core work per token and how fast it can generate text on H100, H200, B200 and B300.
GPU Cloud Price Tracker · Updated daily
What does an H100 cost per hour right now?
Compare on-demand prices for H100, H200, B200 and A100 across 31 cloud providers, from AWS and Google Cloud to RunPod and Lambda, and estimate your bill with the cost calculator. Prices refresh every day.
Free tools and data
Look up any AI chip in seconds
Sourced numbers you can trust and cite: every major AI accelerator compared, number formats decoded bit by bit, and the statistics behind Tensor Cores.
Quick check: can your GPU run it?
Pick a model, a precision and a GPU. Estimate for one user with an 8K-token context.
Explore topics
Everything about AI hardware, in one place
Tensor Cores →
How matrix engines work, generation by generation, from FP16 to FP4.GPU Architecture →
SMs, warps, CUDA cores, caches and the memory that feeds them.AI Chips →
H100, H200, B200, B300 and Rubin compared on compute and memory.Guides →
Precision formats, quantization, sparsity and the basics, explained simply.Simulations →
Interactive explainers that show the math inside the hardware.Glossary →
Every AI hardware term from accumulator to warp, defined in plain language.From the blog
Latest articles

Which Jobs Will Superintelligence Replace? 150+ Jobs Ranked by Risk, Why, and What to Do
An honest, detailed guide to how superintelligence could affect work: more than 150 jobs across 16 sectors ranked by risk, the specific…

What Happens When We Achieve Superintelligence? The First Day, First Year and Next Decade
What would actually happen in the hours, months and years after superintelligence is created? A detailed scenario analysis covering takeoff speed, who…

What Could Superintelligence Do? Its Real Capabilities and the Physical Limits It Cannot Break
What could a superintelligent AI actually do, and what would stay impossible even for it? A detailed look at SI capabilities in…

Positive and Negative Impacts of Superintelligence: A Complete Sector-by-Sector Analysis
What would superintelligence mean for health, work, the economy, science, climate, security, democracy, inequality and human meaning? A balanced, sector-by-sector analysis of…

How Could We Achieve Superintelligence? 8 Pathways, 6 Bottlenecks and the Signals to Watch
A detailed, evidence-based map of how artificial superintelligence could be built: eight technical pathways from scaling and reinforcement learning to automated AI…

Speculative Decoding Explained: How LLMs Generate Text 2–3× Faster Without Losing Quality
Speculative decoding speeds up LLM inference by letting a small draft model guess several tokens ahead and having the big model verify…
FAQ
Tensor Core questions, answered
What is a Tensor Core?
A Tensor Core is a specialised unit inside NVIDIA GPUs that multiplies small blocks of numbers (matrices) and adds the result to a running total in a single operation, written D = A × B + C. Neural networks are mostly matrix multiplication, so Tensor Cores do the bulk of the work when AI models are trained and run.
How is a Tensor Core different from a CUDA core?
A CUDA core performs one multiply-add on single numbers per clock. A Tensor Core performs a whole small matrix multiply-accumulate at once. The first-generation Tensor Core handled a 4×4×4 block, 64 multiply-adds, in each clock. CUDA cores still run general code; Tensor Cores accelerate the matrix math.
Which GPUs have Tensor Cores?
Every NVIDIA data-center GPU since Volta (V100, 2017), including T4, A100, H100, H200, B200 and B300, and every GeForce RTX and professional RTX graphics card since the RTX 20 series in 2018.
Why do Tensor Cores use low precision like FP8 and FP4?
Fewer bits per number means more numbers fit through the hardware and memory each second. Each halving of precision roughly doubles peak throughput, and modern scaling techniques keep model accuracy close to higher-precision results. Tensor Cores still accumulate results in higher precision to limit rounding error.
Do AMD, Google and Intel chips have Tensor Cores?
"Tensor Core" is NVIDIA's name. Other chips have equivalent matrix units: AMD Instinct GPUs have Matrix Cores, Google TPUs use matrix multiply units built as systolic arrays, and Intel GPUs have XMX engines.
How do I earn a TensorCore.si certificate?
Open the TensorCore.si Challenge, pick a level (Foundations, Engineer or Architect) and solve its randomized problems. Score 80% or more and you can download a certificate with your name, score and a unique ID that anyone can verify on the same page. It is free and needs no account.
Is TensorCore.si affiliated with NVIDIA?
No. TensorCore.si is an independent educational website. NVIDIA, CUDA and Tensor Core are trademarks of NVIDIA Corporation.
Understand the hardware that powers AI
Start with the plain-language guide, then put your knowledge to the test.