Skip to content

The engine behind modern AI

See inside the Tensor Core, the chip that powers AI

Every chatbot reply and every AI image is billions of tiny matrix multiplications. Take a GPU apart layer by layer, watch the math run live, then prove you understand it and earn a certificate.

D = A × B + C

Free, no sign-upSourced from NVIDIA docsVerifiable certificates
Exploded view of a GPU package. Conceptual, not to scale.
Hover the chip to take it apart
Point at a layer or its label to see what that part does.
Live

Since you opened this page, one H100 GPU at full speed could have done

0floating-point operations on its Tensor Cores
0years for a person doing one sum per second
0tokens a 70B-parameter model could generate

Theoretical dense FP16 peak of one H100 SXM (989 TFLOPS). Token estimate assumes about 140 billion operations per token; real systems are slower. Where does that speed come from?

  • 2017first Tensor Cores, in the Volta V100
  • 64multiply-adds per clock in a first-gen Tensor Core
  • 528Tensor Cores in an H100 SXM
  • 989 TFLOPSdense FP16 on H100
  • 9 PFLOPSdense FP4 per B200
  • 288 GBof HBM3e on B300
  • 8 TB/smemory bandwidth on Blackwell
  • 4 bitsthe smallest Tensor Core float, FP4

From chip to chatbot

How AI actually runs on a computer

Seven diagrams that show what the processor, RAM, VRAM and GPU each do when you run an AI model, how they hand work to each other, how AI factories scale it to thousands of GPUs, and how AI agents put it all to work.

Learn AI from beginner to master →The future: superintelligence (SI) →

How a language model turns a prompt into an answerText is split into tokens, each token becomes a vector of numbers, the vectors pass through 32 transformer layers of learned weights, and the model outputs probabilities for the next token, then repeats.“Is the RTX 5090 good for AI?”1 TOKENIZE · text → word piecesIstheRTX5090goodforAI?2 EMBED · each token → 4,096 numbers3 TRANSFORMER LAYERS × 32Attentiontokens share contextFeed-forward× learned weights8 billionweights in VRAM,multiplied onTensor Cores4 PREDICT · probability of next tokenYes62%It18%The9%add “Yes” and repeat

How AI works

A language model never sees words. It splits your text into tokens, turns each token into a list of numbers, pushes those numbers through dozens of layers of learned weights, and ends with a probability for every possible next token. It picks one, adds it to the text and runs again: one full pass for every word piece it writes.

  • It's mostly matrix math. Each layer multiplies the token numbers by huge tables of weights. That is exactly the work Tensor Cores are built for.
  • About 2 operations per weight per token. An 8-billion-parameter model needs roughly 16 billion math operations for every token it outputs.
  • Training vs inference. Training learns the weights and takes thousands of GPUs for weeks. Inference only uses them, and small models run on one graphics card.

What is a Tensor Core?Prefill vs decodeGlossary

Why VRAM matters: the memory speed ladder

Peak bandwidth from storage to the fastest AI chips. Generating text means reading the whole model for every token, so the further down this list your model lives, the faster it writes. Bars use a log scale: each step on the scale is about 10 times faster.

NVMe SSD (PCIe 5.0)
14 GB/s
PCIe 5.0 x16 slot
64 GB/s
DDR5 RAM, dual-channel
90 GB/s
DGX Spark unified memory
273 GB/s
RTX 4090 · GDDR6X
1.01 TB/s
RTX 5090 · GDDR7
1.79 TB/s
H100 · HBM3
3.35 TB/s
B200 · HBM3e
8 TB/s
Rubin · HBM4
19.2 TB/s
StorageConnectionSystem / unified RAMGraphics card VRAMData-center HBM

Live simulation

Watch a Tensor Core do in one step what takes 64

Real numbers, computed live in your browser. First a regular core works through D = A × B + C one multiply-add at a time. Then a Tensor Core does the whole tile at once. Then a real AI layer splits into thousands of tiles.

More simulations

Bits vs speed

Fewer bits, more AI per second

Pick a number format and a GPU. Each halving of bits roughly doubles Tensor Core throughput and halves the memory a model needs.

Format

GPU

How the bits are split

SignExponent (range)Mantissa (precision)
Dense Tensor Core peak989 TFLOPS
Weights of a 70B-parameter model140 GB
GPUs needed just to hold them2

Peak figures from NVIDIA specifications. Real workloads reach a fraction of peak.

TensorCore.si Challenge

Solve the math behind AI. Earn your certificate.

17 problems in three levels, from one matrix cell to sizing a 405-billion-parameter model. Every attempt has new numbers. Score 80% and download a certificate with your name and a unique ID anyone can verify.

How it works

One instruction, a whole block of math

A regular core does one multiply-add at a time. A Tensor Core takes small matrix tiles and computes D = A × B + C in a single operation. Big AI layers are thousands of these tiles.

Step 1

Load the tiles

Matrices A and B arrive in a low-precision format such as FP16, FP8 or FP4. The running total C is kept in higher precision, usually FP32.

Step 2

Multiply in parallel

Every row of A meets every column of B at once. A first-generation Tensor Core did a 4×4×4 block, 64 fused multiply-adds, every clock.

Step 3

Accumulate the result

The products are summed and added to C in full precision, giving D. That result feeds the next tile, layer after layer.

See a worked example in What is a Tensor Core? →

AI Model Explorer

What does it take to run Llama, DeepSeek or gpt-oss?

Pick a model and a GPU to see the memory, the Tensor Core work per token and how fast it can generate text on H100, H200, B200 and B300.

GPU Cloud Price Tracker · Updated daily

What does an H100 cost per hour right now?

Compare on-demand prices for H100, H200, B200 and A100 across 31 cloud providers, from AWS and Google Cloud to RunPod and Lambda, and estimate your bill with the cost calculator. Prices refresh every day.

Free tools and data

Look up any AI chip in seconds

Sourced numbers you can trust and cite: every major AI accelerator compared, number formats decoded bit by bit, and the statistics behind Tensor Cores.

See all 9 free tools →

Quick check: can your GPU run it?

Pick a model, a precision and a GPU. Estimate for one user with an 8K-token context.

From the blog

Latest articles

View all posts

FAQ

Tensor Core questions, answered

What is a Tensor Core?

A Tensor Core is a specialised unit inside NVIDIA GPUs that multiplies small blocks of numbers (matrices) and adds the result to a running total in a single operation, written D = A × B + C. Neural networks are mostly matrix multiplication, so Tensor Cores do the bulk of the work when AI models are trained and run.

How is a Tensor Core different from a CUDA core?

A CUDA core performs one multiply-add on single numbers per clock. A Tensor Core performs a whole small matrix multiply-accumulate at once. The first-generation Tensor Core handled a 4×4×4 block, 64 multiply-adds, in each clock. CUDA cores still run general code; Tensor Cores accelerate the matrix math.

Which GPUs have Tensor Cores?

Every NVIDIA data-center GPU since Volta (V100, 2017), including T4, A100, H100, H200, B200 and B300, and every GeForce RTX and professional RTX graphics card since the RTX 20 series in 2018.

Why do Tensor Cores use low precision like FP8 and FP4?

Fewer bits per number means more numbers fit through the hardware and memory each second. Each halving of precision roughly doubles peak throughput, and modern scaling techniques keep model accuracy close to higher-precision results. Tensor Cores still accumulate results in higher precision to limit rounding error.

Do AMD, Google and Intel chips have Tensor Cores?

"Tensor Core" is NVIDIA's name. Other chips have equivalent matrix units: AMD Instinct GPUs have Matrix Cores, Google TPUs use matrix multiply units built as systolic arrays, and Intel GPUs have XMX engines.

How do I earn a TensorCore.si certificate?

Open the TensorCore.si Challenge, pick a level (Foundations, Engineer or Architect) and solve its randomized problems. Score 80% or more and you can download a certificate with your name, score and a unique ID that anyone can verify on the same page. It is free and needs no account.

Is TensorCore.si affiliated with NVIDIA?

No. TensorCore.si is an independent educational website. NVIDIA, CUDA and Tensor Core are trademarks of NVIDIA Corporation.

Understand the hardware that powers AI

Start with the plain-language guide, then put your knowledge to the test.