Free tool · LLM training estimates
AI training time and cost calculator
How long does it take, and how much does it cost, to train a language model? Enter the model size, the number of training tokens and your GPUs to estimate total compute, GPU-hours, days, cost and energy, using the standard 6 × parameters × tokens rule.
- Training compute ≈ 6 × N × D FLOPs, where N is the parameter count (active parameters for mixture-of-experts models) and D is the number of training tokens.
- Time = compute ÷ (GPUs × peak FLOPS × MFU). Model FLOPs utilization (MFU) is typically 35–50% for large runs.
- Check: Meta reports about 3.8 × 10²⁵ FLOPs for Llama 3.1 405B on 15.6 trillion tokens, which is exactly 6 × 405B × 15.6T.
Compute-optimal ≈ 20 × parameters.
Prices are editable examples of on-demand cloud rates, not quotes; reserved and spot prices are often much lower.
How real training runs compare
| Model | Parameters | Training tokens | Compute (6ND) | Reported hardware |
|---|---|---|---|---|
| GPT-3 | 175B | 300B | 3.1 × 10²³ | NVIDIA V100 cluster |
| Chinchilla | 70B | 1.4T | 5.9 × 10²³ | Google TPU v3 and v4 |
| Llama 3.1 8B | 8B | 15T+ | 7.2 × 10²³ | NVIDIA H100 |
| Llama 3.1 70B | 70B | 15T+ | 6.3 × 10²⁴ | NVIDIA H100 |
| Llama 3.1 405B | 405B | 15.6T | 3.8 × 10²⁵ | Up to 16,384 H100 GPUs |
Compute uses the 6ND approximation; published totals can differ slightly because of attention FLOPs and data mixtures.
Frequently asked questions
Why 6 × parameters × tokens?
Each token costs about 2 FLOPs per parameter in the forward pass (one multiply and one add) and about 4 in the backward pass, which computes gradients for both activations and weights. That totals roughly 6 FLOPs per parameter per token, as derived by Kaplan et al. (2020).
What MFU should I use?
Model FLOPs utilization is the share of peak Tensor Core throughput a run actually achieves. Large, well-tuned runs report 35–50%: Google reported 46.2% for PaLM and Meta 38–43% for Llama 3 405B. Small clusters with good code can do better; poorly tuned or communication-heavy runs often get 20–30%.
How many tokens should I train on?
The Chinchilla study found that compute-optimal training uses about 20 tokens per parameter. Modern models deliberately train far longer, such as 15 trillion tokens for Llama 3.1 8B (almost 1,900 tokens per parameter), because a smaller model that was trained longer is cheaper to run.
Does FP8 halve training time?
FP8 doubles the peak Tensor Core throughput on Hopper and Blackwell, but not everything runs in FP8, so real speedups are usually smaller. This calculator applies your MFU to whichever peak you choose.
Does the cost include everything?
No. It covers GPU rental only. Storage, networking, failed runs, experiments, evaluation and staff usually add substantially to the total.
Learn where peak numbers come from in how to read a GPU spec sheet, compare hardware in the GPU specs database, or check memory needs in the LLM VRAM calculator.
Sources
- Kaplan et al., Scaling Laws for Neural Language Models (2020)
- Hoffmann et al., Training Compute-Optimal Large Language Models (Chinchilla, 2022)
- Meta, The Llama 3 Herd of Models (2024)
- Chowdhery et al., PaLM (2022)
- Brown et al., Language Models are Few-Shot Learners (GPT-3, 2020)
Cite this tool
TensorCore.si Editorial. (2026, November). AI training time and cost calculator. TensorCore.si. https://tensorcore.si/training-cost-calculator/