Short answer
A large language model (LLM) is an AI system trained on enormous amounts of text to predict the next word (more precisely, the next token) in a sequence. By repeating that prediction over and over, it can answer questions, write, translate, summarise and write code. Modern LLMs are built on the transformer architecture, have billions to trillions of parameters, and are trained on thousands of GPUs. Well-known examples include OpenAI's GPT models, Anthropic's Claude, Google's Gemini, Meta's Llama, DeepSeek and Qwen.
What does "large language model" mean?
- Large: the model has a huge number of parameters, the adjustable numbers it learns during training. Small models have around 1 billion; frontier models are believed to have hundreds of billions to trillions.
- Language: it is trained mainly on text, such as web pages, books, articles and code.
- Model: it is a mathematical function, a neural network, that turns input text into a prediction of what comes next.
An LLM does not look up answers in a database. Everything it "knows" is stored implicitly in its parameters, learned from patterns in its training data. If you are new to the basics, start with what a neural network is.
How does an LLM work?
1. Text becomes tokens
The model splits text into tokens, small chunks that are often whole words or parts of words. "Tensor cores are fast" might become five or six tokens. A common rule of thumb is that 100 tokens is about 75 English words.
2. Tokens become numbers
Each token is converted into a long list of numbers called an embedding, a point in a high-dimensional space where words with similar meanings sit close together.
3. The transformer processes the context
The embeddings pass through dozens of transformer layers. In each layer, attention lets every token look at the other tokens in the context and decide which ones matter, so "bank" can mean a river bank or a money bank depending on the sentence. Our guide to what a transformer is explains this in detail.
4. The model predicts the next token
At the end, the model outputs a probability for every possible next token. One is chosen, often with some randomness controlled by settings such as temperature (try our LLM sampling simulator), then added to the text, and the whole process repeats. This loop is why chatbots appear to "type" their answers.
How are LLMs trained?
| Stage | What happens | Data |
|---|---|---|
| Pre-training | Learn to predict the next token across a huge dataset | Trillions of tokens of web text, books and code |
| Supervised fine-tuning | Learn to follow instructions and hold conversations | Example prompts and high-quality answers |
| Preference tuning (RLHF and similar) | Learn which answers people prefer, and to be helpful and safe | Human or AI rankings of responses |
| Reasoning training | Learn to think step by step on problems with checkable answers | Maths, code and logic tasks |
Pre-training is by far the most expensive step: frontier models are trained on clusters of tens of thousands of GPUs for months. See how much it costs to train an LLM and AI alignment for the preference-tuning side.
Examples of LLMs
| Family | Developer | Open weights? |
|---|---|---|
| GPT | OpenAI | Mostly no (some open-weight models such as gpt-oss) |
| Claude | Anthropic | No |
| Gemini | No (smaller Gemma models are open) | |
| Llama | Meta | Yes |
| DeepSeek | DeepSeek | Yes |
| Qwen | Alibaba | Yes |
| Mistral | Mistral AI | Some |
Open-weight models can be downloaded and run on your own computer; see how to run an LLM locally.
What hardware do LLMs need?
LLMs run on GPUs and other AI accelerators because they are made of huge matrix multiplications, the work tensor cores are built for. Two numbers matter most:
- Memory: the model's weights must fit in GPU memory. An 8-billion-parameter model needs about 16 GB at FP16, or about 5 GB at 4-bit. See how much VRAM you need for an LLM.
- Memory bandwidth: generating each token means reading the weights, so speed depends on GPU memory bandwidth.
What LLMs can and cannot do
Strengths: writing and editing, summarising, translation, answering questions, writing and explaining code, brainstorming, and acting as the "brain" of AI agents.
Limitations:
- Hallucinations: they can state false information confidently, because they predict plausible text rather than checking facts.
- Knowledge cutoff: they only know what was in their training data, unless connected to search or documents with RAG.
- Context limits: they can only consider a limited number of tokens at once, although modern context windows are very large.
- Cost: large models need expensive hardware to train and run.
Frequently asked questions
Is ChatGPT an LLM?
ChatGPT is an application built on top of OpenAI's GPT large language models, with extra features such as memory, web search and tools.
What is the difference between an LLM and AI?
AI is the broad field of making machines do intelligent tasks. LLMs are one type of AI model, specialised in language. See AI vs machine learning vs deep learning.
How big is a large language model?
From about 1 billion parameters for small models to hundreds of billions or more for frontier models. Size is often written as "8B" or "70B", meaning 8 or 70 billion parameters.
Can I run an LLM on my own computer?
Yes. Smaller open-weight models run well on a gaming GPU or a Mac with tools such as Ollama or LM Studio.
Sources and further reading
- Vaswani et al., Attention Is All You Need (2017)
- Brown et al., Language Models are Few-Shot Learners (GPT-3) (2020)
- Ouyang et al., Training language models to follow instructions with human feedback (2022)




Leave a Reply