Edited by 
What you will learn
- How a language model chooses each word from a probability list.
- What temperature, top-k, top-p and min-p actually do.
- Why the same question can get different answers.
- Why sampling can turn a model's uncertainty into a confident hallucination.
At every step, a language model outputs a score (logit) for every possible next token. Sampling settings decide how those scores become the one token it writes. The probabilities here are illustrative, chosen to behave like a real model, so you can see each setting's effect clearly.
The model's next-token probabilities
Draw a token
Results
How to use it
- Mouse: choose a prompt, move the sliders, then press Sample 1 token to spin the wheel or Sample 1,000× to see the statistics. Click a bar for its maths.
- Keyboard: click the simulator, then use S or Enter to sample, A for 1,000 samples, ← and → for temperature, ↑ and ↓ for top-p, [ and ] for top-k, 0 for greedy and 1 to 4 for the prompt.
What each setting does
| Setting | What it does | Typical use |
|---|---|---|
| Temperature | Divides the logits before softmax. Below 1 sharpens the distribution toward the top token; above 1 flattens it. 0 always picks the top token (greedy). | 0–0.3 for facts and code; 0.7–1 for chat; above 1 for brainstorming |
| Top-k | Keeps only the k most likely tokens and renormalises. | 20–50 in many chat setups |
| Top-p (nucleus) | Keeps the smallest set of top tokens whose probabilities add up to p. | 0.9–0.95 |
| Min-p | Drops tokens less likely than min-p × the top token's probability, so the cut-off adapts to how confident the model is. | 0.05–0.1 |
Watching a hallucination happen
Choose prompt 4, "The first person to walk on Mars was". Nobody has, but the model has no database of facts: it only knows which words tend to follow. Names like Neil score highly because they follow "first person to walk on" so often. Sample 1,000 times at temperature 1 and roughly half the answers start with a false name. Now try greedy (press 0): the model confidently picks the top token every time, and here the top token is wrong. Lower temperature makes answers consistent, not true. Read why AI makes mistakes for the full story.
Experiments to try
- Prompt 1 at temperature 1.0: how often does the model say Sydney? Now set top-p to 0.5.
- Prompt 2 at temperature 2.0: even "2 + 2" starts producing wrong digits.
- Prompt 3: compare temperature 0.3 and 1.5 and count the different words you get. This is the creativity dial.
- Set temperature to 2 and min-p to 0.1: min-p cleans up the long tail while keeping variety.
Where this runs
Sampling itself is cheap: the expensive part is producing the probabilities, one full pass through the model per token. See that cost in the LLM inference simulator and the mechanics in the attention simulator.
Leave a Reply