Skip to content

What Is Reinforcement Learning? How AI Learns by Trial and Error (With Examples)

Reinforcement learning (RL) teaches an AI agent to make decisions by rewarding good actions and penalising bad ones. How RL works, key terms like policy and reward, famous examples from AlphaGo to ChatGPT and reasoning models, and its limits.

5 min read

What Is Reinforcement Learning? — cover image from TensorCore.si

Short answer

Reinforcement learning (RL) is a type of machine learning in which an agent learns by trial and error: it takes actions in an environment, receives rewards or penalties, and gradually learns a strategy (a policy) that earns the most reward over time. RL powered DeepMind's AlphaGo, helps robots learn to walk, and is a key ingredient in modern AI assistants: RLHF made ChatGPT helpful, and large-scale RL is what turns language models into reasoning models.

RL in plain English

Think of training a dog. You do not explain the rules; you reward the behaviour you want. Over many attempts, the dog learns which actions lead to treats. Reinforcement learning works the same way, except the "dog" is a computer program and the "treat" is a number.

How reinforcement learning compares with other machine learning

Supervised learningUnsupervised learningReinforcement learning
Learns fromLabelled examples (input → correct answer)Unlabelled dataRewards from its own actions
GoalPredict the right labelFind structure or patternsMaximise total reward
FeedbackImmediate and exactNoneDelayed and noisy
ExampleSpam filterCustomer clusteringGame-playing AI, robot control

For how these fit into AI as a whole, see AI vs machine learning vs deep learning.

Key terms

  • Agent: the learner and decision maker.
  • Environment: the world the agent acts in, such as a game, a simulator or a conversation.
  • State: what the agent observes at a given moment.
  • Action: a choice the agent makes.
  • Reward: a number that says how good the outcome was.
  • Policy: the agent's strategy, mapping states to actions.
  • Value: the expected future reward from a state or action.
  • Episode: one complete run, such as one game.

How the learning loop works

1. The agent observes the current state. 2. It chooses an action using its policy. 3. The environment changes and returns a reward. 4. The agent updates its policy to make rewarding actions more likely. 5. Repeat millions of times.

A central challenge is exploration vs exploitation: should the agent try something new that might be better, or stick with what already works? Another is credit assignment: when a game is won after 200 moves, which moves deserve the credit?

Main approaches

  • Value-based methods such as Q-learning learn how good each action is in each state, then pick the best. DeepMind's DQN used a neural network to play dozens of Atari games from raw pixels in 2013 to 2015.
  • Policy-based methods directly adjust the policy to increase reward. PPO (proximal policy optimization) is one of the most widely used.
  • Actor-critic methods combine both: an "actor" chooses actions and a "critic" estimates their value.
  • Model-based methods learn a model of the environment and plan ahead, as in AlphaZero and MuZero.

Combining RL with deep neural networks is called deep reinforcement learning.

Famous examples

  • AlphaGo (2016): beat world champion Lee Sedol 4–1 at Go, a game long thought too complex for computers. Its successor AlphaZero learned chess, shogi and Go from self-play alone.
  • Games: OpenAI Five (Dota 2) and DeepMind's AlphaStar (StarCraft II) reached top human levels.
  • Robotics: robots learn to walk, grasp and manipulate objects, often in simulation first.
  • Chip design and data centers: RL has been used to optimise chip layouts and cooling systems.
  • Recommendation and ads: systems that optimise long-term engagement.

RL in ChatGPT and modern LLMs

RLHF: making models helpful

After pre-training, a large language model is good at predicting text but not at being a helpful assistant. Reinforcement learning from human feedback (RLHF) fixes this: people rank model answers, a reward model learns those preferences, and the LLM is trained with RL to produce answers the reward model scores highly. This was central to ChatGPT; see How does ChatGPT work?.

RL for reasoning

Since 2024, labs have applied RL on problems with checkable answers, such as maths and code with tests. Models are rewarded for correct final answers and learn to think step by step, check their work and backtrack. OpenAI's o1 and DeepSeek-R1 (which used an efficient method called GRPO) showed how powerful this is.

Agents

RL is increasingly used to train AI agents that use tools, browse and write code over many steps, rewarding them for completing real tasks.

Limitations

  • Reward hacking: agents find loopholes that earn reward without doing what designers wanted, a key concern in AI alignment.
  • Sample inefficiency: RL often needs millions of attempts, which is why simulators matter.
  • Instability: training can be sensitive to settings and hard to reproduce.
  • Sim-to-real gap: skills learned in simulation may not transfer perfectly to the physical world.
  • Reward design: defining the right reward for real-world goals is hard.

What hardware RL needs

RL training alternates between generating experience (running the policy many times) and updating the model. For LLMs, the generation phase is essentially large-scale inference, so RL for reasoning models uses huge GPU clusters for both sampling and training. Small RL projects (games, simple robots) run fine on a single consumer GPU.

Frequently asked questions

Is reinforcement learning the same as deep learning?

No. Deep learning is about using deep neural networks; RL is about learning from rewards. Deep reinforcement learning combines the two.

Who invented reinforcement learning?

Its modern foundations were developed by Richard Sutton and Andrew Barto in the 1980s, building on earlier ideas from psychology and control theory. They received the 2024 ACM Turing Award for this work.

Is RLHF still used?

Yes, along with related methods such as DPO and RL from AI feedback. Most assistant models use some form of preference training after supervised fine-tuning.

Can I learn RL as a beginner?

Yes. Start with simple environments in the open-source Gymnasium library and classic algorithms like Q-learning, then move to deep RL libraries.

Sources and further reading

Think you understand it now? Prove it.

Solve real Tensor Core math in the TensorCore.si Challenge and earn a certificate with your name.

Take the challenge

Leave a Reply

Your email address will not be published. Required fields are marked *