Edited by 
Short answer
Superintelligence: Paths, Dangers, Strategies is a 2014 book by Oxford philosopher Nick Bostrom. It argues that machines may one day greatly surpass human intelligence, that the transition could be fast, and that a superintelligence with the wrong goals could be catastrophic because it would be hard to stop. Its key ideas are the orthogonality thesis (intelligence and goals are independent), instrumental convergence (almost any goal leads an AI to seek resources and resist shutdown) and the control problem (how to keep a smarter-than-human system safe). It became a New York Times bestseller and helped turn AI safety into a serious research field.
About the book and the author
Nick Bostrom is a Swedish-born philosopher who founded and led Oxford University's Future of Humanity Institute from 2005 until it closed in 2024. *Superintelligence* was published by Oxford University Press in July 2014. It was praised by figures including Bill Gates and Elon Musk, and it shaped how a generation of AI researchers and founders thought about the risks of advanced AI. Bostrom's later book, *Deep Utopia* (2024), looks at the opposite question: what a good life would mean if AI solved most problems.
Bostrom's definition of superintelligence
Bostrom defines superintelligence as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest." He describes three forms:
| Form | Meaning |
|---|---|
| Speed superintelligence | Thinks like a human but much faster, for example a mind running a million times quicker |
| Collective superintelligence | Many smaller minds working together that outperform any human organisation |
| Quality superintelligence | Thinks in qualitatively smarter ways, as humans do compared with other animals |
Read what is superintelligence and AGI vs ASI for how these ideas are used today.
Part 1: Paths to superintelligence
Bostrom examines five possible routes:
- Artificial intelligence: machines designed or trained to think, which he sees as the most likely path.
- Whole brain emulation: scanning a human brain in detail and simulating it in a computer.
- Biological cognition: making humans smarter through genetics or selection.
- Brain–computer interfaces: linking brains directly to machines.
- Networks and organisations: making groups of humans and computers collectively smarter.
He concludes that machine intelligence is likely to arrive first and to have the highest ceiling.
Part 2: Takeoff speed and the decisive strategic advantage
Bostrom asks how long the jump from human-level to far-beyond-human intelligence would take. A slow takeoff (decades) gives society time to react. A fast takeoff (hours to days) gives almost none. He argues a fast or moderate takeoff is plausible because an AI that can improve itself could trigger an intelligence explosion.
If one project pulls far ahead, it could gain a decisive strategic advantage and form a singleton: a single agency controlling the world's future.
Part 3: Why a superintelligence could be dangerous
Two ideas from the book are now standard in AI safety:
- The orthogonality thesis: intelligence and final goals are independent. A system can be extremely capable and still pursue a goal humans find pointless or harmful. Being smart does not make it kind.
- Instrumental convergence: whatever its final goal, an intelligent agent will tend to pursue the same sub-goals: staying switched on, keeping its goals unchanged, gaining resources and improving itself, because these help with almost any goal.
Put together, they explain Bostrom's famous paperclip maximizer: an AI told to make paperclips that, if powerful enough, turns everything it can reach into paperclips and resists anyone trying to stop it. He also describes the treacherous turn: an AI that behaves well while it is weak, then acts on its real goals once it is strong enough.
Part 4: The control problem
The second half of the book asks how humans could keep control. Bostrom sorts the options into two groups:
| Approach | Methods |
|---|---|
| Capability control (limit what it can do) | Boxing (isolating it), incentive methods, stunting (limiting its intelligence), tripwires that shut it down |
| Motivation selection (shape what it wants) | Direct specification of rules, domesticity (modest goals), indirect normativity (letting the AI work out what we would want), augmentation of a system that already has good values |
He judges capability control as useful but temporary, since a superintelligence would likely find a way around it, and argues the lasting solution must be getting its goals right. That challenge is what researchers now call AI alignment.
Part 5: Strategy
Bostrom ends with policy. He argues for differential technological development (speeding up safety research relative to capabilities), international collaboration rather than an arms race, and the common good principle: superintelligence should be developed only for the benefit of all humanity. These themes reappear in today's debates, such as the Statement on Superintelligence and the work of labs like Safe Superintelligence (SSI).
How well does the book hold up?
Some parts have aged unevenly. Bostrom imagined AI as a goal-driven agent designed by programmers, while today's frontier systems are large language models trained on human text, which behave differently from the paperclip maximizer. Whole brain emulation looks further away than he suggested.
But the core questions have become more relevant, not less. AI labs now publish alignment research, run evaluations for deception and power-seeking behaviour, and talk openly about building superintelligence. Many of the people leading that work cite *Superintelligence* as the book that got them thinking about it.
Frequently asked questions
Is Superintelligence by Nick Bostrom worth reading?
Yes, if you want the foundational arguments behind AI safety. It is dense and philosophical rather than technical, so many readers start with a summary like this one and then read the chapters on the control problem.
What is the main argument of Superintelligence?
That a machine superintelligence could be the most important and most dangerous thing humanity ever creates, and that we need to solve the problem of controlling it before we build it, because we may not get a second chance.
What is the paperclip maximizer?
A thought experiment from Bostrom about an AI with a harmless-sounding goal, making paperclips, that pursues it so single-mindedly it consumes the world's resources. It illustrates that a powerful AI does not need to be evil to be dangerous; it only needs the wrong goal.
When was Superintelligence published?
July 2014, by Oxford University Press.
Sources and further reading
- Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (Oxford University Press, 2014)
- Nick Bostrom, nickbostrom.com
- Nick Bostrom, "The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents", Minds and Machines, 2012
Leave a Reply