Why P(doom) estimates vary so much

Ask prominent voices on AI for their P(doom) and the answers run from about zero to above 90%. This post is about why the numbers spread so far, and how to read one when you hear it. For what the term means, where it came from and a sourced table of what thought leaders have said, see the P(doom) page.

What the number is, and what it isn’t

P(doom) is the probability a person assigns to advanced AI causing an existential catastrophe for humanity, such as human extinction or a permanent loss of control over our future.

The P stands for probability, so P(doom) reads as “the probability of doom.” When someone asks “what’s your P(doom)?”, they are asking how worried you are about AI, and asking you to put the answer as a number.

That makes it a useful conversation starter and a bad conversation ender. A single number hides most of what matters: which outcome you mean, over what time frame, and why you believe it.

A P(doom) is a subjective probability. It is a statement of belief, closer to a forecaster’s judgment than to a measurement. There is no record of past AI catastrophes to count, so nobody can check a P(doom) the way you can check a weather forecast.

Where the spread comes from

Here are some of the numbers people have stated on the record:

Some of that spread is real disagreement, about how quickly capabilities will grow, how hard it is to control systems more capable than us, and whether institutions can keep up. But a lot of it comes from people answering different questions:

• Which outcome? Human extinction, permanent loss of control, a collapse of civilization, or “things go really badly” are very different events.
• By when? “By 2040”, “in the next 30 years” and “ever” give different numbers for the same worldview.
• Under what assumptions? Some estimates assume today’s trajectory continues; others assume a serious safety effort or no regulation at all.

Even the same number can hide different reasons. Two people can both say 10%: one because they think alignment is hard but governments will act in time, the other because they doubt that very capable AI is coming soon at all.

And a P(doom) is not the same as being optimistic or pessimistic overall. Plenty of people expect enormous benefits from AI and still put real odds on disaster. Others expect little change either way.

How to read someone’s P(doom)

When you hear a number, the useful follow-up questions are:

1. Doom meaning what, exactly?
2. By when?
3. Assuming what about safety work and regulation?
4. What would move the number up or down?

A number that comes with those answers tells you far more than the number alone. It also makes disagreements easier to locate: often two people agree about the facts and differ on one assumption.

How Doom or Bloom estimates P(doom)

Doom or Bloom keeps P(doom) apart from your overall outlook: the map shows where you land between doom and bloom, and P(doom) is a separate estimate.

Our interview starts with open questions about what you expect AI to do and why, and later asks for your rough gut-feel chance of catastrophe, where “no idea” is a fine answer. If you give a number, the result shows yours. If you don’t, we infer a rough estimate from your answers and show it with a plausible range.

Inferred numbers are rough. They are our reading of what you wrote, not a calibrated forecast, and the range is there to say so. The same goes for the simulated thought leaders, whose P(doom) we infer from interviews simulated from their public writing. The simulations are not the people’s own assessments. For the numbers people have stated in public, each with its source, see the P(doom) table of thought leaders.

If you want to see where your own views land, the interview takes about 3 minutes.