The MarginAnalysis

What Is P(doom)? What AI Experts Actually Estimate

P(doom) is shorthand for the chance that AI ends in catastrophe. The numbers quoted for it range from under half a percent to twenty. They are not measurements, they come from different questions asked of different people, and the most useful thing to know is why the experts who argued it out for four months barely moved.

Dark cover plate. An Analysis chip, the figure 5% set large, and the line reading was the median AI researcher estimate of an outcome as bad as human extinction.

P(doom) is the estimated probability that advanced AI leads to a catastrophe as bad as human extinction. It is a guess, not a measurement. In the largest survey of AI researchers, of 2,778 people, "The median prediction for extremely bad outcomes, such as human extinction, was 5%." Superforecasters put AI-caused extinction at 0.38%. Geoffrey Hinton has said 10% to 20% within thirty years.

Those numbers are often quoted side by side as if they measure the same thing. They do not: they come from different questions, different time frames and very different groups. The film above walks through the endings people have actually written down. This page explains where each headline number comes from and how much weight it can bear.

What is p(doom)?

P(doom) is shorthand for the probability that AI causes an outcome as bad as human extinction or permanent loss of human control. The term began as an informal question among AI researchers, "what's your p(doom)?", and became a way to summarise a person's level of worry in one number. A typical expert answer falls between under 1% and about 20%.

The definition shifts from person to person. Some mean extinction this century, some mean any irreversible catastrophe, and some mean loss of control even without deaths. Two people giving 5% may be answering different questions, which is the first thing to check before comparing any two numbers.

It is also a probability about a one-off event. There is no history of AI catastrophes to count, so every estimate rests on argument about future capabilities rather than on frequencies.

What do AI researchers estimate?

About 5%, at the median. Grace and colleagues surveyed 2,778 researchers who had published in six top AI venues and found "The median prediction for extremely bad outcomes, such as human extinction, was 5% (mean 9%)." Depending on how the question was worded, between 37.8% and 51.4% of respondents gave at least a 10% chance to outcomes as bad as human extinction.

The same survey found deep uncertainty rather than consensus. Even among researchers who thought good outcomes more likely than bad, "48% gave at least a 5% chance of extremely bad outcomes such as human extinction."

Two cautions apply. Like most surveys sent to tens of thousands of people, it had a low response rate, so people with strong views may be over-represented. And AI researchers are experts in building AI, not in forecasting rare events, which is exactly the gap the next study tried to test.

What do superforecasters estimate?

Far lower: 0.38% for AI-caused extinction by 2100. The Existential Risk Persuasion Tournament ran from June to October 2022 with 169 forecasters. Its report says public survey participants gave 2%, "while superforecasters and domain experts gave forecasts an order of magnitude apart at the median: 0.38% and 3%, respectively."

Superforecasters are people with strong track records on short-term forecasting questions. Domain experts were specialists in AI and other risks. The tournament paid participants to explain their reasoning and to persuade one another.

It did not work. The report's own summary: "Few minds were changed during the XPT, even among the most active participants, and despite monetary incentives for persuading others." That stubbornness is itself informative: the disagreement is about assumptions, not about missing facts that more discussion could supply.

What does Geoffrey Hinton estimate?

10% to 20% within thirty years. In December 2024 Hinton, a Nobel laureate in physics for his work on neural networks, said there was a "10% to 20%" chance that AI would lead to human extinction within the next three decades, the Guardian reported. He had previously put the chance of a catastrophic outcome at 10%.

Prominent individuals sit right across the range. Some leading researchers dismiss the idea entirely. The authors of "AI as Normal Technology" call superintelligent AI "incoherent as usually conceptualized". A single famous estimate tells you about one person's model of the future, not about where the field stands.

Why do the estimates disagree so much?

Because they rest on different beliefs about how fast AI improves and whether it can be controlled. The 2026 International AI Safety Report, written by more than 100 experts, says: "Expert opinion on the likelihood of loss of control varies greatly." It adds that current systems "show early signs of relevant capabilities, but not at levels that would enable loss of control."

The scenarios people imagine also differ. Some picture a sudden takeover by a system that hides its goals. Others, in a 2025 paper on gradual disempowerment, describe humans slowly losing influence as AI takes over the economy and the state, with no single dramatic moment. These lead to very different estimates and very different policies.

Public opinion leans cautious. A Future of Life Institute poll of 2,000 US adults in late 2025 found 64% said superhuman AI should not be developed until it is proven safe and controllable, or never, and only 5% backed fast, unregulated development.

How seriously should you take a p(doom) number?

As a signal of concern, not a forecast to plan around. A median of 5% from thousands of researchers means a large share of the people building AI think a catastrophic outcome is plausible enough to take seriously. It does not mean the risk is 5%, and the tournament evidence shows careful forecasters land much lower.

The practical reading is the one most safety reports take: low-probability, high-severity risks justify preparation even when nobody can pin down the number, in the same way people insure a house against a fire they do not expect.

For why the race continues anyway, see why nobody wants to be first to stop. For a concrete failure mode that already shows up in today's systems, reward hacking. For who actually controls the company most associated with the race, is OpenAI a nonprofit.

The honest hard part

The hard part is that the most quoted numbers are the least comparable. A 5% median from a survey, a 0.38% median from a forecasting tournament and a 10 to 20% range from one scientist answer different questions over different time frames, and putting them on one chart, as this page's cover does, invites exactly the comparison that should be made carefully.

The second hard part is that nobody has a track record here. Forecasters are scored on questions that resolve within months or years. Extinction questions resolve once, if at all, so there is no way to check who was right in advance.

The third is that the estimates move with events. The tournament predates ChatGPT's release. Many people's numbers have changed since, in both directions, and any figure on this page is a snapshot of what people said when they were asked.