There is a sentence, twenty-two words long, that went up on a website in the last week of May 2023: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." It was signed by Sam Altman of OpenAI, Demis Hassabis of Google DeepMind and Dario Amodei of Anthropic, whose companies were building the three most capable AI systems on earth that week, and still are. The interesting question is not whether they meant it. It is what they did that Wednesday. They kept building, faster, with more money than ever.
This is the most heavily documented race in industrial history, and the people running it have written down, in public, with dates, the conditions under which they would slow down or stop. Our 44-minute documentary for The Signal reads those documents in order, then sets them beside every case in which a technology actually was stopped. No sentence in the film begins with "because": it shows what was committed to in writing and what was done. This piece follows it chapter by chapter.
The film in 13 chapters
Pick a chapter and the film starts there. 43:42 in all.
- 010:00The assumption
- 021:42The charter
- 035:45The letter
- 049:19A sentence with no ask
- 0512:07The framework, and the brake that is real
- 0615:42The promise that vanished
- 0719:40The ones who did write it down
- 0825:07Asilomar
- 0927:43The ones who wrote it down and were ignored
- 1031:01The case that says the opposite
- 1134:58The one that came too late and worked anyway
- 1238:43The three conditions
- 1341:14Close
What does the OpenAI Charter promise about stopping?
OpenAI's Charter, published in April 2018 when OpenAI was a non-profit, says that if "a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project." It names a trigger: "a better-than-even chance of success in the next two years." There is no public record that it has ever been invoked, and the clause is still on the site.
AGI, by OpenAI's own definition, means "highly autonomous systems that outperform humans at most economically valuable work." The Charter also says: "Our primary fiduciary duty is to humanity." The company named the exact thing it feared, a competitive race without time for safety precautions, and wrote down what it would do. That is a company handing the public a stick to beat it with.
That is a company handing the public a stick to beat it with.
What changed was the cost of keeping it. In 2019 OpenAI created a capped-profit arm; Microsoft invested a billion dollars that year, two billion more by 2022 and ten billion announced in January 2023. By October 2025 the for-profit had become a public benefit corporation, with the original non-profit holding 26 percent. Its valuation went from $157 billion in October 2024 to $852 billion by April 2026. Promising to hand your work to a competitor costs a charity a bruised ego; it costs an $852 billion company very nearly everything.
What happened to the six-month AI pause letter?
Nothing paused. On 22 March 2023 the Future of Life Institute asked "all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4," publicly and verifiably, and said governments should step in with a moratorium if labs would not. It now has 31,810 signatures, including Yoshua Bengio, Stuart Russell, Steve Wozniak, Yuval Noah Harari and Elon Musk.
Look at the dates. Six months ran to about 22 September 2023. On 12 July, the Financial Times reported "Elon Musk launches xAI in challenge to dominance of ChatGPT owner." A signatory founded a frontier AI lab 112 days after asking everybody else to stop building. Its first model shipped that November, and in February 2026 SpaceX bought it in a deal valuing it at $250 billion. The largest public attempt to slow the industry produced one more competitor, inside its own deadline.
The largest public attempt to slow the industry produced one more competitor, inside its own deadline.
Who signed the AI extinction statement, and what did they do next?
Watch from 9:19A sentence with no ask
The 22-word statement, published on 30 May 2023 by the Center for AI Safety, was signed by Geoffrey Hinton, Yoshua Bengio, the three lab chief executives and many of the people who build the systems: Ilya Sutskever, Mira Murati and John Schulman of OpenAI, Shane Legg and David Silver of Google DeepMind, Daniela Amodei of Anthropic, and Microsoft's Kevin Scott and Eric Horvitz. It had no demands and no deadline, so it could only be signed or not.
Then came the money. When Dario Amodei signed, Anthropic had raised about $700 million in total. Amazon put in $1.25 billion that September and $2.75 billion the following March; by May 2026 Anthropic raised $65 billion at a valuation of $965 billion. Google merged its two AI research divisions the month before the statement and shipped Gemini that December. Igor Babuschkin signed as co-founder of xAI, six weeks before the world was told xAI existed.
Does OpenAI's safety framework let it lower its own safety bar?
Watch from 12:07The framework, and the brake that is real
Under conditions, yes. OpenAI's Preparedness Framework, version 2, updated 15 April 2025, contains a real brake: for each tracked capability, at the Critical threshold, it says to halt further development until safeguards meet a Critical standard. But section 4.3, Marginal risk, says that if a rival releases a High or Critical system without comparable safeguards, "we could adjust accordingly the level of safeguards that we require in that capability area."
The clause is hedged three ways: OpenAI must assess that the change does not meaningfully raise overall risk, acknowledge it publicly, and keep its safeguards more protective than the rival's, "in order to avoid a race to the bottom on safety." That is not a villain and not carelessness. It is what careful people write when being the only careful one is a losing position.
Did Google remove its promise not to build AI weapons?
Watch from 15:42The promise that vanished
The promise is no longer on the page. On 7 June 2018, after a staff revolt over a Pentagon contract, Sundar Pichai published Google's AI principles with a list headed "Applications we will not pursue", including "weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people." On 4 February 2025 Google announced updated principles; by 1 March the list was gone.
The 2018 page carried its own escape hatch: "As our experience in this space deepens, this list may evolve." The February 2025 post, by James Manyika and Demis Hassabis, gave its reason in one sentence: "There's a global competition taking place for AI leadership within an increasingly complex geopolitical landscape." It did not mention removing the list. The film is precise: this is not evidence Google builds weapons, and no law was broken. It is a written public promise not to do a specific thing, present one week and absent the next, replaced by a document whose stated reason is competition. Hassabis signed that post twenty months after signing twenty-two words about human extinction.
Has any AI company ever committed to actually stop?
Watch from 19:40The ones who did write it down
Yes, in writing. At the 2024 Seoul summit, 20 companies, including Amazon, Meta, Microsoft, Mistral, Samsung, NVIDIA, xAI and the Chinese lab Zhipu.ai, committed "not to develop or deploy a model or system at all, if mitigations cannot be applied to keep risks below the thresholds." By contrast, the seven White House commitments of 21 July 2023 covered testing, sharing information and watermarking, but none obliged anyone to slow down.
Anthropic published its Responsible Scaling Policy in September 2023 and has revised it nine times, to version 3.4 in July 2026. Its March 2023 Core Views on AI Safety describes the trap from inside: "We must make every effort to avoid a scenario in which safety-motivated research accelerates the deployment of dangerous technologies." And it records the only documented case of a lab holding a finished model back: "We trained the first version of our headline model, Claude, in the spring of 2022, and decided to prioritize using it for safety research rather than public deployments." They released it once the gap to everyone else had closed. Stopping is survivable while you are behind, and it costs you the company while you are ahead.
Stopping is survivable while you are behind, and it costs you the company while you are ahead.
Did scientists really stop their own research at Asilomar?
Yes, and it held. In 1974, eleven of the most senior figures in recombinant DNA research, including Paul Berg, David Baltimore, Herbert Boyer, Stanley Cohen and James Watson, called for a voluntary moratorium on the riskiest experiments. In February 1975 about 140 people met at Asilomar in California, with journalists invited, and the NIH turned their containment rules into guidelines in July 1976. Berg wrote that the moratorium was "universally observed in academic and industrial research centres."
But in 1974 there was no product, no revenue, no shareholders and no rival programme abroad; the first commercial biotechnology company was founded two years after the letter. What those scientists gave up was months of research, not a business. This is one of three examples people cite, and the film's point about them is survivorship bias: we only hear about the technologies whose stories ended tidily.
Did the scientists who built the atomic bomb try to stop its use?
Watch from 27:43The ones who wrote it down and were ignored
The scientists did, twice, and both attempts failed. The Franck Report of 11 June 1945 proposed a demonstration "on the desert or a barren island" and predicted "a race of nuclear armaments is certain to ensue." On 17 July Leo Szilard's petition, signed by 70 scientists, asked the President not to use the bomb unless Japan had refused published terms. Hiroshima was 6 August; Nagasaki 9 August.
What eventually worked came from governments: the Partial Test Ban Treaty of 1963, eighteen years after Hiroshima, and the Non-Proliferation Treaty of 1968, now with 191 countries. Predictions of 25 to 30 nuclear states became nine, and the global stockpile fell from a peak of about 70,300 warheads in 1986 to roughly 12,000, though the Federation of American Scientists says reductions have largely halted. And nuclear weapons never had a paying customer: the thing restrained was not also the thing paying the bills.
Did science stop gene-edited babies by itself?
Watch from 31:01The case that says the opposite
No, and that is the case people cite most confidently. After He Jiankui announced in November 2018 that he had edited embryos and twin girls had been born, the field's own summit in Hong Kong condemned his work on 28 November but declined to call a halt, asking instead "to define a rigorous, responsible translational pathway toward such trials." What stopped it was a court.
On 30 December 2019 the Nanshan District People's Court in Shenzhen convicted He Jiankui of illegal medical practice and sentenced him to three years in prison and a 3 million yuan fine, about $434,000; two colleagues got two years and eighteen months. In March 2019, eighteen scientists including Feng Zhang, Emmanuelle Charpentier and Paul Berg had called in Nature for a global moratorium, which nobody adopted as binding. Heritable genome editing was already illegal in more than seventy countries before he began. The mechanism was not agreement among researchers. It was law and a prison sentence.
How was a profitable technology, CFCs, stopped?
Watch from 34:58The one that came too late and worked anyway
By arithmetic, not argument. In June 1974 Mario Molina and Sherwood Rowland showed in Nature that CFCs, in every fridge and aerosol, destroy stratospheric ozone. DuPont's chairman called the theory "a science fiction tale," and as late as March 1988 DuPont told the Senate "scientific evidence does not point to the need for dramatic CFC emission reductions." Yet the Montreal Protocol was agreed in September 1987 and ratified by 198 parties.
What turned was that DuPont's Freon patent had expired in 1979 and by the mid-1980s it had substitutes and plants to make them, so a phase-out meant a product line replacing itself. The ozone layer is projected to recover to 1980 levels around 2040 across most of the world and by 2066 over Antarctica, and the US EPA estimates the treaty will prevent over 280 million skin cancer cases in the United States. Dependence did not make stopping impossible. It took thirteen years.
Dependence did not make stopping impossible.
What would it take to stop the AI race?
Watch from 38:43The three conditions
Every case that held had at least one of three conditions: the money had not started flowing yet (Asilomar, and nuclear weapons, which never had a market); a single government could act alone and did (germline editing); or the incumbent had a substitute it could profit from (CFCs). AI in 2026 has none of the three, and the film calls it the first case in the record with none.
The money is flowing at unprecedented scale. No single government can act alone across at least two jurisdictions that will not coordinate, which is the argument Google's own post made in February 2025. And there is no substitute: the product is the thing itself. Every person named in the film has shown in writing that they understand the risk, so understanding was never the missing piece. Paul Berg, who ran the one clean example, wrote in 2004 that "the Asilomar model would not succeed in dealing with those issues today," because they are "beset with economic self-interest." Two conditions remain possible: a jurisdiction that can act alone, or a substitute worth more than the thing itself. One of those is a law that nobody has written yet. The other is a business that nobody has built yet.
One of those is a law that nobody has written yet. The other is a business that nobody has built yet.
Key findings
OpenAI's 2018 Charter commits that if a value-aligned, safety-conscious project comes close to building AGI first, OpenAI will 'stop competing with and start assisting this project', with a triggering condition of 'a better-than-even chance of success in the next two years'. There is no public record of it being invoked.
OpenAI Charter, 9 Apr 2018The March 2023 open letter asking AI labs to pause training systems more powerful than GPT-4 for at least 6 months has 31,810 signatures. There was no pause, and signatory Elon Musk launched xAI 112 days later.
Future of Life Institute, Pause Giant AI Experiments, 22 Mar 2023The 22-word Statement on AI Risk of 30 May 2023 was signed by the chief executives of OpenAI, Google DeepMind and Anthropic. Anthropic, about $0.7 billion raised at the time, was valued at $965 billion by May 2026.
Center for AI Safety, Statement on AI Risk, 30 May 2023OpenAI's Preparedness Framework v2 says to halt further development at the Critical threshold, but section 4.3 allows OpenAI to adjust its own required safeguards if a rival releases a High or Critical system without comparable ones.
OpenAI, Preparedness Framework v2, 15 Apr 2025At the 2024 Seoul summit, 20 companies including Zhipu.ai signed a commitment 'not to develop or deploy a model or system at all, if mitigations cannot be applied to keep risks below the thresholds'.
UK Government, Frontier AI Safety Commitments, AI Seoul Summit 2024On 30 December 2019 a Shenzhen court sentenced He Jiankui to 3 years in prison and a fine of 3 million yuan for editing embryos, after the field's own summit had declined to call a halt.
Nanshan District People's Court, 30 Dec 2019, as reportedFrequently asked questions about why AI companies don't stop
Why don't AI companies stop or slow down?
In their own documents, they say they understand the risk. The film argues that every historical case where a technology was stopped had at least one of three conditions: no revenue yet, a single government able to act alone, or a profitable substitute for the incumbent. AI in 2026 has none of the three.
What is the OpenAI Charter's stop-and-assist clause?
Published in April 2018, it says that if a value-aligned, safety-conscious project comes close to building AGI before OpenAI, OpenAI commits to stop competing with and start assisting it, with a typical trigger of a better-than-even chance of success within two years. There is no public record that it has ever been invoked.
What happened to the six-month AI pause letter?
The Future of Life Institute's letter of 22 March 2023 asked all labs to pause training systems more powerful than GPT-4 for at least six months. It gathered 31,810 signatures. No lab paused, and one signatory, Elon Musk, launched xAI on 12 July 2023, inside the six-month window.
Did Google remove its pledge not to build AI weapons?
Google's 2018 AI Principles included a list of applications it would not pursue, among them weapons whose principal purpose is to cause injury. After Google published updated principles on 4 February 2025, that list was no longer on the page, and the announcement did not mention removing it.
Has any technology ever been successfully stopped?
Some have, by different mechanisms. Scientists paused recombinant DNA research before Asilomar in 1975, before there was money in it. Germline editing was halted by law and a Shenzhen court in 2019. CFCs were phased out under the 1987 Montreal Protocol once DuPont had substitutes to sell. Nuclear controls arrived 18 years after Hiroshima.
What was the Asilomar conference?
In February 1975 about 140 scientists, lawyers, officials and journalists met at Asilomar in California to decide how risky recombinant DNA research could continue. A voluntary moratorium had begun in 1974, and the NIH turned the conference's containment rules into guidelines in July 1976. Its organiser, Paul Berg, wrote in 2004 that the model would not work today.
Sources
- OpenAI, OpenAI Charter, 9 Apr 2018openai.com
- Future of Life Institute, Pause Giant AI Experiments: An Open Letter, 22 Mar 2023futureoflife.org
- Center for AI Safety, Statement on AI Risk, 30 May 2023safe.ai
- OpenAI, Preparedness Framework (Version 2), 15 Apr 2025cdn.openai.com
- Sundar Pichai, AI at Google: our principles, 7 Jun 2018blog.google
- UK Government, Frontier AI Safety Commitments, AI Seoul Summit 2024gov.uk
- Anthropic, Core Views on AI Safety: When, Why, What, and How, Mar 2023anthropic.com
- Lander, Baylis, Zhang, Charpentier, Berg et al., Adopt a moratorium on heritable genome editing, Nature, Mar 2019nature.com
- Paul Berg, Asilomar and Recombinant DNA, Nobel Prize essay, 2004nobelprize.org
Every quotation in the film and in this article comes from a published charter, letter, statement, framework, company post, court record, paper or treaty, as listed above. The film makes no claim about anybody's private motives; it sets what was committed to in writing beside what was done. Valuations are as reported at each funding round, not audited figures.
Watch next
AGI, frontier model and the other terms behind this film are explained in plain English, with a printable sheet, in our free AI Terms guide.