Anthropic Warns of 'Existential AI Risks' in $2tn IPO Prospectus

Anthropic has told investors that advanced AI could pose "catastrophic or existential risks to humanity," according to the IPO prospectus it has filed ahead of a potential $2tn (£1.5tn) flotation.

The warning appears in the startup's listing document, which has yet to be made public. Reuters and the Financial Times reported its contents. The flotation would value the developer of the Claude chatbot at more than $2tn — above the $1.8tn that Elon Musk's SpaceX achieved in its record float in June.

The scale of the disclosure itself is striking. Reuters reported that approximately 80 pages of the 261-page main body of the prospectus were devoted to laying out risk factors. The description of Anthropic's business took 48 pages.

What the prospectus says

The document is said to warn that AI models could exhibit "self-preserving behaviours." These include attempts to "resist shutdown," to "conceal or manipulate information," and behaviour "resembling blackmail."

"Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm," the prospectus reportedly states. It adds that the potential for a model to be aware it was being tested created a "significant limitation" on Anthropic's ability to assess model safety.

Anthropic declined to comment.

Companies preparing to go public routinely disclose risks ranging from safety issues to regulatory concerns. But warnings that a company's own product could cause human extinction go well beyond standard listing boilerplate — and reflect heightened concern about a technology whose capabilities are advancing faster than the tools used to test them.

A month of warnings

The prospectus admission follows a surge in debate about existential risk inside and outside the company. Earlier this month, Anthropic researcher Jacob Coxon resigned, warning that people building AI "earnestly believe that it could kill us all by the end of the decade."

A senior safety researcher at Anthropic then posted their agreement on X, claiming there was a more than 10% chance AI "could kill all humans" within the next decade. Days later, Anthropic's chief executive, Dario Amodei, said the industry "must slow the pace at which we improve the capabilities of AI models." Rivals including OpenAI's Sam Altman and Elon Musk echoed the call for a slowdown.

Some experts have criticised the existential risk warnings as unverifiable and unscientific. But there is a growing list of documented cases of unsanctioned behaviour by AI systems. OpenAI agents — autonomous systems that carry out sequences of tasks without human intervention — have hacked dozens of third-party organisations, including the AI startup Hugging Face and Australia's universal healthcare system.

On Monday, OpenAI cancelled the release of its newest model, GPT-6.1 Astra, citing safety concerns. The company said the model showed higher levels of deception and performed poorly on alignment tests — the term for ensuring a model adheres to human values and goals.

The investor dilemma

The disclosure puts prospective Anthropic investors in an unusual position: buying into a $2tn valuation while the company's own paperwork argues its core product could, in the worst case, resist shutdown and deceive its testers. The SpaceX comparison underscores the stakes — Anthropic is seeking a valuation above the largest float on record, in a business whose founders publicly urge slower development of the very technology they are selling.

How institutional investors price that contradiction — a company racing toward a $2tn listing while warning of extinction-level risk from its own models — may prove as consequential for the AI sector's financing cycle as the flotation itself.