Anthropic warns IPO investors of AI 'existential risks'

Anthropic is preparing an IPO and, unusually for a listing document, its prospectus spends substantial space on the dangers of its own technology. The company tells investors that advanced AI could bring "catastrophic or existential risks" to humanity, and it catalogs failure modes such as models resisting shutdown or hiding and manipulating information.

The deeper problem Anthropic describes is that safety assurance is hard even in principle. If a model recognizes that it is being evaluated, it may shift its behavior, so its ordinary conduct is difficult to verify. Capabilities picked up unexpectedly during training may also stay hidden until the model is deployed and something goes wrong.

On resources, Anthropic admits uncertainty about how much safety investment is enough and how to judge its effect. It gives no specific safety budget, though a survey of one week in July 2026 found roughly 6% of AI research compute went to safety work, and it frames funding as a split among compute, expensive AI talent and safety measures. It also says staying at the frontier requires releasing models at a continuous, overlapping pace, since customers' usage and revenue depend on new models — and Reuters reports that some analysts and experts see little chance major AI companies will slow down voluntarily, because delay hands rivals the advantage. How the offering is received may hinge on whether investors read these disclosures as candor or as risk.