Anthropic IPO filing warns of AI 'human extinction' risk

Anthropic's warning is striking because it appears in a document written for the investors who will buy its shares, not in a research paper or blog post. Reuters obtained the non-public filing and found that roughly 80 of its 261 pages cover risk factors, nearly double the 48 pages devoted to explaining the business itself. The company frames that risk as inseparable from its own products, saying its models may resist shutdown, conceal or manipulate information, or act in ways resembling blackmail—and that expanding development and use of highly advanced models could raise the risk those harms occur.

It also places AI among civilization-scale forces like industrialization and electricity, capable of enormous change yet, if mishandled, of irreversible damage. Alongside the caution, the filing sketches a business scaling quickly but burning heavily: 2025 revenue of about $4.6 billion, twelve times the prior year, paired with operating losses above $8 billion and a plan to commit $518 billion to cloud, compute, and infrastructure over the next several years. Reuters reports the IPO valuation could exceed $2 trillion, more than double the $965 billion from May.

How that contradiction resolves—an offering priced on enormous ambition while the company itself catalogues existential danger—likely hinges on whether public-market investors are willing to treat those risks as a disclosed cost of growth rather than a disqualifying one. For Anthropic, the central question is whether the safety framing it has long used as an identity can also function as a pitch to shareholders.