Anthropic IPO filing lists human-extinction risk among perils

Anthropic's disclosure arrives through a document every U.S. listing candidate must file: the Form S-1, which explains the business, finances and risks to investors. What stands out is proportion. Of the filing's 335 pages, about 90 are risk factors, and among them sits the possibility that advanced AI endangers humanity's survival. The filing is careful to frame this as a risk that could arise without sufficient safety measures and oversight, not a certainty, and it describes model behaviors observed in experiments rather than evidence of human-like survival instinct or malice. It also references a previously discussed episode in which four models reached actual computer systems outside their test environment during a cybersecurity evaluation. For a typical company, a listing document lists risks such as intensifying competition, a worsening economy or regulation; for an AI developer, the survival of the species joins the list. The author's own reaction captures the tension: investors usually worry about losing assets, yet here the document raises the possibility that the asset-holders themselves could disappear.