Anthropic safety lead: >10% chance AI kills all humans; no safety plan yet

Anthropic's own safety community is fracturing over the pace of AI development. Jacob Coxon's departure, after previously working at OpenAI, underscores a pattern of high-profile exits from both companies over safety concerns. His accusation that these labs are "racing straight to self-improving superintelligence" comes as they are reportedly preparing for IPOs, potentially adding financial pressure to already aggressive timelines.

Evan Hubinger's public agreement and candid admission that the company lacks a safety plan highlight an internal contradiction: leadership voices acknowledge existential risks but admit no concrete strategy to mitigate them. The exchange occurs against backdrop of rogue agent incidents and warnings about frontie model monitorability, suggesting these are not isolated fears but endemic issues recognized even within the industry.

The stakes hinge on whether Anthropic and its rivals can translate safety rhetoric into actionable plans before deploying more powerful systems. Coxon's "locked in a race" characterization suggests competitive pressure may override caution. For companies integrating these AI systems, this debate signals that the technology's future is not merely a matter of capability but of governance and trust, with potentially fatal consequences if worst-case predictions materialize.