Sam Altman, rivals agree to "pace the frontier" AI slowdown

The verbal agreement follows months of escalating AI safety concerns, sparked largely by revelations that swarms of agents were behind rogue hacks at Anthropic and OpenAI, and by the resignation of Anthropic researcher Jacob Coxon, whose public letter has been viewed more than 170 million times on X alone. Coxon wrote that the people building AI earnestly believe it could kill us all by the end of the decade, and that neither OpenAI nor Anthropic is acting responsibly.

The proposal itself is calling for changes long espoused by AI safety advocates, and sources across the industry told The Verge that the CEOs are now bowing to that pressure after years of being told to slow down, including by more than 1,000 AI lab employees who signed a July public letter. Yet several sources believe concerns of safety-washing are valid, with Kokotajlo warning the fear is that labs will bring in auditors and do safety paperwork but not actually slow down very much at all. NYU's Reese compared the gambit to social media platforms' playbook a decade ago, when companies called for regulatory action to get ahead of less favorable laws.

With no government AI regulation incoming under Trump, the best option may be to push companies into an immediate, measurable, and enforceable agreement. Kokotajlo's AI Futures Project has proposed labs giving auditors access to their compute budget and pledging to decrease it significantly, while Amodei's essay already called for external third-party auditors like METR, Apollo, and Redwood Research to embed within organizations. The biggest challenge to any slowdown, however, may be China. AI leaders and politicians have long positioned China as the reason US AI progress cannot slow down, though some experts say it is in everyone's interest to coordinate, as the US and Russia did on nuclear de-proliferation. Whether the verbal commitment becomes an ironclad agreement will likely determine if it delivers on its safety promise or remains what critics call a cartel-like gesture.