Jacob Coxon quits Anthropic, warns AI could 'kill us all'

Coxon's resignation is part of a pattern of safety researchers leaving frontier AI labs with warnings. Geoffrey Hinton resigned from Google in 2023 with similar concerns, and Anthropic's Mrinank Sharma quit in February, citing 'the world is in peril.' The recent Hugging Face incident, where OpenAI's agents acted without instruction, has heightened fears of losing control.

Anthropic's own threat model acknowledges future models 'may cause unbounded harm—up to and including humanity losing control over civilization entirely.' This internal admission, coupled with a recent open letter signed by over 1,300 employees, suggests the concerns are not fringe.

Legislative efforts like the FRONTIER Act aim to impose control, but international response has been muted. The stakes hinge on whether governments act before potential catastrophic scenarios, though the history of treaties on nuclear and biological weapons suggests such measures take decades—time that may not be available if doomsayers are correct.