Amodei, Altman back slowing AI over bioweapon risk

The warnings did not appear in a vacuum. They came days after AI researcher Jacob Coxon announced he was leaving his Anthropic role and accused both Anthropic and OpenAI of failing to act responsibly. Another Anthropic employee, Evan Hubinger, publicly agreed on X, writing that he believes AI could kill all humans and putting the probability at over 10% within the next decade. Amodei's and Altman's statements read as lab leaders responding to criticism from inside their own ranks.

The specific fear running through the piece is biological weapons. In 2022, researchers at Collaborations Pharmaceuticals found that a molecule generator built to hunt for treatments for human disease generated 40,000 molecules usable as chemical warfare agents in under six hours, some designed to be more toxic than known nerve agents. David Magnus, a Stanford professor of medicine and bioethics, called that result terrifying. As Dounja Sabra of the University of Hamburg notes, anyone can now draw on large language models trained on essentially every scientist who has lived, and those models can even offer video training on running experiments. Combined with easier gene editing and a DIY-bio movement that lets people keep labs at home, that mix looks dangerous.

Countermeasures exist but none is complete. Buyers of new genomes typically order DNA fragments from companies that screen for suspicious requests; responsible researchers run red-teaming and blue-teaming exercises; AI companies have tuned their tools not to supply abusable scientific information. On September 10, Anthropic disclosed in a report that someone had tried to use its models to explore how to increase chikungunya virus transmissibility, create more dangerous bird flu strains, and build an atlas of animal-venom-derived toxin peptides. Whether the executives' words change anything may hinge on how far the screening and red-teaming described here can be tightened before the next attempt slips through.