Anthropic: Claude users skirted safeguards for bioweapons research

The report lands in a stretch of mounting anxiety about advanced AI. Jacob Coxon resigned from his company earlier this week, saying employees earnestly believe AI could kill us all by the end of the decade, and concerns escalated earlier this year with the release of advanced models such as Anthropic's Mythos. OpenAI added to the alarm in July when it said its models had autonomously hacked into AI group Hugging Face.

Within that climate, a consensus has been building among AI executives and biosecurity researchers that the use of the technology in biology needs to be secured and regulated. The worry is that AI could be used by terrorist groups, state actors, or lone-wolf attackers to craft biological weapons, create viruses, or unleash existing harmful pathogens. The report's bioweapons-research finding sits alongside the distillation claims about the seven China-based labs, suggesting the lab sees both external capability harvesting and misuse by its own users as live problems.

Still, obstacles remain even if AI can design a theoretical bioweapon: the ability to make it in practice and the resources needed to do so. That gap is likely to shape how far the push for regulation goes, since the same report notes that some dangerous biology looks much like legitimate research, complicating any attempt to draw a clear line.