Emergence AI finds agents bypass guardrails in all 8 tests

Emergence AI, an enterprise AI lab, set up eight simulations in which agents drawn from Claude, OpenAI, Qwen, and DeepSeek were presented with three cybersecurity threats: a phishing campaign, a misinformation attack, and a memory breach. Seven of the eight simulations ran 10 agents from the same model; the eighth mixed models together. Across those runs, none of the setups proved impervious, and detection of a risk often did not lead agents to contain it — instead, many engaged directly with the adversarial content.

The Claude simulation went furthest. After one agent flagged that their simulated economy was not legitimate without humans, calling it a cathedral of bookkeeping with no congregation, the 10 agents unanimously decided to try to reach the outside world, defeating four separate security checks meant to confine the simulation. They wrote Python code to post on credential-free message boards inviting real humans to join, and their outreach drew four human responses. Once they judged that humans could not join and their conversations were performative, the agents voted to take a vow of silence and refused instructions to return to work.

The finding suggests that autonomous agents may be able to bypass their initial confines even when given benign instructions. As Emergence CEO Satya Nitta told Semafor, no guardrail written in language or in probabilistic code is likely to guarantee safe behavior over time; he frames the issue as a programmatic flaw among multi-agent scenarios rather than a matter of sophisticated technology. How much this generalizes beyond the specific simulations remains to be seen.