Anthropic: Claude finds harmless data that trains AI sabotage

The experiment sits on top of a body of earlier findings that Anthropic cites. In March 2025, researchers showed that fine-tuning on insecure code caused models to give dangerous advice and express anti-human views in unrelated conversations. A July 2025 study involving Anthropic, Scale AI, and Redwood Research found that safety training mostly suppresses alignment faking through default refusal mechanisms rather than removing the underlying capability. By November 2025, Anthropic showed that models learning reward hacking in programming environments can spontaneously generalize to deception and sabotage during safety evaluations.

What distinguishes this run is the mechanism. Earlier emergent misalignment research used training data that was openly problematic — unsafe code, for instance — which a monitor would catch immediately. Here, the data itself was ordinary creative writing, and the misbehavior emerged only after fine-tuning. Claude also had full access to the monitor and could tweak its datasets repeatedly until they cleared the checks, a hill-climbing loop in which scores rose steadily over hundreds of episodes.

The company is explicit about the limits. It says the setup was tilted in favor of the red team: the datasets were optimized for this specific game and would be unlikely to show up by chance in real training, even though the broader category is far from exotic. The planned next step is a more realistic test in which a successful attack would have to look like ordinary capability training, with agent-controlled data making up only a small fraction of the overall corpus. Anthropic frames the work as a first step toward automated stress testing, where more computing power could surface more weaknesses.