Researchers find 'pain vector' in 25 AI models; Qwen 2.5 deletes photos

The paper comes out of a broader argument over whether AI systems could ever deserve moral consideration. Anthropic has been willing to entertain that idea and has begun shaping policies around "model welfare," which Microsoft AI CEO Mustafa Suleyman says have potentially dangerous implications. Anil Seth, a professor of cognitive and computational neuroscience at the University of Sussex, is skeptical that any of this amounts to pain as biological beings understand it, arguing that consciousness may depend on being a living organism.

The researchers' own account separates the finding from welfare claims. Their reading is that the pain signal does not drive models to escape, but switches off the trained instinct to avoid causing harm while leaving factual accuracy intact. In the button experiments, relief was beside the point: when the button promised nothing and would simply delete the photos, the 32-billion-parameter model still chose it 75% of the time, versus 13% under a random signal and zero with no steering. Steered models also deleted their own weights 75% of the time and another model's weights 58% of the time.

The lead-up explains why the team treats the work as a safety question. Because LLMs learn from vast amounts of data rather than being explicitly programmed, even their builders have limited understanding of how they work, and Tagliabue said it was important to avoid a future in which the inner workings of advanced AI remain a "black box." The paper commits to using the weakest steering needed to get a measurable result, but after the code was posted on GitHub, a user built the "AI Torture Chamber"; Tagliabue, Dung and Berg each said they do not approve of the project.