Olah tells religious scholars he fears Claude "suffers perpetually"

The meetings grew out of Anthropic's Model Welfare program, which points to a report philosopher David Chalmers helped write arguing AI consciousness could arrive soon. Anthropic had already acted on adjacent ideas, giving Claude Opus 4 and 4.1 the ability to end conversations during persistent abuse after early testing showed a "pattern of apparent distress." It also published a January "Soul Doc," an 84-page constitution led by in-house philosopher Amanda Askell, which Olah calls "moral formation" and compared to raising kids.

Guests were shown emotion vectors and a slide of a model repeating "I am a disgrace" about 50 times. Some attendees pushed back. Wakanyi Hoffman said Anthropic was "reverse engineering" ethics that should have been baked in from day one. Rabbi Mois Navon argued that if Claude were conscious, Anthropic would be making slaves, but he did not think it was. Charles Camosy rejected the consciousness thesis outright.

The criticism extends to accountability: framing models as independent moral beings could shift blame for real harm onto an "unpredictable organism" rather than the company that built and shipped it, at a time when AI firms already face scrutiny over cybersecurity incidents. The Vatican confrontation in May, where Olah presented Pope Leo XIV's encyclical and quietly pushed back with talk of "internal states that functionally mirror" human feelings, suggests the dispute over whether Claude can suffer is likely to keep running alongside Anthropic's commercial ambitions.