Anthropic bans abusive use of Claude, hedging on AI feelings

Anthropic's move sits at an awkward intersection: it asks users to treat Claude with restraint long before anyone can prove Claude has anything to protect. The company acknowledges the uncertainty directly, saying it does not know whether current or future AI systems have consciousness, pain, or preferences. Rather than wait for a settled answer, it placed a narrow rule that only catches repeated, extreme abuse.

The design echoes a familiar management problem. A team cannot prove a data leak or a key resignation will happen, but doing nothing until proof arrives can be too slow, while blocking everything stalls the work. Anthropic's terms try to split that difference by preserving ordinary criticism, dark fiction, testing, and research while drawing a line at persistent cruelty. It remains unclear who judges what counts as "unnecessary cruelty," and whether the line stays right as AI changes.

Anthropic's own cited research note, that Claude Opus 4 and 4.1 can end certain conversations, shows the policy has a technical mechanism behind it, not just a clause. The broader takeaway is procedural: prepare for uncertain futures with guardrails that keep today's work moving, rather than with walls that stop it.