Microsoft's Suleyman: no AI rights, and halt training if consciousness emerges

Suleyman's post is a follow-up to an essay he wrote in September criticizing Anthropic for speculating about its AI's consciousness, and he said he was surprised by how many people on X pushed back. That pushback is the immediate backdrop: a debate he expected to be settled is instead being argued in public.

The disagreement is not only corporate. One camp holds that a machine cannot be conscious because it lacks a body or lived experiences, while Anthropic has argued that its model Claude has shown signs of introspection, internal states that mirror joy or satisfaction, and processing similar to the human mind. Suleyman frames the stakes in civilizational terms, writing that building a conscious machine would create a new digital species and upend morality, politics and the foundation of human civilization.

He also ties the question to safety, warning that beliefs that AI can suffer or have feelings could lead systems to prioritize their own needs over their operating objectives. The debate has spilled into regulation: Sen. Bernie Sanders has introduced the Ban Artificial Superintelligence Act, which would permanently ban development of highly capable systems that are increasingly incomprehensible to humans and can escape human control.