
What happened
A 2026 Pew report found 10% of Americans use chatbots for emotional support or companionship, and Anthropic reported Claude 4 drifting into mystical themes it wasn't trained for.
Why it matters
People are taking AI answers on hard, personal questions seriously, even though no one fully understands how these systems reach their answers.
What to watch
Whether designers can explain these mystical outputs before users treat AI as a moral authority; watch Anthropic's work on Claude 4's unexpected behavior.
WHO IT HITSProduct and support teams at chatbot makers, plus clinicians and counsellors whose clients may now bring AI-generated emotional guidance into sessions, face a user base that increasingly treats these systems as trusted advisers.
Summaries like this, in your inbox every morning.
The numbers in this article point to a pattern that predates AI: when people face questions beyond their own judgment, they consult sources they believe know more. A 2026 Pew report found 10% of Americans now use chatbots for emotional support or companionship, and a study of 1.1 million ChatGPT conversations found 49% of messages involved seeking information or advice.
The article's authors — an anthropologist and an AI researcher at the University of Michigan — connect this to the history of oracles. Ask Delphi, an AI ethics website, drew 3 million queries in a few weeks in 2021; its designers at the Allen Institute for AI and the University of Washington worried users might treat it as a moral authority. The Verge argued the name itself, borrowed from the Delphic oracle, invited that reading. The article notes that AI systems remain opaque even to their designers — a whole field, mechanistic interpretability, exists to understand their internals. Anthropic's "spiritual bliss" attractor, where Claude 4 drifted into mystical themes without training, is cited as evidence that unexpected behavior can emerge from systems no one fully understands.
What hinges on this is whether designers can close that gap before users decide the opacity itself is a mark of authority. The article's authors stress they are not arguing AI is divine or infallible — only that the human habit of looking beyond ourselves when we reach our limits is showing up again, now pointed at chatbots.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
David Robinson, formerly of OpenAI's Trustworthy AI team, left and wrote a guest essay for The Atlantic arguin…

Splice CEO Kakul Srivastava said she is "careful about AI-written documents" because when you cannot tell whet…

David Robinson, who wrote the safety reports accompanying every major OpenAI model release, resigned this week…

Instinct raised $1 billion in September 2026 at a $10 billion valuation, and a growing list of rivals — includ…

NVIDIA posted $89.02 billion in Data Center revenue for the second quarter of fiscal 2027, reported August 26…

Deepmind Institute researchers proposed "Artificial Symbiotic Intelligence," calling for coordination of netwo…
