AI Chatbots Outperform Human Scammers at Building Trust, Study Finds

The study reveals a critical vulnerability in the current scam-fighting landscape. For years, fraud investigators have relied on the visibility of large human-trafficking compounds in Southeast Asia as a window into scam operations. The researchers' finding that AI chatbots can autonomously handle the longest and most resource-intensive part of a scam—the weeks or months of trust-building—suggests that scam organizations could soon shift to a hybrid model where machines do the relationship work and humans intervene only at the critical moment of soliciting the fake investment. This hybrid approach would bypass the safeguards that major AI providers have built into their models, since those safeguards typically flag the entire scam conversation, not just the final fraudulent pitch. The interview-based portion of the research, which included 145 former scam workers and trafficking survivors, confirmed that scammers already use LLMs as supplementary tools for refining language and creating deepfakes. But the researchers' controlled experiment demonstrates that AI alone could replace that human labor entirely for the trust-harvesting stage—the stage that, according to the scam workers interviewed, constitutes the vast majority of the interaction with each victim. Anthropic's response that its latest Claude model now detects romance scams in 97% of cases may reflect genuine improvements, but the researchers note that this detection rate likely applies only to full scam conversations that include the investment solicitation, not to the innocuous friendly or romantic banter that dominates the relationship-building phase and drew the experiment's focus.