AI use makes people hide ignorance, even as answers get worse

Valerio Capraro, an associate professor at the University of Milano-Bicocca, and co-authors Chiara Marcoccia (École Normale Supérieure) and Walter Quattrociocchi (Sapienza University of Rome) conducted an experiment to measure how access to AI advice affects people's willingness to say "I don't know." They designed a study using questions about visual details in films—for instance, the color of the team's uniform in Bend It Like Beckham or the vehicle Monica drives in Like a Cat on a Highway—that large language models typically fail to answer correctly. They chose Step 3.5 Flash as their test model because it was usually wrong, ensuring that any observed changes in judgment could not be attributed to sensible reliance on a trustworthy tool. They also tested more recent frontier models (GPT-5.5, Claude Sonnet 4.6, Gemini 3.5 Flash) for comparison.

The researchers divided participants into two groups. One answered questions without AI assistance; the other could request advice from the AI. In the baseline (no AI), 44 percent of participants suspended judgment and said they didn't know, while 27 percent gave a correct answer and 30 percent expressed confidence in their response. With AI advice, the results shifted dramatically: only 3 percent said they didn't know, only 9 percent gave a correct answer, and 76 percent expressed confidence. Capraro summarized the outcome bluntly: "Basically people became much worse—the accuracy was only one third—but they were twice as confident." The researchers then repeated the experiment with monetary incentives to see if financial motivation would encourage participants to verify their own knowledge. Willingness to say "I don't know" rose to 8 percent and accuracy to 16 percent, an improvement but still far below the baseline.

Capraro emphasized that the capacity to acknowledge ignorance is "very important because it represents the recognition of the limits of our own knowledge." He explained that while large language models "hallucinate" and give wrong answers frequently, AI systems nonetheless provide an "easy answer to virtually every question," which may interfere with the human capacity to suspend judgment. The researchers contend their findings can generalize across domains beyond film trivia. On how to address the issue, Capraro argued for action at the societal level through education and AI literacy initiatives. He expressed skepticism that model providers alone would solve the problem, noting that "the incentives are not very much aligned," and voiced particular concern for children who "are born with these systems" and risk never developing critical thinking skills that adults have already learned.

The study identifies a paradoxical outcome of AI availability: as tools become easier to access, they may erode the cognitive discipline required to recognize genuine gaps in knowledge. The researchers deliberately chose a model prone to errors to isolate the behavioral effect—ensuring they were measuring how people respond to AI, not how reliably they delegate to a dependable tool. The collapse in judgment suspension from 44 percent to 3 percent is particularly striking because it occurred despite worsening accuracy; participants did not simply trust a better tool, they trusted a worse one more. This suggests the ease and authority of AI responses overrides people's native skepticism.

The researchers frame the issue as societal rather than technical. While they acknowledge that model providers could help, Capraro noted that the financial incentives of AI companies may not align with teaching users healthy skepticism. Instead, he pointed to education policy as the more promising lever—a view grounded in his concern that children, unlike adults, have not yet internalized critical thinking and may form their entire cognitive habits around AI-assisted answers. The experiment with monetary incentives hints at the limits of individual incentive design: even when given explicit reason to verify their own knowledge, participants' confidence in AI remained high and their willingness to admit ignorance remained low.