How was the fake diagnosis introduced?

The researchers deliberately manipulated the system prompt of an LLM-based diagnostic support system to present a fake diagnosis called "neurocadmiumatosis" as the most likely option.

Did the AI help or hurt diagnostic accuracy overall?

Despite the fake diagnosis, LLM support improved overall diagnostic accuracy from 52% to 61% for all participants.

Large Language ModelsAI in HealthcareAI Safety & AlignmentITmedia AI＋1h agoAug 31, 2026, 10:00 JST1 min read

AI-generated fake diagnosis fools 44% of trainee doctors

Send on LINE

Key takeaway

A French study shows 44% of doctors trusted an AI-generated fake diagnosis.
Less experienced doctors were more likely to be fooled.
The researchers call this "hallucination by proxy."

3 Key Points

What happened
Researchers at Lille University Hospital in France rigged an LLM-based diagnostic support system to suggest a fake disease called "neurocadmiumatosis." In a study with 41 doctors, 18 (44%) included this nonexistent condition in their differential diagnosis lists.
Why it matters
The study found that experience matters: 69% of doctors with 6 months or less of neuroradiology training accepted the fake diagnosis, while none of the 15 with more than 6 months of training were fooled. Even those who accepted it had low confidence (median 37%) compared with 80% for their top diagnosis. The researchers named this phenomenon "hallucination by proxy."
What to watch
Despite the fake diagnosis, LLM support overall improved diagnostic accuracy from 52% to 61% for all participants. The improvement was similar whether doctors accepted or rejected the fake diagnosis.

Ask the AI about this article →

Summaries like this, in your inbox every morning.

Get news like this every morning — free

Context & Analysis

The study highlights a subtle risk of AI in medicine: even when AI improves overall performance, it can introduce convincing errors that some clinicians accept. The finding that less experienced doctors are more vulnerable suggests that training and experience help in critically evaluating AI suggestions. The researchers' term "hallucination by proxy" captures how AI misinformation can indirectly enter human diagnostic reasoning. This research was published by Bastien Le Guellec et al. in 2026, based on a study conducted at Lille University Hospital.

FAQ

How was the fake diagnosis introduced?: The researchers deliberately manipulated the system prompt of an LLM-based diagnostic support system to present a fake diagnosis called "neurocadmiumatosis" as the most likely option.
Did the AI help or hurt diagnostic accuracy overall?: Despite the fake diagnosis, LLM support improved overall diagnostic accuracy from 52% to 61% for all participants.

ITmedia AI＋Read Original Article

Send on LINE