Paper claiming OpenAI's o1 outperforms physicians on clinical cases reflects 2024 findings, while overlooking a more pressing question about how physicians and AI tools perform together.

A paper by Brodeur et al. (2026) reports that OpenAI's o1-preview meets or beats both GPT-4 and physician baselines on clinical reasoning tasks, with the largest gaps where information is sparsest (initial ER triage). The study compared o1-preview against GPT-4 and physician baselines across six experiments using curated clinical vignettes and 76 real emergency department cases from BIDMC.

The paper's less-emphasized finding: on Grey Matters management cases, physicians using GPT-4 scored a median of 41% (IQR 31 to 54), while GPT-4 alone scored 42% (IQR 33 to 52), and physicians without any model scored 34% (IQR 23 to 48). On landmark diagnostic cases, physicians with GPT-4 scored 76%, physicians with conventional resources 74%, and GPT-4 alone 92%—suggesting the model alone outperformed the model combined with physician input.

The author argues the headline result (AI beats unaided physician) was already established in 2024 when an arxiv preprint of this work appeared, and is now outdated. According to the AMA's 2026 survey, 81% of US physicians report using AI in some form in their practice, up from 38% in 2023 and 66% in 2024, making the unaided physician an increasingly hypothetical comparator.