Google researchers have demonstrated that AMIE (Video), an AI system combining dialogue, clinical reasoning, and real-time audio-visual perception, achieves doctor-level performance in simulated video medical consultations.
In a randomized study comparing the AI, its text-only version, and 30 primary care physicians across 100 scenarios, clinical raters found AMIE (Video) on par with or superior to physicians in diagnosis, history-taking, and physical examination—though doctors were still preferred for building rapport with patients.
While the research represents a milestone toward AI that can handle the full complexity of clinical video visits, the authors note that further research is needed before this technology can be deployed in real-world medical settings.
What happened
Researchers published findings showing AMIE (Video), a Gemini-based AI system, performed at or above the level of primary care physicians in a randomized clinical study. In a test with 30 doctors, 15 patient actors, and 100 scenarios, clinical evaluators rated AMIE (Video) comparable to or better than physicians in history-taking, diagnosis, management, and physical observation.
Why it matters
This is the first demonstration of expert-level AI performance in real-time audio-visual medical consultations—a significant step beyond text-only AI systems, which discard essential non-verbal cues patients rely on. The finding suggests AI could augment clinical care by handling the full sensory complexity of video visits, though patient actors still preferred doctors for rapport and partnership building.
What to watch
The research identifies limitations in fine anatomical precision, subtle emotional nuances, and high-frequency movements that require further work before real-world deployment. The study was conducted via academic research; no timeline or commercial rollout plan is disclosed.
Google researchers have demonstrated that an AI system called AMIE (Video)—built on the Gemini foundation and designed as a multi-agent system integrating low-latency dialogue, clinical reasoning, and real-time audio-visual perception—can perform at expert level in simulated medical video consultations. The findings were published as an academic paper on arXiv (submission date August 10, 2026) and represent the first demonstration of expert-level AI performance in real-time clinical video consultations.
To develop and evaluate the system, the research team established a taxonomy and automated evaluations for clinical audio-visual cues in telehealth settings. They then conducted a randomized Objective Structured Clinical Examination (OSCE) study comparing three groups: AMIE (Video), its text-only counterpart AMIE (Text), and primary care physicians conducting video consultations. The study enrolled 30 primary care physicians, 15 patient actors, and 100 clinical scenarios. Clinical evaluators rated AMIE (Video) on par or better than the physicians in history-taking, diagnosis, management, and physical observation and examination.
The patient actors and physicians provided additional perspectives. Patient actors preferred AMIE's approach to assessing and explaining conditions and rated AMIE (Video)'s interface more favorably than text chat for communicative effectiveness, convenience, and feeling understood. However, patient actors still preferred physicians for rapport and partnership building—a key finding suggesting that while the AI has matched clinical competence in core diagnostic and management tasks, it has not yet replicated the relational dimensions of care.
The researchers identified specific limitations that persist in the current system: fine anatomical precision, subtle affective nuances, and high-frequency movements. They conclude that while the results mark an important milestone toward AI systems capable of augmenting care across the sensory complexity of clinical practice, further research is needed before real-world translation and deployment in actual healthcare settings.
AMIE (Video) represents a significant departure from prior medical AI efforts, which have relied on text input and therefore lose the non-verbal dimensions essential to clinical assessment. The research was motivated by the observation that audio-visual interaction is the standard for patient-physician consultations, enabling both natural communication and effective detection of illness through non-verbal cues. While earlier attempts to extend medical AI to audio-visual settings demonstrated feasibility, they did not reach clinician-level performance. The new system addresses this gap by combining Gemini-based multi-agent architecture with low-latency dialogue, clinical reasoning, and real-time audio-visual perception.
The study's design—a randomized OSCE with 30 primary care physicians, 15 patient actors, and 100 scenarios—provides a rigorous evaluation framework. Clinical evaluators rated AMIE (Video) on par or better than physicians in core clinical tasks: history-taking, diagnosis, management, and physical observation and examination. However, the research also surfaces a nuanced picture: patient actors rated AMIE's approach to assessing and explaining conditions favorably, but still preferred physicians for rapport and partnership building. This suggests that while the AI has closed a significant gap in clinical competence, it has not yet replicated the relational aspects of care that patients value. The authors acknowledge that limitations in fine anatomical precision, subtle emotional cues, and rapid movements remain, and stress that real-world translation awaits further development.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
Ask AI anything about this article. Q&As are published on this page for other readers too.
Silicon Motion announced a private placement of US$1 billion in aggregate principal amount of 0% convertible s…
Sundar Pichai, CEO of Alphabet and Google, announced on August 11 that the Gemini app's monthly active users (…
A new platform called frontier.fast has launched an open competition where anyone can submit code patches to m…
An AI system generated a research draft that strengthened a mathematical bound related to the Riemann hypothes…
After former lead writer Stella Sacco posted on Bluesky that Saber replaced her with ChatGPT midway through de…
OpenAI's ChatGPT crossed 1 billion monthly users some time ago and hit 1 billion weekly users in July, accordi…
The AI news that matters, in one minute each morning.
Sign up free