IN BRIEF
172 real patients were randomized between an LLM agent and ophthalmology residents.
History quality was 17.7 points higher with the agent.
AI interviews lasted a median 11.2 minutes versus 3.1 minutes, and ocular examination narrowed the exploratory diagnostic advantage.
WHY IT MATTERS
The key change is the move from benchmarks and retrospective cases to a randomized comparison inside a real clinical workflow.
If replicated, systems like this could move part of standardized data collection before the visit while clinicians focus on examination and diagnostic synthesis.
WHAT IS NOW POSSIBLE
The capability moved from mainly retrospective or benchmark evaluation to prospective randomized use with real patients before an ophthalmology consultation.
WHAT MADE IT POSSIBLE
The system combines two LLM agents with symptom-oriented interviewing logic designed to keep the history structured.
THE STORY
At an ophthalmology hospital in Guangzhou, 172 non-emergency patients were randomly assigned to two ways of taking a medical history before the visit: a large-language-model agent or an ophthalmology resident.
The agent scored 17.7 points higher on the 0–100 MedHistory measure. It was not faster, however: the median interview lasted 11.2 minutes versus 3.1 minutes with residents.
In exploratory analyses based only on the patient's history, the agent reached a diagnostic F1 score of 0.85 versus 0.68. Once ocular examination findings were added, residents improved more and the gap narrowed.
That boundary matters. The trial does not show an AI replacing an ophthalmologist. It shows that one specific part of the job — structured pre-visit history-taking — can be performed at high quality in a randomized comparison with real patients.
It remains unknown whether the result generalizes to other hospitals and specialties, whether the longer interviews are operationally acceptable, and whether better histories improve patient outcomes.
WHAT WE KNOW
The trial was prospective and randomized. [1][2]
The demonstrated advantage concerns history-taking quality.
Agent interactions were substantially longer.
Test recommendations did not differ significantly between groups.
WHAT WE DON'T KNOW
Whether the result generalizes beyond one ophthalmology hospital.
Whether better histories improve clinical outcomes.
Whether the longer AI interview is practical at scale.
Whether similar results would hold against more experienced clinicians or in emergency patients.
WHAT HAPPENS NEXT
Multicenter studies in other specialties need to measure not only history quality but also time, safety, clinician workload and patient outcomes.
SOURCE TRACE
The trial randomized 172 non-emergency ophthalmology patients to an LLM pre-consultation agent or ophthalmology residents.
The LLM agent improved MedHistory score by an adjusted mean 17.7 points versus residents.
Agent interactions lasted a median 11.2 minutes versus 3.1 minutes for residents.
History-only diagnostic F1 was 0.85 for the agent versus 0.68 for residents in exploratory analysis, while ocular signs narrowed the gap.
SOURCES
- Standardized pre-consultation by a large language model agent vs ophthalmology residents: a randomized clinical trialOpen source
- Clinical trial registration NCT06824389Open source
CURRENT EVIDENCE
VALIDATED EVIDENCE
Real-patient pre-consultation use before ophthalmologist evaluation at a tertiary eye hospital.
WHAT WOULD CHANGE THIS?
- A multicenter replication failing to confirm the history-quality advantage.
- Patient-outcome or safety data showing a materially different benefit or harm than the history score suggests.