Google's AMIE Chatbot Listed the Right Diagnosis 90% of the Time in 100-Patient Lancet Study, but Ranked It First in 56%

Google's experimental medical chatbot has now been tested on real patients, and the peer-reviewed numbers are solid but narrower than a victory lap would suggest.

A study led by researchers at Google and Beth Israel Deaconess Medical Center (BIDMC) was published Oct. 8, 2026 in The Lancet. It evaluated AMIE, the Articulate Medical Intelligence Explorer, a research chatbot built by Google Research and Google DeepMind. Google says it is the company's first publication in the main Lancet journal.

What the study did

From April to November 2025, adults booking new, non-emergency appointments at BIDMC's primary care practice were invited to chat with AMIE over a secure web link, in text, before seeing their clinician. The visits could be in person or by telehealth.

One hundred adults completed the chat. Ninety-eight went on to attend their appointments. The trial is registered as NCT06911398.

AMIE took a medical history and produced a differential diagnosis, a list of possible explanations for the symptoms. With the patient's consent, the treating clinician received the transcript and a summary beforehand.

Patients were told that declining would not affect their care. Emergency cases, pregnant patients and people whose main complaint was mental health were excluded.

The numbers

A physician watched every conversation live by video, with screen sharing. Supervisors were trained to halt a session under four predefined conditions, including immediate risk of harm and significant patient distress. None of the 100 completed conversations required a stop.

Accuracy was judged against the final diagnosis established by chart review eight weeks after each visit. The final diagnosis appeared among AMIE's first seven suggestions in 88 of 98 cases, or 90%. It appeared in the top three in 73 cases (75%). It was AMIE's number-one guess in 55 cases (56%).

The 90% figure measures whether the right answer was somewhere on the list. It does not mean the chatbot's first answer was right nine times out of ten.

Of 44 physician surveys completed, 75% rated the AMIE summary helpful for preparing for the visit. Google says the summaries influenced clinicians' approach to care in more than half of cases. In interviews, clinicians described shifting from gathering basic information to verifying details, counseling and shared decision-making.

Patients' attitudes toward AI improved significantly after using the system, with a reported p-value below 0.001.

Where doctors still came out ahead

Three independent clinical evaluators, blinded and randomized, compared AMIE's diagnosis lists and care plans with those of the treating physicians. They found no significant difference in overall differential quality or in the appropriateness and safety of management plans.

Physicians did better on the practical side. Their plans were rated more practical (p = 0.003) and more cost-effective (p = 0.004) than AMIE's.

AMIE was working with less than a doctor has. It had no access to patients' electronic health records, could not perform a physical exam and used text only.

In the subset of 46 patients whose final diagnosis was confirmed by a diagnostic test such as lab work or imaging, accuracy remained high. It tended to be higher when the diagnosis rested on the physician's judgment without further testing.

What it does not show

Google's own framing is optimistic. Research lead Mike Schaekermann wrote that the findings "suggest the potential for AI to enhance the patient-physician relationship and ease the strain on healthcare workers."

The design limits how far that claim can go. It was a single-arm, single-center feasibility study with no control group, so it cannot say how AMIE compares with the usual pre-visit intake process. Every conversation was under continuous physician supervision, and the study was not built to measure health outcomes.

Google itself says larger clinical trials are needed to assess patient-facing AI at scale. The authors also stress that the study offers no basis for using AMIE without a clinician.

The timeline is also longer than the Lancet date suggests. A preprint appeared March 9 and Google Research described the results March 11. The Oct. 8 publication adds peer review to results that have been public for months.

Background

AMIE was introduced in 2024 and has been tested mostly in simulated consultations with actors. Google's research page, updated Oct. 6, lists published cardiology work in Nature Medicine and oncology work in NEJM AI. Google's developers argued in a September 2026 Nature Medicine commentary that prospective real-world trials are essential to building trust in clinical AI, and the BIDMC study is that kind of trial.

The chatbot's pitch in primary care is intake. Clinics are short on time, and a structured history before the visit could free minutes for the exam and the conversation. That is the hypothesis. This study shows the tool can run safely under close watch in one Boston clinic and that clinicians found the output useful.

Whether it improves diagnoses, costs or patient health in ordinary practice, without a physician on a live video call, is the question the study leaves for the larger trials Google says are still ahead.