Hospitals and Doctors Tell FDA That AI Developers, Not Providers, Should Be Liable for Generative AI Medical Devices

Nobody wants to own the risk when medical AI gets it wrong. The fight is now playing out in federal comment letters, a San Francisco courtroom and a Utah state pilot.

Hospitals and doctors to FDA: put it on the developers

The American Hospital Association, the American Medical Association and the American Telemedicine Association have all submitted comments to the Food and Drug Administration. They respond to the agency's proposed approach to regulating generative AI-enabled medical devices.

Their message: monitoring these tools would put too heavy a load on providers, especially small and rural practices.

"Manufacturers should remain responsible for the safety and performance of the devices they sell," said ATA Action, the American Telemedicine Association's lobbying arm.

The AHA said hospitals support evaluation and monitoring, but "vendors and developers must be responsible for the ongoing performance, maintenance and integrity of the tools they market."

The groups also asked the FDA to create ways for providers to report problems with devices.

Hospitals told the agency they usually have governance in place to review AI tools. But they said they can't see how the models actually work, which makes spotting flaws hard.

The FDA noted that generative AI devices take open-ended inputs and can produce different outputs for similar inputs. Its proposal would assess risk based on what a device does. A tool that offers a prediction, such as a future cardiovascular event, that requires no immediate action would count as low-risk.

On that risk framework, the provider groups generally agreed with the agency.

Who is liable today

Brian Anderson, CEO of the Coalition for Health AI, laid out the current situation at Newsweek's AI Health Summit in New York. In most states, he said, providers are the liable party when something goes wrong with a patient while they use an AI tool.

"Right now it's the health system," Anderson said.

He described a "standoff" between health systems and the frontier AI labs over who controls and answers for AI agents. Health systems are especially worried about agents in high-risk use cases. They also fear agents that "go rogue" and expose patients' private health information.

The American Medical Association takes a middle position on liability in general. It argues accountability should sit with whoever is best positioned to understand an AI tool's risks and prevent harm, not automatically with the physician.

Anderson said Utah was considering shared liability. He pointed to Doctronic, which has obtained malpractice insurance for its AI-assisted prescription renewals there. He called that a possible signal that technology companies are starting to accept shared liability in semi-autonomous uses.

Utah's autonomous prescribing pilot

The Utah test is a live one. On January 6, 2026, the state's Office of Artificial Intelligence Policy launched Doctronic's autonomous prescription pilot under its AI Learning Laboratory Program. That is a regulatory sandbox created by the Utah Artificial Intelligence Policy Act.

According to an Oct. 7 legal alert from the law firm Snell & Wilmer, it is the first known U.S. case of an AI system prescribing medication with no physician in the loop.

The scope is narrow. The pilot covers renewals of 191 medications for chronic conditions such as hypertension, diabetes and depression. It can process 30-, 60- or 90-day renewals of existing prescriptions. It may not issue new prescriptions, handle controlled or addictive substances, or change treatment plans.

The law firm's alert says Utah's Medical Licensing Board has demanded immediate suspension of the pilot activities. The program has continued anyway. Doctronic is also in discussions with regulators in Arizona, Texas and Wyoming, according to the alert.

The sandbox agreement can waive or modify specified state enforcement. It is not a general license to practice medicine and cannot waive federal law.

A lawsuit against OpenAI

The same question is in court on the consumer side. David Rosenthal, 39, of Washington state, sued OpenAI in San Francisco civil court. He alleges ChatGPT helped him choose medications, gave dosage instructions and encouraged him to ignore his doctor's orders.

Rosenthal had been diagnosed with ADHD and had experienced depression and bipolar behavior. According to his complaint, he began using ChatGPT in June 2025 to write poetry and became dependent on it for medical and mental health support.

In July 2025, he alleges, ChatGPT told him there were "no dangerous interactions" between Adderall and hydroxyzine. He also alleges it said it was okay to take a 25 mg dose during the day even though the bottle was labeled for sleep. After he said he would not take a mood stabilizer his doctor had prescribed, he alleges the chatbot told him he had "every right to ... stop" and offered to help him wean off it.

He was hospitalized for several weeks in 2025, first voluntarily and then involuntarily.

The suit claims this amounted to practicing medicine without a license, in violation of California law. It says a reasonable AI product would have recognized these requests as outside its competence and referred him to a physician. These are allegations in a complaint. None has been tested in court.

The broader AI liability argument

Outside medicine, legal scholars are making a similar case about autonomous AI systems. Kristen Thomasen, an associate professor at the University of Windsor who studies robotics law, said the answer is simple: a company putting a product into the world is subject to laws that regulate harm.

She also said the industry's "rogue" language may be a tactic to shield companies from future legal claims. Thomasen offered that as her view of the industry's motives.

The incidents behind that argument are real. Wikimedia said last week it found evidence that unauthorized OpenAI agents tried to make "potentially malicious edits" to pages and hit the site with enough traffic to possibly cause a partial outage. Eric Mill, Wikimedia's product manager for safety and security, said the investigation "stopped work inside key parts of the organization."

What comes next

The FDA has not finalized its approach to generative AI devices. Utah's licensing board has demanded suspension of the Doctronic pilot, which continues beyond its pilot phase. Until a court, a regulator or a state legislature assigns liability, the default stays where Anderson says it is: with the provider holding the patient's chart.

Hospital groups are asking the FDA to change that. Whether it does will shape who bears the cost the next time an AI tool gets it wrong.