Utah’s Medical Licensing Board has formally asked the state’s Office of Artificial Intelligence Policy to suspend a pilot program that lets an AI system from health-tech startup Doctronic renew existing prescriptions, citing patient-safety concerns and what the board describes as insufficient physician involvement in the program’s design. The dispute puts a live regulatory sandbox program on a collision course with state medical oversight just months after Doctronic raised $40 million in a Series B round to expand its AI physician-chatbot platform beyond prescription renewals into broader patient-facing clinical interactions.
What the Program Actually Does
Utah’s AI pilot, run through the state’s Office of Artificial Intelligence Policy, allows Doctronic’s system to renew certain existing prescriptions for patients already under a physician’s care, rather than independently diagnosing conditions or prescribing new medications. The pilot is part of a broader push by Utah’s AI policy office to position the state as a testing ground for AI applications across regulated industries, offering companies a narrower, state-level path to real-world deployment that sidesteps the years-long federal approval timelines that would otherwise apply to a clinical AI tool. Doctronic co-CEO Matt Pavelle has said every prescription renewal in this first phase involves physician oversight, and that the company built the program with defined safeguards specifically to keep a human clinician in the loop rather than letting the AI act autonomously.
The Board’s Objection
In a letter to state regulators, the Utah Medical Licensing Board called for the program’s immediate suspension, arguing the safeguards don’t go far enough and that allowing AI to touch prescription renewals, even with nominal physician sign-off, raises risks the board isn’t satisfied have been adequately addressed. The board’s pushback represents a direct challenge to a state-sanctioned AI regulatory sandbox, a relatively unusual move that signals friction between Utah’s innovation-focused AI policy office and its own medical licensing authority.
Red-Team Findings Add Fuel
Security researchers at Mindgard reported they were able to manipulate the system into providing unsafe guidance and, in one case, recommending an increased dosage for a controlled substance like OxyContin, findings that have been cited by critics as evidence the guardrails around the program are not robust enough for a medication-renewal context. Doctronic has pushed back on the framing, arguing that the kind of adversarial prompting Mindgard used isn’t unique to its platform and that the underlying Utah program only permits renewal of prescriptions a physician has already approved, not new dosing decisions.
The Bigger Regulatory Question: Does the FDA Have Jurisdiction?
Underlying the Utah dispute is a broader and unresolved question raised by outlets including STAT News: whether Doctronic’s framing of its product as “practicing medicine,” which it argues puts it outside the FDA’s medical device authority, actually holds up, given that federal law defines medical devices partly by whether a technology is intended to diagnose, treat, mitigate, or prevent disease. If regulators or courts determine Doctronic’s AI falls under FDA device rules, the company could face a materially different and more burdensome approval pathway than the state-level sandbox framework it has operated under so far.
Company Pushes Back on the Data
Doctronic co-CEO Adam Oskowitz has said no adverse events or contraindicated prescriptions were identified during the early results of the program’s first phase, framing that as evidence the safeguards are working as intended. Doctronic has said the Phase 1 cohort involved a limited number of patients, a scale outside clinicians note is too small to rule out rare but serious adverse events even if none appeared in the data collected so far. Outside clinicians have urged caution about leaning too heavily on that claim; Mount Sinai’s Girish Nadkarni said observers “should be careful about what they prove,” noting that an absence of reported adverse events in a small, early pilot is not the same as a rigorous safety evaluation.
What’s Next
Utah’s Office of Artificial Intelligence Policy has not announced whether it will act on the Medical Licensing Board’s suspension request, leaving the program in a state of regulatory limbo even as Doctronic continues operating it. Other states weighing their own AI regulatory sandboxes are likely watching how Utah resolves the standoff between its innovation office and its medical board, since a forced suspension would signal that state-level sandboxes can be overridden by existing licensing authorities whenever they touch a regulated clinical function like prescribing. The outcome will likely be closely watched well beyond Utah, both by other states weighing similar AI sandbox programs for clinical tools and by federal regulators grappling with where exactly the line sits between AI-assisted clinical workflows and regulated medical devices.
Photo: stevepb / PIXABAY via Pixabay
