Utah AI sandbox launches new health care pilots — with question marks

SANTA FE, N.M. — Utah announced three new pilots last week to test artificial intelligence capabilities that can refill chronic condition-related prescriptions, write initial acne prescriptions and offer virtual physical therapy, but a regulator acknowledged the state still has key questions to address as the model scales nationwide.

Utah regulators have “evidentiary standards, [and we are] working on governance guidance. We're working on a liability bill to begin building the statutory framework around” its AI sandbox, Zach Boyd, director of Utah’s Office of Artificial Intelligence Policy — or OAIP — said at the ADAPT conference in Santa Fe, New Mexico on Thursday. 

Through the AI sandbox, Utah has allowed companies to test new technologies in exchange for regulatory relief and state oversight. Utah started AI pilots in 2024 with a mental health support app, but the sandbox did not garner national attention until this year, as the state allowed AI to refill certain prescriptions with clinician oversight through a pilot with health technology company Doctronic.

The Utah Medical Licensing Board fiercely opposed that effort, calling for an immediate suspension of it in an April letter to the Utah Department of Commerce and criticizing the state for not consulting them.

“We hit a nerve,” Boyd said.

The Utah Medical Licensing Board has since formed a working group that meets with state regulators once a week, he explained, adding that the détente came from six months of discussions and updates to the AI pilot.

In an interview with Nextgov/FCW, Boyd noted that Utah removed some drugs from the list of prescriptions AI can refill following scrutiny from the Utah Medical Licensing Board. The list of 190 drugs dropped headache reliever Butalbital and antiarrhythmic medication Flecainide, according to OAIP.

But Utah regulators are still mulling over how the state should split legal liability between AI and the human clinicians who oversee them if patients are harmed.

Regulators have interpreted the law to mean that the AI sandbox does not have authority to change liability rules, but they have pulled several short-term levers to address the question, Boyd explained. OAIP has required Doctronic and AI pilot companies that have engaged with the sandbox since then to carry special liability insurance and not waive liability.

However, “in the long run, we think [insurance is] less sustainable, and we need a way to hold the vendor directly liable for the actions of their automated system,” Boyd said, adding that regulators expect the default is that physicians will bear too much liability.

Boyd hopes the state will unveil a liability bill this month or next month during Utah’s interim legislative session. The bill will kick off a “multi-year process” to refine liability, he said.

Another big question facing Utah’s sandbox is whether the Food and Drug Administration will regulate pilot AI technology. The FDA generally requires software that treats or diagnoses patients to undergo pre-market authorization, which some lawyers told STAT could include the pilot AI technology like Doctronic’s AI prescription refiller.

The FDA has so far been silent on the Utah experiment.

Boyd declined to speak for the FDA but said he hopes to work with federal regulators on creating a unified regulatory pathway for AI.

“It’s really appealing to people to have kind of a vet once, deployed everywhere, aggregate oversight across jurisdictions” model, Boyd told ADAPT attendees. 

“A model like that feeds nicely into the federal government and the kind of oversight that they’ll also want to exercise,” he added.

Utah has tapped six independent evaluators to assess AI companies’ technical claims and share results with the state. They include the Coalition for Health AI, Clarion AI Partners, Glacis Technologies, mpathic AI, Vega Health and Stanford Clinical Excellence Research Center.

Much like with liability, there are question marks about the independent evaluators.

Utah spelled out the general expectations for health care software piloted through the state’s AI sandbox. The technical document says those technologies should account for managing AI input uncertainty, clinical context, patients’ emotional intelligence and other factors throughout its pilot.

“Half of [the document] is probably wrong, but it's definitely, it's got to be, about the most mature thing that exists,” Boyd told ADAPT attendees.

With those expectations, future AI pilots will have to contract with an independent evaluator, Boyd said.

The data evaluators will assess will depend on the specific AI pilot’s intended application, patient population, outcomes and specifications, Mark Sendak, CEO and co-founder of Vega Health, told Nextgov/FCW. 

The startup would work with the state and the pilot company on crafting the appropriate data analysis while bringing in external partners with the necessary expertise to assess specific pilots, he said. 

But because AI models often change over time and will expand to serve new patient populations, Sendak expects independent evaluators to constantly repeat assessments and share that data with the state.

Utah has also signed master agreements with health systems Intermountain Health and University of Utah Health, allowing them to pitch AI pilots that the state can approve.

“One of the underrated advantages of a sandbox is it helps the regulator learn so much,” Boyd told ADAPT attendees.