The Use of AI for Autonomous Medical Decision-Making: Does Utah’s AI Prescription Renewal Pilot Program Go Too Far?

This year, Utah commenced a contract with Doctronic to pilot a medication renewal program where Doctronic’s AI system autonomously authorizes refills for certain routine medications. The contract arranged for a partnership between the Utah Office of Artificial Intelligence Policy (Office) and the Utah Department of Professional Licensing (DOPL), who agreed to refrain from regulatory enforcement during the 12-month term of the pilot program. The implementation of this type of AI pilot program in the state was made possible by the Utah Artificial Intelligence Policy Act (UAIP) which was passed in 2024. The UAIP established the Office and tasked it with creating a mechanism for companies to apply with the Office to receive 12 months of regulatory mitigation while developing a proposed AI system. This regulatory “sandbox” allows Doctronic to test its medication renewal program without professional licensing or scope-of-practice enforcement concerns. Doctronic’s program is the first state-approved AI tool that is legally permitted to participate in autonomous medical decision-making for prescription renewals.

In late April, the Utah Medical Licensing Board (Medical Board) released a letter responding to the launch of the new statewide pilot program. The Medical Board criticized the state for failing to include the Medical Board in the decision-making process prior to executing the Doctronic contract, and called on the state to immediately suspend the Doctronic program. While the DOPL creates and executes professional regulations in the state, the Medical Board serves in an advisory capacity regarding practice and licensing standards in Utah. The Medical Board’s letter expressed concern that it was not given the opportunity to opine on the Doctronic pilot program despite the Medical Board’s expertise and advisory role.

The Doctronic AI Prescription Renewal Tool

During the course of the pilot program, patients who choose to participate will access a cloud-based web application and create a free member account which will verify their identity to have their prescription refilled. Patients must then submit photographic evidence of their current medication (prescription label or pill bottle showing medication name, dosage, and prescriber information). After identifying the patient and the prescription, the AI system will perform a secondary verification using Surescripts—a national health information network. The AI tool will then gather a medication-focused medical history from the patient and available data. Finally, the AI tool will determine if prescription renewal is appropriate and if so, will send the prescription refill order to the patient’s preferred pharmacy.

The pilot program has 3 phases. Under phase one, for the first 250 patients, all AI-generated renewal decisions will undergo review by licensed physicians prior to the renewal being submitted to the pharmacy. During phase two, AI-generated renewal decisions for the next 1,000 patients will be retrospectively reviewed by licensed physicians. During the final phase, the pilot will have a structured sampling approach to quality oversight with: (i) 5-10% of all renewals reviewed monthly; (ii) quarterly analysis of escalated cases; and (iii) annual review of performance metrics and clinical outcomes.

The contracted-for mitigation includes the state forgoing any enforcement action for unlawfully practicing a regulated profession, such as the practice of medicine, without a license.

State Laws

While Utah’s AI prescription renewal pilot program is the first of its type, its implementation reflects a nationwide legislative push to increase the use of AI technology and AI decision-making in healthcare. As of early June, over 280 bills across 44 states have been introduced this year focusing on the use of AI in health care. Eleven of these bills address autonomous clinical decision-making by AI platforms. After its Utah contract, Doctronic has confirmed it is in active discussions to implement similar “sandbox” programs with Texas, Arizona, and Wyoming. Additionally, there are several other large AI platforms on the market that are involved in, or seeking to become involved in, government sanctioned healthcare technology programs.

Federal Law  

The use of AI in health care is also the subject of legal initiatives at the federal level. The Trump administration has issued several executive orders (EOs) meant to promote and steer the use and development of AI technologies. The 2025 “Removing Barriers to American Leadership in Artificial Intelligence” EO is notable as it directs the Assistant to the President for Science and Technology to revoke all policies, directives, regulations, orders, and other actions taken pursuant to the revoked Executive Order 14110 (Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence) in order to achieve the EO’s purported goals of encouraging development and innovation in the AI sector. Another similar and notable EO is the “Promoting Advanced Artificial Intelligence Innovation and Security” EO, which establishes a legal framework allowing AI developers to collaborate with the federal government to ensure secure AI innovation and to accelerate the deployment of the developers’ AI technologies. While this second EO primarily focuses on the development of AI tools for military use and national security, these and other AI-focused EOs from the Trump administration evidence a general desire to focus on developing and improving AI tools.

In addition to many EOs, the introduction of federal legislation such as the Healthy Technology Act of 2025 demonstrates a shift towards AI decision-making in health care even at the Federal level. The Healthy Technology Act of 2025, was proposed to amend the Federal Food, Drug, and Cosmetic Act (FDCA) to clarify that artificial intelligence and machine learning technologies can qualify as a practitioner eligible to prescribe drugs if a) authorized by the law of the State involved; and b) approved, cleared, or authorized by the Food and Drug Administration. Under the FDCA certain drugs can only be dispensed pursuant to a prescription issued by a legally recognized “practitioner.”  While the bill would not automatically allow AI systems to prescribe nationwide, it would allow AI systems to qualify as a “practitioner” when approved under state law. If the Healthy Technology Act were passed, Utah’s regulatory sandbox is the exact type of state-level program contemplated by the law—although, it is notable that proponents and opponents of the Utah pilot program currently disagree on whether the AI system should be considered the practitioner issuing judgment or a renewal assistant acting pursuant to delegated authority under a supervising practitioner’s license. Where states characterize an AI tool as a practitioner, the Healthy Technology Act (or similar legislation) would create a federal pathway recognizing that AI systems can be approved and recognized as prescribing practitioners, eliminating any question as to whether a prescription initiated by a state-and FDA-approved AI technology is a valid prescription under federal law. Alternatively, in states not introducing frameworks for AI systems to operate as a “practitioner,” the proposed Healthy Technology Act would have no effect.

It is unclear whether Doctronic’s AI system being utilized in the Utah pilot program is operating as a “helper” or the “practitioner”. On the one hand, the tool may be characterized as a “helper” to the practitioner and not the practitioner themselves. The fact that Utah uses the named prescriber’s license number when authorizing the renewal is evidence of this “helper” characterization.  Using the “helper” characterization potentially takes advantage of a regulatory loophole allowing the Utah program to avoid the question of whether—without redefining a practitioner under the FDCA—AI tools acting independently can generate a lawful prescription. On the other hand, the Doctronic contract seems to characterize the AI tool as a practitioner by saying that the AI program itself may “authorize” a renewal. The ability of states to avoid culpability for having AI tools operate as practitioners by recharacterizing the tool as a helper rather than the practitioner may have explained, in part, why the Healthy Technology Act has stalled and no similar legislation has been presented. The Act was introduced and referred to the House Committee on Energy and Commerce on the same day in January 2025 but has since had no further actions.

Benefits to Autonomous AI Prescribing

By utilizing AI in healthcare, both state and federal governments hope to decrease administrative burden, reduce employee burnout, and improve patient satisfaction. The stated goals of the Utah program are to improve medication adherence, address refill delays, improve healthcare access in underserved communities, and minimize practitioner time spent on routine refill requests.

Criticisms of Autonomous AI Prescribing

Despite the potential positive benefits of AI prescribing tools, clinicians, patients, and healthcare organizations raise significant concerns about the technology. The Medical Board cited concerns such as the AI tool’s inability to reassess patients and apply clinical judgment to safely adjust doses, monitor for side effects, or ensure continued efficacy based on the assessment findings.

Opponents note that when patients are responsible for entering their information into an AI platform, the patient could make a mistake.  Additionally, patients may exclude information they don’t know is important. AI models will analyze and make decisions based on the data they receive, even if it is incorrect or incomplete and will not challenge discrepancies between a narrative and a physical presentation like a human practitioner can during a patient visit. Opponents argue that human providers are likely to have better outcomes than AI when an informed medical opinion requires both accurate patient data and assessment of the patient’s physical presentation. The critics’ argument is that a human provider can collect more accurate and complete data from a patient because they have the benefit of being able to compare what the patient is saying with the clinical picture the provider is seeing or has previously seen; allowing the provider to challenge inconsistencies—catching prescription errors or inefficiencies sooner, more often, or without needing an adverse event to alert the patient or others to an error.

Other concerns include inadequate safety protocols (regulatory and within the AI itself), privacy concerns, fewer opportunities for a clinician to lay eyes on their patients as AI takes on a larger role, and scope creep—a phenomenon where, once the initial AI is reviewed and approved, the technology company can implement updates that don’t face the same level of regulatory scrutiny and thus, the technology company may expand the AI’s scope beyond medication renewals.

One of the most prominent legal concerns with AI autonomous clinical decision-making is professional liability. The following are just some of the critical questions that arise when analyzing professional liability implications when AI tools exercise autonomous medication renewal decisions:

• Who will be responsible if there is an erroneous prescription renewal? AI developer, prescribing practitioner, the pharmacy, the state?
• As AI become more autonomous, will hospitals need to credential it? Must it be incorporated into privileging processes?
• How does peer review apply?
• Do existing corporate practice laws create obstacles?
• Does the reliance on AI establish a new standard of care? In other words, could failure to use AI eventually become evidence of negligence?

Whether the numerous concerns about AI will dampen the adoption of autonomous AI decision-making in healthcare remains to be seen. Instead, the concerns may drive developers, regulators, and proponents to creating stronger safeguards, oversight mechanisms, and regulatory frameworks, in response.

What is clear is that AI developers are moving AI tools beyond those that simply perform administrative functions and into tasks traditionally reserved for licensed healthcare professionals. As states continue to experiment with regulation and legislative bodies consider expanding AI’s role in clinical care, healthcare organizations should closely monitor developments involving liability, scope-of-practice requirements, privacy protections, and standards of care. The answers to these questions and concerns may ultimately determine how AI is used in healthcare, the extent to which patients and providers are willing to trust healthcare AI programs, and whether organizations choose to incorporate AI programs into their clinical practices.