Oljato – Monument Valley, Utah, USA – October 19, 2018 : Welcome to Utah State Sign situated along US-163 at the border with Arizona near Monument Valley.

...

Alexis Kayser

By Alexis Kayser

Healthcare Editor

0

Share

Newsweek is a Trust Project member

See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.


This is a preview of the October 8 edition of Access Health—Tap here to get this newsletter delivered straight to your inbox every Thursday.


Good afternoon. Utah captured the nation's attention this week as it took another major step toward allowing artificial intelligence to perform clinical tasks traditionally reserved for licensed medical professionals.

Read More on Health Care

On Monday, the state announced an expansion of its AI regulatory sandbox, introducing five new agreements and a network of independent evaluators tasked with assessing the safety and effectiveness of participating technologies.

Among the new pilots is one from Nolla Health that could eventually allow AI to prescribe certain acne treatments without physician approval. Another, from August AI, will explore automated prescription refills for chronic disease medications, with physician authorization required.

Intermountain Health and University of Utah Health also received agreements to expedite reviews of future AI pilots. The state hopes the resulting evidence will inform permanent regulations and potentially guide other states.

This week, I spoke with Dr. Brian Anderson, CEO of the Coalition for Health AI (CHAI), and Brenton Hill, the organization's general counsel. CHAI is one of six independent evaluators partnering with Utah.

Anderson sees the initiative as an opportunity to generate evidence that voluntary industry standards alone cannot provide.

"I think some of the most urgent questions are centered around [whether] can these models be trusted," he said.

Hill emphasized that the sandbox isn't intended to give developers free rein. Instead, regulators can evaluate new applications individually before deciding whether broader adoption is appropriate.

"We're not going to go roll these out all at once. We're going to test them one by one to make sure that they're safe and effective," he said, describing the state's approach.

The implications extend beyond Utah, particularly as other states consider similar regulatory frameworks.

And Utah's experiment is just one piece of a much larger debate over how AI should be governed. That debate is the subject of Newsweek's cover story this week, which explores how the technology is reshaping American politics and scrambling traditional partisan divisions. Our new AI Policy Scorecard also lets you see where every member of Congress stands on five major areas of AI policy.

There's more from Anderson and Hill on what this experiment could mean for hospitals, patients and the future of AI regulation in this week's Pulse Check—read on!


In Other News

Major health care headlines from the week


Pulse Check

Executive perspectives on key industry issues

Brenton Hill (left) is general counsel for the Coalition for Health AI (CHAI), and Dr. Brian Anderson (right) is its CEO.

...

Below, find a portion of my interview with Dr. Brian Anderson, CEO of CHAI, and Brenton Hill, the organization's general counsel. We discussed Utah's new health care AI regulatory sandbox, CHAI's role in evaluating participating technologies and what the experiment could mean for hospitals nationwide.

Editor's Note: Responses have been lightly edited for length and clarity.

For people reading about Utah's announcement for the first time, what's most important to understand?

Anderson: Part of the importance of this news is all about building trust and generating the kind of evidence that we need to move at the speed we need to move in the health care space to deliver the safest, most effective AI to the residents of Utah.

Part of CHAI's excitement is partnering with the Office of AI Policy and the Pro-Human AI Initiative to begin generating evidence on the kinds of AI models that we can trust.

Our role, as a member of the third-party evaluator ecosystem, is to help bring together a network of third-party evaluators to test these models and monitor their performance as part of the AI sandbox approach.

The opportunity is to rapidly test and monitor AI models, generating evidence around model performance that the state can then use to make regulatory decisions about those models.

One of CHAI's roles is to bring together evaluators who will conduct evaluations according to best practices generated through a clinician-led, industry-driven consensus process.

That might involve an algorithm to identify patients at high risk for sepsis or generate prescriptions, as we've seen with some of the semi-autonomous or autonomous agentic tools in Utah.

What are the best practices for developing those models? What are the best practices for deployment, governance and monitoring? What are the metrics and methods for evaluating them?

Bringing that consensus perspective to a set of evaluators who will approach evaluation and monitoring with the rigor that society demands is an important step. I think that is a critical path to unlocking the use cases AI promises, including delivering care to people in rural Utah who might otherwise have to drive hundreds of miles to see a provider.

How is Utah's approach different from what we're seeing in other states? Is it truly one of a kind?

Hill: I would say the Utah sandbox is one of a kind in that they are one of the first, if not one of the only ones, to actually have an infrastructure in place to test and deploy these technologies on real patients within their state.

As far as I know, they're leading the nation in taking those licensing requirements, providing regulatory relief from them and allowing direct-to-consumer applications that would normally require medical licensing.

What's pretty unique is that they're leading the way and are so willing to share what they're doing with other states looking into similar programs.

The other thing being explored, which I find really impressive, is reciprocity with other states.

If another state has a sandbox, and a technology has gone through Utah's process, received the requisite approvals and is actually in use, there could be reciprocity so that other states could use similar technologies in their own sandboxes.

That wouldn't mean Utah is setting the expectations for other states, but I believe there are conversations happening about that possibility.

So this could have implications for health systems and patients across the country, not just in Utah?

Hill: Yeah, absolutely.

What exactly will CHAI be responsible for evaluating, and where does its role end and the state's regulatory authority begin?

Hill: First and foremost, there are still a number of details that need to be worked out. We're going to spend the next 90 days with the state of Utah working through those details, so we don't have all the answers.

What I can say is that CHAI is a convener of our partner program. We're going to use that existing program and the members that have the capabilities the state needs to perform evaluations.

CHAI's role will mostly be convening organizations with the requisite capabilities to perform those evaluations on behalf of the state. Then it's really up to Utah to make sure those evaluations meet the state's requirements.

By no means will a CHAI-verified or CHAI-certified designation be the final decision point.

The state of Utah is responsible for protecting its constituents. We're there to help provide the infrastructure and be a natural extension of the state, performing evaluations that the state doesn't have the resources or expertise to conduct itself.

Brian, CHAI has been working on independent assurance for AI for years, including its AI "nutrition labels." What does partnering with a state government make possible that voluntary industry efforts alone haven't?

Anderson: The AI sandbox allows a pathway for health AI technology companies to deploy their models across Utah in a safe manner that involves post-deployment monitoring and governance.

One goal is to generate the evidence we need to learn. This is a new space. We're still learning, and as the space evolves rapidly, we need foundational knowledge to make the most informed decisions.

It also allows innovators to get the regulatory relief they need to innovate rapidly and partner with health systems and patients in a way that builds trust and transparency.

I think this is the first time a state has created a sandbox with the infrastructure and regulatory process that allows for premarket testing and post-deployment monitoring of models. We haven't seen that happen in another state.

That's wholly unique and very different from CHAI doing that on its own. There's obviously a regulatory component to it, too.

Hill: The other thing I would add is that this allows us to put our frameworks, best practices and everything we've been building over the last several years into a place where we can learn together with companies, other stakeholders and state entities.

We can take the evidence we've been building in theory and actually test it in an approved way at the state level.

We're not only putting frameworks out there. We're also benefiting from the learnings coming in.

What we're hoping to do is provide frameworks and partners to help the state meet its legislative mandate, but also bring those learnings back into CHAI and the community so we can spread them across organizations and stakeholder groups throughout the nation.

Intermountain Health said during Monday’s press conference that it has more than 300 AI initiatives underway. How do you determine which applications warrant this new level of scrutiny, particularly when health systems already have internal governance processes?

Hill: I think it's up to the state to decide which vendors qualify for the sandbox.

I would frame the qualifying use cases as those where an organization wants to use technology to perform a service on a real patient that would normally require a licensed human clinician.

Prescription refills are probably the best example we have at this point.

If an organization wants to replicate a process like prescription renewals that would normally require a license, it would need to work with the state through the sandbox, get that use case approved and then deploy it.

There are probably an infinite number of potential use cases, depending on the organization, its risk tolerance and its maturity.

But if a health system is building a tool it wants to test within the sandbox, it will have to meet the same rigor as any other company.

What I thought was interesting from the press conference was that both University of Utah Health and Intermountain said they were going to put patient safety first.

What I heard from them is that they're going to be very selective about the use cases they pursue and make sure they're safe for patients.

They're not trying to say, "Let's open this up for everyone to come and try all these new technologies, and we'll see what happens."

They're still very much focused on putting patients first.

Intermountain and University of Utah Health will have umbrella agreements intended to expedite reviews of future AI pilots. How can independent evaluations remain rigorous without becoming another bottleneck for hospitals trying to deploy new technology?

Hill: We shouldn't be thinking about this as a bottleneck.

We're testing frontier use cases here, things humans would normally perform and that have never been automated or replicated by technology.

I don't see this as a bottleneck because, right now, none of these use cases would be legal to pursue in the state without this waiver.

I see this as the first step in a wave of innovation that can happen at the state level.

States can say, "There are an innumerable number of use cases that we could be trialing here. There's an innumerable number of potential benefits of the technology, but we're not going to roll these out all at once. We're going to test them one by one to make sure they're safe and effective."

Then, if the state determines that the program is going well and a given AI system has demonstrated that it's competent to perform under a waiver, maybe it can consider further legislation to allow that use.

By no means are they saying they're just going to open everything up.

I wouldn't look at this as stifling innovation. I would see it as the first opportunity we've seen for a state to say, "We're going to test frontier use cases that normally wouldn't be possible, and we're going to do it in a safe space."

Utah says data from these pilots will inform future permanent health care regulations. What questions do you hope the state can answer so other regulators don't have to reinvent the framework?

Anderson: I think some of the most urgent questions center on whether these models can be trusted.

When you have semi-autonomous or autonomous agentic tools operating in health care, trust is at a premium.

Generating evidence about a model's performance and answering basic questions about how it performs in specific clinical scenarios, in a prospective manner, is hugely valuable.

If another state can use that evidence, it can make more informed decisions about a vendor that might have generated promising evidence in Utah, without necessarily having to go through the same rigor and monitoring again.

That could be enormously valuable, particularly if states pursue reciprocal recognition.

Another important question is: What are the most effective ways to monitor agents?

We've seen a lot of questions about how to do that safely and effectively. We'll be approaching this with significant rigor to generate the transparency and insights that patients demand.

Hill: My hope is that one of the most basic questions answered during this pilot is: What does it take for clinicians and patients to trust AI in their care?

We're seeing that trust is one of the biggest inhibitors to AI being deployed at scale across organizations. My hope is that Utah can answer that question.

The other question I really hope they answer is what a liability structure looks like between all these different parties.

That's one of the top things on a lot of organizations' minds right now.

We could be deploying AI at scale for just about anything, but contractually, operationally and even hypothetically, we still don't fully understand where liability is going to sit.

That often prevents organizations from deploying at the scale they want because there's not a lot of predictability.

My sincere hope is that we get a better sense of what that structure might look like, so organizations can be insured, patients can have relief if something goes wrong and all the requisite parties can be held accountable in a way that's equitable and fair.

Do you have a sense yet of how patient trust will be measured once tools are cleared and vetted through the pilot process?

Hill: I know there's a lot of thought around it, but whether there's a formal mechanism or not, I personally don't have that information.

Anderson: I hope we capture sentiment and feedback from patients who are using these tools.

Capturing patient feedback on their experience, informed consent and disclosure will be really important.

However AI touches the patient, the patient's sentiment associated with that is going to be important to capture and share.

My hope is that this sandbox approach captures that kind of information.

Anything else that folks should know?

Anderson: I think this is a very exciting and watershed moment within the nation.

We have a challenging political landscape at the federal level, and we have states beginning to move forward with what I see as rigorous yet appropriate regulatory frameworks to ensure we're generating the evidence we need in health AI.


C-Suite Shuffles

Where health care leaders are coming and going


This is a preview of the October 8 edition of Access Health—Tap here to get this newsletter delivered straight to your inbox every Thursday.

View 0 Comments

Request Reprint & Licensing

Submit Correction

View Editorial & AI Guidelines