AI is being rapidly integrated into healthcare, and its role is expanding quickly. Yet, despite this rapid adoption, AI trust among patients remains critically low. That is a problem because trust is fundamental to healthcare. Patients who trust their healthcare providers are more likely to share information, follow recommendations, and actively engage with their care. In my current professional role, I spend a considerable amount of time thinking about the reasons why patients should trust AI-driven healthcare technology.
There are several possible reasons driving AI distrust. One widely cited barrier to trust is explainability: How does AI reach its conclusions? If a patient cannot understand how and why an AI system arrived at an answer, many people argue patients will likely have skepticism about trusting it.
But, does understanding how something works really determine whether we trust it? Patients (and, if I am being honest, many doctors too) do not understand how an MRI machine works. They do not understand the physics behind magnetic resonance or how the machine transforms signals into an image. Yet, we use MRIs every day because we have learned they are reliable and produce valuable results. Our trust does not come from understanding the technology. It comes from years of experience, validation, and consistent performance.
AI may have to earn trust in the same way. Patients are likely to care less about the technical architecture of an AI model than whether it performs reliably and produces meaningful results. If an AI system consistently does what it is supposed to do, and if those results are independently validated, understanding exactly how it reached its conclusion may become less important for many patients and their loved ones. In this sense, trust may be longitudinal, developed through repeated, positive experiences. When we first heard about self-driving cars, many of us — including me — could not imagine getting into one. Today, companies, such as Waymo, are operating autonomous vehicles in real-world environments, and our perception of them is beginning to change. At some point, people may trust an autonomous system more than a human driver. That trust will come not from understanding the algorithms that help these vehicles run, but rather from seeing consistent performance over time.
However, experience alone does not ensure trust. It also hinges on what happens to the data AI relies on. Healthcare has a long history of collecting enormous amounts of personal information, and patients often have little visibility into what happens to that information once it enters a provider or payer system. This opacity may be particularly concerning for communities that have historically been underrepresented in medical research or have experienced healthcare discrimination. Patients may be more willing to trust AI if they can trace what happens to their information, understand who can use it, and have some degree of control over its use. Making health data more patient-centric, rather than something that effectively belongs to the hospital, insurer, or physician practice, could go a long way toward establishing trust.
Keeping a human in the decision loop is another potential solution. But, our thinking around this needs to be more nuanced too. We often assume that having a doctor review an AI decision will automatically make patients more comfortable. However, in communities where healthcare trust is already low, the presence of another healthcare provider may not be enough. Perhaps the human in the loop should not always be a doctor. In low trust communities, an advocate or independent reviewer, solely focused on representing the patient’s best interests, may ultimately be more effective at fostering trust. The advocate’s role is not simply — or even primarily — to explain AI to the patient, but to give the patient confidence that there is someone questioning the AI and its results.
Finally, institutional validation can play an important role in building trust. Organizations, such as the FDA and major academic medical centers, still carry credibility, even if institutional faith has eroded some in recent years. Knowing that an AI system has undergone rigorous regulatory review or independent validation may give patients an additional reason to trust it and accept diagnoses and treatment recommendations. Institutional reputation alone though is not enough. Patients should be able to see the evidence: How the system was tested, how well it performs, whether it works for people like them, and who is accountable when it experiences failures.
Perhaps, we are asking the wrong question around institutional trust. The goal should not be to make patients understand AI well enough to trust it. Instead, the aim should be to make AI worthy of trust. Patients do not need to understand every algorithm any more than they need to understand the physics of an MRI. Fundamentally, they need to know that the technology has been tested, that it works, that their information is protected, that someone is accountable, and that they have a voice when something goes wrong. This should be the focus of healthcare organizations and anyone working to develop AI-driven healthcare technology.
