• Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to secondary sidebar
  • Skip to footer

How eClinicalWorks Agentic AI Is Helping Revenue Cycle Leaders Improve Financial Performance
  • Opinion
  • Health IT
    • Behavioral Health
    • Care Coordination
    • EMR/EHR
    • Interoperability
    • Patient Engagement
    • Population Health Management
    • Revenue Cycle Management
    • Social Determinants of Health
  • Digital Health
    • AI
    • Blockchain
    • Precision Medicine
    • Telehealth
    • Wearables
  • Life Sciences
  • Investments
  • M&A
  • Value-based Care
    • Accountable Care (ACOs)
    • Medicare Advantage

Four Questions to Ask When Evaluating Healthcare AI Vendors: Balancing Native EHR Infrastructure with Specialized Decision Intelligence

by Mallika Edwards, Chief Product Officer, Xsolis 11/18/2026 Leave a Comment

  • LinkedIn
  • Twitter
  • Facebook
  • Email
  • Print
Four Questions to Ask When Evaluating Healthcare AI Vendors
Mallika Edwards, Chief Product Officer, Xsolis

Four Questions to Ask When Evaluating Healthcare AI Vendors

By Mallika Edwards, Chief Product Officer, Xsolis

Interoperability used to live in the IT requirements document. It was something to confirm during procurement and forget once the contract was signed. In my conversations with health system data and platform leaders, that is no longer how they operate.

Recently, I sat down with a group of these leaders to talk candidly about how they evaluate AI vendors. Interoperability is no longer a background requirement. It has become a lens for judging whether a vendor understands how a health system actually runs, often before the vendor has a chance to prove anything.

The “wait for the EHR” default

Many health systems carry a reasonable-sounding assumption: the EHR vendor already in the building will eventually build whatever specialized capability comes next. One system, one relationship, one fewer integration to manage. The numbers reflect it: 79% of U.S. hospitals use AI from their EHR vendor, versus 59% using third-party tools, and a recent JAMA Viewpoint attributes much of that gap to EHR vendors’ advantages in infrastructure, procurement, and pricing. For workflows that live entirely inside the EHR, that is often the right call. Health system informatics leaders have said as much, while also seeing a “massive opportunity” for third parties in workflows that extend across systems.

That is where I’d push back. A system built to be the record for everything is not the same as intelligence built to understand one exceptionally complex judgment. Utilization management, medical necessity determination, and concurrent review aren’t generic problems that improve as a platform adds surface area. They are one specific decision, made under time pressure, spanning clinical documentation, payer rules, and financial workflows, with real consequences for patients and for the organization’s finances. 

Waiting for an EHR vendor to develop its own native solution to each new workflow is also not a neutral choice. It’s a bet, and the cost rarely arrives as a line item. It accumulates in denials, in length of stay, and in staff hours spent on manual review. It compounds quietly until someone names it.

In one 2026 survey, 69% of senior health system technology leaders cited vendor management and integration as a top obstacle to executing AI. So before you wait for the EHR, ask every AI vendor the same four questions.

1. Does it work inside the chart or beside it?

The concerns I heard were operational, not technical. If moving from a patient’s chart into a specialized platform means a separate login, a new browser tab, and a mental context switch, that friction repeats across every case a clinician touches in a day. Access inside the chart, where a user moves directly from the EMR into the corresponding record without leaving their workflow, removes it. This last mile is still a work in progress for most vendors. Integration partners that work alongside EMRs, or technology like SMART on FHIR®, can help close the gap between where a vendor’s capabilities end and what end users expect. 

2. Does completed work flow back automatically?

When work finished in one system has to be re-entered in another, the cost isn’t just time. It’s a data integrity risk. Structured data that moves automatically between systems means work isn’t redone, and it can power decisions and reporting well beyond the task it was created for. The payoff reaches past IT: staff get time back, and leadership gets a trustworthy picture that spans systems instead of living in fragments. The burden is widespread and affects all care team members. 

3. What specific decision has this AI been refined on, and for how long?

Clean integration tells you a platform respects your workflow and understands that the intelligence behind the decisions may not exist in the workflow system. Every vendor will say its AI is specialized, so the claim itself is worthless. Look for focus over breadth: years of iteration on one decision, not a decision that is one of many a platform touches. Then look at how much time the model has spent being trained in real conditions, including payer changes, volume surges, and policy shifts, which show how a model behaves when conditions aren’t ideal.

4. Has it been tested against real outcomes, by organizations other than yours?

Peer validation matters more than a vendor’s own description. Health systems already act on this: nearly all those surveyed test third-party AI before deploying it, yet fewer than half have a dedicated environment for doing so. That makes scrutiny by other organizations more valuable, not less. Ask whether the AI has been measured against results that matter, like denials, length of stay, and reviewer workload, rather than internal accuracy metrics alone, which governance frameworks too often favor over downstream outcomes. Ask whether it has passed repeated governance review at health systems with different populations, workflows, and risk tolerances. Each review is an independent test the vendor can’t control. If the only evidence is a demo and a roadmap slide, you have your answer. 

Ask all four questions

The first two questions get a platform into the room. They are necessary, but not sufficient. The second two determine whether the intelligence has earned the right to make the decision. Experience on a single decision accumulates slowly, and it can’t be shipped as a feature release, which is exactly why it deserves weight in an evaluation.

The EHR is the system of record. The intelligence working inside it should be judged on what it has learned, not just on how quietly it fits. If you’re evaluating AI right now, ask all four questions, including the ones that are harder to answer in a demo.


About the Author:

Mallika Edwards is Chief Product Officer at Xsolis, where she leads strategy and roadmap for the company’s solutions portfolio, including the Dragonfly AI platform. She has 30 years of product management experience in healthcare IT, including senior roles at Cerner Corporation, where she worked on interoperability strategy, and holds a B.S. in Computer Science from the University of Kansas.

  • LinkedIn
  • Twitter
  • Facebook
  • Email
  • Print

Tap Native

Get in-depth healthcare technology analysis and commentary delivered straight to your email weekly

Reader Interactions

Primary Sidebar

Subscribe to HIT Consultant

Latest insightful articles delivered straight to your inbox weekly.

Submit a Tip or Pitch

Featured Insights

How eClinicalWorks Agentic AI Is Helping Revenue Cycle Leaders Improve Financial Performance

Most-Read

Innovaccer-CHIME 2030 Survey Reveals AI Will Automate Nearly Two-Thirds of Administrative Work and Do 3x the Work With the Same Staff

Innovaccer-CHIME 2030 Survey Reveals AI Will Automate Nearly Two-Thirds of Administrative Work and Do 3x the Work With the Same Staff

CB Insights’ 2026 Digital Health 50 Reveals How Agentic AI Is Becoming the Clinical Provider and Medication Access Engine

Oura Acquires Counsel Health for $625M: Scaling Continuous Biometrics into On-Demand Medical Care

M&A: Oura Acquires Counsel Health for $625M: Scaling Continuous Biometrics into On-Demand Medical Care

WittKieffer 2026 Healthcare CIO Survey: 82% Sit on Executive Teams, Yet Only 36% Have Formal Succession Plans

WittKieffer 2026 Healthcare CIO Survey: 82% Sit on Executive Teams, Yet Only 36% Have Formal Succession Plans

CVS Rebrands Health Services Segment, Launches Model for Retail Pharmacy Reimbursement

CVS Health Formalizes Tax-Free Split-Off to Carve Out CVS Pharmacy by 2028

SAMHSA and ONC Invest $20M in Behavioral Health IT Initiative

ONC Finalizes HTI-5 Deregulatory Rule to Cut $1.55B in Health IT Compliance and Accelerate FHIR APIs

Redesign Health Merges with Sheba Medical Center’s ARC Global and Agrees to Acquire AVIA

Redesign Health Merges with Sheba Medical Center’s ARC Global and Agrees to Acquire AVIA

Sword Acquires Headspace in All-Cash Deal Worth Up to $300M to Unify Physical and Mental AI Care

M&A: Sword Acquires Headspace in All-Cash Deal Worth Up to $300M to Unify Physical and Mental AI Care

M&A: Francisco Partners to Acquire Weave in $650M Take-Private Deal

M&A: Francisco Partners to Acquire Weave in $650M Take-Private Deal

Epic Launches One-Click Care Everywhere Diagnostic Image Exchange

Epic Launches One-Click Care Everywhere Diagnostic Image Exchange

Secondary Sidebar

Footer

Company

  • About Us
  • 2026 Editorial Calendar
  • Advertise with Us
  • Reprints and Permissions
  • Op-Ed Submission Guidelines
  • Contact
  • Subscribe

Editorial Coverage

  • Opinion
  • Health IT
    • Care Coordination
    • EMR/EHR
    • Interoperability
    • Population Health Management
    • Revenue Cycle Management
  • Digital Health
    • Artificial Intelligence
    • Blockchain Tech
    • Precision Medicine
    • Telehealth
    • Wearables
  • Startups
  • Value-Based Care
    • Accountable Care
    • Medicare Advantage

Connect

Subscribe to HIT Consultant Media

Latest insightful articles delivered straight to your inbox weekly

Copyright © 2026. HIT Consultant Media. All Rights Reserved. Privacy Policy |