The AMA says physician oversight can’t substitute for adequate evidence and system design. That puts the burden on whoever holds the usage records.
In many hospitals, an AI tool is reading patient records through a shared service account that no one can tie to a person. That gap matters more now. The American Medical Association’s new framework, released October 29, 2026, divides responsibility among developers, physicians, and the health systems that deploy AI tools. Of the three parties, only one controls who and what can reach patient data.
That party is the health system. The framework says healthcare organizations should be accountable for how AI systems are selected, implemented, governed, and monitored in real-world settings. Physicians, in the same document, answer for the clinical judgment they can meaningfully exercise, and the document adds that physician oversight cannot substitute for adequate evidence and system design.
Read those two positions together. If a clinician cannot be expected to catch what was invisible at the bedside, the question of what could have been seen moves upstream, to whoever deployed the tool and whoever holds the records of how it was used. The framework is a policy position, not a statute. But it tells a CISO or a privacy officer which question is coming and who will be asked.
Where the Record Goes Missing
Consider a familiar sequence. AI tools arrive through the side door. A documentation assistant gets connected to the electronic health record by one department. A summarization plug-in is trialed by another. Each works through a shared service account because that is the fastest way to make the integration run, and each is approved in a meeting that produces a policy document and little else.
The result is a record that exists in fragments. The identity log shows an account. The application log shows a request. The vendor holds its own telemetry. Nobody holds a single, attributable account of which person, or which agent acting for a person, read which patient records under which policy.
A 2026 annual survey from our company puts a number on the fragmentation. Of the 459 security, compliance, and technology professionals surveyed in the second quarter of 2026, 50 percent said they could not produce a complete AI data access audit record within one business day, and only 17 percent could do it within an hour. The figures are self-reported and span all industries, so they are not a healthcare measurement, but they describe the same fragmentation hospitals live with.
Executives sense it. Grant Thornton’s 2026 AI Impact Survey found that 78 percent of nearly 1,000 senior business leaders lack full confidence that their organization could pass an independent AI governance audit within 90 days. Grant Thornton sells advisory services in this space, so read it as a measure of confidence, not a failure rate. Taken with the survey above, it points to the same weakness. Organizations believe they govern AI and cannot show it quickly.
Why the Current Approach Fails
Most health systems answer the liability question with paperwork. An AI use policy, a vendor questionnaire, a risk assessment completed at procurement. These are necessary and they are not evidence. A policy says what should happen. A log shows what did.
Vendor assurances fail for a different reason. A developer can document intended use, limitations, and performance, and developers should carry that burden. A developer cannot produce your access record. Whatever a manufacturer monitors about its model, it does not see which of your staff or workflows reached which patient files through your integration. That evidence lives on your side of the line.
Shared accounts fail twice. HIPAA’s technical safeguards require unique user identification and audit controls, and the Security Rule contains no AI exemption. A log entry that says “service account” does not identify the person, the workflow, or the purpose. The regulators regulate the data, not the model, and they will not accept “the assistant did it” as an answer to “who accessed this record.”
What Needs to Change
Start with accountability. Every AI tool and agent that can reach protected health information needs a named human owner and a human authorizer behind each request. Ownership of AI risk is unsettled in many organizations, and an unowned agent is an unanswerable question.
Then put people and agents under one governance model. An AI agent is a second kind of identity acting in your environment, and it earns the same scrutiny as the employee beside it, with one identity model, one policy set that expresses who may see which data under which conditions, and one audit record. Two separate regimes, one for humans and another for agents, guarantee that the evidence will live in two places and will not reconcile when someone needs it.
Make the evidence a byproduct of how access works. A record assembled after an inquiry begins is a reconstruction, and reconstructions are argued over. A record written at the moment of access, protected against tampering, and available to the compliance team on demand is evidence. Test it the way an investigator would. Pick one patient record, one date, and one AI tool, and time how long it takes to produce a complete account of AI access. If the answer is longer than a business day, that is your finding, and you want it before a lawyer does.
Finally, change what you ask vendors. Request documentation of intended use, known limitations, and performance, which is what developers should be able to supply. Then ask what their product logs when it is connected to your patient data, and whether each action can be tied to a person. The answers will tell you where your own side of the evidence must come from.
Before Anyone Asks
Courts and regulators will take years to settle who pays when clinical AI fails. Whatever they decide, someone will ask which person, or which agent acting for a person, opened which record and why. A health system should know its answer first.
About Frank Balonis
Frank Balonis is the Chief Information Security Officer at Kiteworks, where he oversees corporate security and compliance, technical support, and customer success.
