For decades, one of the most daunting tasks in healthcare has been managing and making sense of the vast amounts of data generated annually. From patient and outcomes data to billing and scheduling data, the list goes on and on.
The predominant challenge lies not just in quantity but like this data: much if not most of it is unstructured, i.e., data that isn’t predefined and searchable on a table, such as text messages, images, videos, audio files, and emails.
Historically, healthcare organizations have tackled this issue with a kind of brute-force approach, involving considerable human effort and cumbersome processes. Teams of data analysts have been assigned the tedious task of data processing, often using tools no more sophisticated than Excel spreadsheets. These spreadsheets then serve as a basis for programmers, either contracted from outside the organization or hired internally, to create automated systems.
Anyone who has worked within healthcare organizations is all too familiar with this approach. And we all know it is fraught with inefficiencies.
I see this up close in my work as I strive to analyze productivity across various medical specialties. This analysis is crucial for enhancing physician performance and guiding key operational decisions such as hiring, staffing, and scheduling.
This is no simple task. The complexity arises from the need to amalgamate information from disparate sources. Data must be compiled from billing records (exported as CSV files), patient data (sourced from both the organization’s and hospitals’ EMR systems), and scheduling software (to track working hours and durations). Additionally, there’s a need to factor in patient outcomes, like hospital stay durations, to gauge the effectiveness of treatments.
This intricate process involves around 30 steps, some of which are conducted in Excel while others rely on automation tools. Once the analysis is complete, the decision to automate it further usually involves either programming a solution, which then needs to be maintained and continually updated.
We could bring in external programmers or leverage a third-party tool, but this can introduce friction and disconnect between the data and the insights users need. Or, we can hire internal programmers who will be more integrated, but who will add to the organization’s overhead and possibly even entrench cumbersome maintenance tasks.
The healthcare industry as a whole has attempted to respond to these kinds of challenges with the development of interoperability standards like Fast Healthcare Interoperability Resources (FHIR). This specification aims to streamline how healthcare data is structured, facilitating better data sharing and insights. It is a big step forward from the previous data standards. Yet despite these efforts, interoperability standards have struggled to fully deliver on their promise of seamless integration across various business silos.
Enter AI. Recent hype notwithstanding, the use of machine learning tools to make sense of huge data sets is not exactly a new idea for healthcare organizations. For years, Natural Language Processing (NLP) and similar technologies have been familiar fixtures within hospitals and provider organizations.
And yet, with the recent explosion in tools like ChatGPT and other sophisticated AI solutions, the applications for healthcare are becoming much more readily apparent. I believe AI has the potential to solve the decades-old problem described above.
The great potential of using AI tools for complex data analysis is to be able to dump unstructured data from multiple sources into a single environment, and then query the AI for direct answers to our questions.
Nikhil Krishnan, who writes the Out-Of-Pocket newsletter, recently sent out a meme that describes this well:
The unstructured data that is supposed to be going into the FHIR pipe, is instead pouring out into the air unconstrained—but that’s ok because AI tools can take that data and transform it into something usable. Essentially, custom-built AI tools may be able to bypass the complex and often inefficient traditional processes, including data standards like FHIR, that have long been a major bottleneck to uncovering business insights.
Instead of the “data wrangling” that so many of us have grown accustomed to, we can build a predictive engine that allows us to dump whatever unstructured data we have into it, and then start asking questions like we do with ChatGPT or similar tools.
This will be transformative. Data analysis teams used to processing requests from throughout the organization in weeks or months will now be able to do so in days. Questions left to the wayside, because they were deemed low priority, may finally be answered. Meanwhile, we’ll be able to explore the nuances of business-critical questions in far greater detail, faster, and with less friction than ever before.
This is the promise: that hospitals and healthcare organizations stand on the brink of a period of unprecedented clarity in their business processes, and that the once daunting task of structuring and analyzing massive datasets can be revolutionized. The advancement could lead to more informed decision-making, better resource allocation, and ultimately, enhanced patient care.
About Dr. Hallock
Dr. Hallock leads clinical innovation at Access TeleCare, bringing together clinical quality and data analytics to demonstrate superior outcomes for patients and hospital partners. A strategic leader with more than twenty years of experience within some of the nation’s most highly-developed, clinically-integrated networks, he is adept at aligning clinical and administrative objectives to produce optimal quality, safety, efficiency, and revenue results.
