Healthcare IT professionals have long faced data challenges – from sorting out storage issues during the Big Data age to complying with HL7 standards to adapting infrastructure amidst a rapid shift to telehealth. While these advancements have all improved information sharing across the industry and ultimately the patient experience, it’s also created mountains of data. In fact, RBC Capital reports that the healthcare industry generates 30% of the world’s data with that number expected to grow to 36% by 2025.
This deluge of data has only created new challenges for IT professionals, much of which has been discussed here on HIT Consultant. While most focus is spent on data interoperability or rising storage costs, what few discuss is the invaluable insights buried within the mountains of data being created every day. This dark data holds critical business intelligence but is hidden within redundant, obsolete, or trivial data, as well as being siloed within different departments. As a result, its largely become inaccessible.
Uncovering this data may seem like a fool’s errand. And with traditional data management practices that may have been the case. But new technologies such as data fabrics and deep learning are making this process much easier, presenting new opportunities for both the IT team and the business as a whole.
Uncovering the Dark Data in Healthcare Opportunity
One of the biggest challenges associated with dark data is that the vast majority of it is unstructured. These PDFs, diagnostic images, clinical trial documents, case notes, etc. hold a team’s most valuable insights but are impossible to easily identify and glean information from. This data is also full of nuance. For example, different names for similar treatment side effects, abbreviations for diseases, sentence structures, or handwritten notes all of which may have different values based on the project at hand.
These factors make traditional search methods ineffective. Assigning a researcher to take on this task manually will certainly come with delayed project timelines and human error which is bound to happen when scanning through hundreds of documents to pull out specific information.
The result? Time wasted missed insights and limited ROI.
Advancements in deep learning algorithms are proving to be a solution to this issue. The natural language processing made possible by these algorithms enables valuable text and insights to be identified at a faster rate and accuracy than traditional search methods.
Now the researcher or healthcare IT professional tasked with a weeks-long research project can complete their task in a matter of hours or days.
Making The Data Work For You
Once dark data is unlocked, healthcare IT professionals can provide significant benefits to the organization, especially those in healthcare where data-backed decision making is critical to a variety of factors from product safety to public health to the patient experience. While the opportunities for leveraging dark data with the help of deep learning are ultimately endless, some near-term applications include:
– Clinical Trial Design: Determining the viability of a clinical trial is critical especially as the cost of trials continues to skyrocket. By leveraging previously dark data, companies can conduct more accurate risk assessments by analyzing insights that are key to decision making such as disease indications, clinical trial protocols, site details and subject demographics.
– Medical Imaging Detection: Images are one of the biggest causes of dark data given the challenges most computer applications have with reading this type of content. However, as case numbers around diseases such as cancer continue to rise, greater accessibility to screening images can help oncologists accelerate the identification of concerning tissue and improve the accuracy of initial diagnosis.
– Patient Care: When diagnosing or determining a path of care, medical professionals typically rely on the data most readily accessible to them. By unlocking dark data, these individuals can have access to rich insights that lie within complex texts and documents to identify alternative treatments, potential side effects and more that can greatly enhance the patient experience.
What The Future Holds
The healthcare industry is experiencing a data renaissance. Between enhancements to EHRs, new data interoperability protocols and accelerated digital transformation, the industry’s information sharing is enhancing tremendously. We’ve been experiencing this recently with the COVID-19 pandemic. Real-time data sharing and access to novel insights has helped influence public health guidelines, identify appropriate treatment types and accelerate the development of various vaccines.
As the industry’s IT practices become more advanced, we can ensure a future of enhanced response to emerging disease, improved patient care and a healthier population. And uncovering dark data will play a key role in making this future a reality.
About Dr. Christopher Bouton
Dr. Christopher Bouton is the founder & CEO of Vyasa, a provider of deep learning AI analytics software. Prior to Vyasa, Bouton founded big data analytics company Entagen which was acquired by Thomson Reuters and was head of integrative data mining at Pfizer. Dr. Bouton holds a Ph.D. in molecular neurobiology from Johns Hopkins University.
