Health AI is only as good as the data it’s built on. And when it comes to sleep, much of that data is fundamentally flawed.
As health and wellness apps race to integrate sleep tracking and AI-driven coaching, they’re building on a signal that is inconsistent, incomplete, and often misunderstood. That matters because sleep is a driver of nearly every major health outcome, from mental health and metabolic function to cardiovascular risk and chronic disease.
This isn’t abstract for me. My father survived a terminal cancer diagnosis for 14 years by prioritizing rest. Sleep is not just a pillar of health; it’s foundational. When something that essential is measured incorrectly or interpreted without sufficient rigor, the consequences are real.
Yet many apps rely on wearable data and large language models to deliver guidance that appears personalized, but often lacks the science behind it to make it reliable. This will become a bigger issue as regulators begin to take a closer look at the safety and effectiveness of these technologies.
Wearables Have an Accuracy Problem
Sleep data is hard. Not “we need better prompts” hard. Scientifically hard.
I’ve had a front-row seat to the health and wellness space for many years, learning an incredibly important lesson: sleep data needs to be personalized to be effective.
Over the past decade, we’ve analyzed nearly a quarter trillion sleep data points across roughly 1 billion hours of sleep, collected from 500+ wearables, apps, and devices to create a SleepScore™ for every person’s night of sleep.
What that scale reveals is massive fragmentation, significant biases with results varying widely not just across devices, but within the same device over time.
Put simply: If you slept with five different wearables tonight, you’d wake up with five different answers.
Published performance evaluations show that some rings can overestimate total sleep time by nearly 1 hour per night and some general wearables may underestimate overnight wakefulness and deep sleep by nearly 1 hour.
These discrepancies fundamentally change how sleep quality is interpreted.
And even this assumes consistent usage. In reality, nearly 85% of wearables users don’t wear their devices every night, creating gaps in the data that make longitudinal insights even less reliable.
AI Coaching Without Reliable Data Creates False Confidence
Large language models offer health and wellness apps a shortcut to coaching, which can lead to unreliable and sometimes misleading guidance. We have all read the horror-stories where LLM’s coached people to self harm. Think about sleep – it is a process that we are not conscious for, so we are even more likely to trust what we are told.
Sleep is one of the most foundational yet inconsistently measured signals in human health. If AI struggles to interpret something this universal, it raises broader questions about how well it can handle more complex or less visible conditions.
One reason why LLMs struggle with sleep is that it is inherently personal.
Should someone take melatonin? Well, that depends on several factors that need to be taken into consideration. When should they go to bed and wake up? Every person has different individual factors like chronotype, lifestyle, and underlying health conditions, to influence that decision. These decisions cannot be answered accurately without high-quality, individualized data.
In some cases, the models are only loosely connected to the user’s actual data. A user might ask why they keep waking up in the middle of the night and receive generic advice, such as going to bed at the same time or limiting screen exposure, rather than an analysis grounded in months of their own patterns.
This gap between perceived intelligence and actual insight isn’t unique to sleep and is emerging across health AI more broadly. Systems generate confident answers, but often lack the validated, longitudinal data required to make those answers truly meaningful.
Even when models attempt to incorporate context, they introduce new risks. Prior interactions can shape future responses in ways users don’t see, leading to subtle but important misinterpretations over time.
Some platforms attempt to mitigate this by restricting what their AI can say.
No user who is struggling with sleep wants to read “I can’t advise you on that,” when asking for help. Apps would be advised to think carefully about their sleep strategy, as users don’t want to feel misled about the promise of “AI coaching” that might not actually work.
And in both cases, the root issue is the same: without accurate, comprehensive data, no amount of algorithmic sophistication can produce meaningful outcomes.
Bad data produces bad advice. Normal sleepers get labeled insomniac. Poor sleepers get told they’re doing fine.
Sleep makes this failure visible, but the implication is much broader: if we don’t address the gap between data quality and AI output, we risk scaling systems that feel intelligent, but don’t actually improve health outcomes.
What Scientific Rigor in Health AI Actually Looks Like
If sleep exposes the limitations of today’s health AI systems, it also clarifies what better looks like.
High-quality, validated measurement is non-negotiable. Without it, everything that follows from analysis and coaching to intervention is built on unstable ground.
But measurement alone isn’t enough. Meaningful health AI requires longitudinal data that captures patterns over time and systems that can reconcile inputs from multiple sources, account for variability, and translate raw signals into context-aware insights.
Most importantly, it requires a structured approach to behavior change. Effective systems follow a disciplined cycle: identify the issue, attribute a likely cause, suggest an intervention, monitor the outcome, and adapt over time. Without that feedback loop, “coaching” becomes little more than educated guesswork.
These capabilities require sustained investment in research, clinical validation, and data infrastructure, far beyond what most consumer grade approaches are designed to support.
Which leads to a strategic reality for the industry: not every company should build this themselves.
Health and wellness platforms increasingly face a choice: either invest deeply in the scientific and technical infrastructure required to deliver reliable health insights, or partner with specialists who have already built it.
The companies that get this right will be the ones that understand where rigor matters most and are disciplined about how they deliver it.
Sleep makes this tradeoff visible, but the implication is broader: as health AI scales, the gap between perceived intelligence and actual scientific value will define which products earn trust and which quietly lose it.
About Colin Lawlor
Colin Lawlor is a globally recognized leader in sleep science and digital health, with over 30 years of executive experience, 15 of them dedicated to redefining how we measure, understand, and improve sleep. As founder and CEO of Sleep.ai and former executive at ResMed, Colin has pioneered some of the most scientifically validated sleep technologies in both consumer and clinical markets, including the first contactless, sonar-based sleep tracking systems.
Under his leadership, Sleep.ai has transformed over 800 million hours of consumer sleep data into AI-driven health insights and coaching that help partners across health, wellness, and technology deliver personalized, science-backed sleep solutions. His team’s innovations have powered more than 100 scientific publications and led to the first-ever government-reimbursed digital sleep intervention in Germany.
A passionate advocate for using sleep to prevent chronic conditions like heart disease, dementia, and diabetes, Colin is known for connecting the dots between science, technology, and real health impact. Originally from Ireland and a graduate of Trinity College Dublin, Colin has a lifelong curiosity that’s taken him from playing the bagpipes to learning (and occasionally mangling) German. He firmly believes that a pint of Guinness may be “good for you,” but having one too many will definitely impact your sleep.
