In 2016, I asked the head of cardiac surgery at a large hospital what it would take to clinically validate the work we were doing with wearable data at Welltory.
As I remember it, his answer was roughly, “Jane, don’t get into clinical validation. It’s hell, and it has very little to do with real life.”
Then he brought up blood pressure. Think about how much work it took, he said, before a doctor could look at a number from a cuff and know what to do with it, and still it’s incredibly variable.
Two years later, we were talking to a Silicon Valley venture fund. They brought in physicians to evaluate our work on heart rate variability, or HRV. This was before people compared HRV scores over breakfast. The physicians thought HRV from wearables would never gain widespread acceptance. Doctors, they told us, wouldn’t use it.
At the time, I thought the problem was education. HRV is a measure of how much the time between heartbeats varies, and doctors have been studying it for decades. In one study from 1987, researchers analyzed 24-hour ECGs from 808 people who had survived a heart attack. The group with HRV below 50 milliseconds had 5.3 times the mortality risk of the group above 100. Later studies found associations with cardiovascular risk in people without known heart disease, too. Those are findings about specific measurements in specific populations. If your watch says 42 milliseconds this morning, the first thing I’d want to know is what it usually says. For you, 42 might be an ordinary Tuesday. The 1987 post-heart-attack study · The 2013 meta-analysis
Eight years after that first conversation, I don’t think it’s about education anymore.
A woman brings a graph to her appointment
Imagine a 58-year-old woman with high blood pressure. She takes a beta-blocker. At her appointment, she pulls out her phone and shows her doctor three months of resting heart rate from her ring. Over the past two weeks it’s been running about five beats a minute higher than usual. The doctor looks at the graph, nods, and goes back to the blood pressure they just took.
This year the American Medical Association and Medscape surveyed 2,222 physicians in the US, Canada, the UK, France, Germany, and Spain. Ninety-seven percent said they reviewed wearable data in some capacity. The highest rate of integration into practice in any of the six countries was 6%. I kept thinking about what happens in the few seconds between looking at the graph and going back to the cuff. AMA and Medscape survey findings
An older study of 392 recorded primary care visits gives you a sense of what the graph is up against. The median visit was just under 16 minutes. The typical visit covered six topics, and most got about a minute. In a more recent interview study, primary care clinicians worried about how much additional patient data they’d be expected to handle. Once the clinic asks patients to send readings between visits, there’s also a very ordinary question that someone has to answer. Who’s supposed to look at them, and when? Primary care visit study · Clinician interviews · Study on clinical responsibilities
Give the doctor a whole afternoon with that graph, though, and some of the questions still go unanswered.
Someone who works on a digital platform at a large US hospital told us that his mattress sensor kept flagging sleep apnea while he was away on business. Her English bulldog was sleeping in the bed. Usually it’s less obvious. How many nights did the woman actually wear the ring? If it spent four nights on the charger, which part of her week does the graph describe?
Then there are the five beats. In a study of more than 92,000 Fitbit users, day-to-day resting heart rate varied within individuals by about three beats per minute on average, measured as a standard deviation. That tells you something about ordinary variation across the study population. It gives you no cutoff for a two-week change in this woman. She might have been sleeping badly. She might have replaced the device. The graph gives the doctor a reason to ask what happened around the time it changed. It can’t answer the question for her. Fitbit resting heart rate study
And was the ring tested in people like her? The INTERLIVE checklist for validating optical heart rate measurements recommends excluding people taking medications that affect cardiovascular function, including beta-blockers, in its general testing protocol. The authors also discuss testing in particular clinical populations. A study can follow the general protocol beautifully and leave her doctor looking for a second study. INTERLIVE validation recommendations
For researchers - isn’t it a perfect environment? But what should a doctor do with data from a device tested only on healthy people during that visit with a 58-year-old woman with hypertension?
Even a perfectly accurate reading can be the wrong measurement for the decision. The SHIFT heart failure trial enrolled patients in sinus rhythm with a resting heart rate of at least 70 beats per minute, among other criteria. If a ring shows 64 while someone is asleep, how would you use that number to decide whether they meet the same criterion? You’d first have to establish how the two measurements relate. SHIFT trial
What happens when the device keeps producing readings and nobody knows what to do with them? Cardiologists described a 70-year-old woman with intermittent atrial fibrillation who took 916 ECGs on her smartwatch in a year. She developed health anxiety and made 12 visits to clinics and emergency departments. Her treatment didn’t change. A later observational study of people with atrial fibrillation also found greater healthcare use among wearable users. It couldn’t establish what caused the difference. I keep coming back to the woman who took 916 ECGs. Imagine how much of her year she spent wondering whether she needed to take another one. The case report · The observational study
What did you call “resting”?
One thing the industry could do tomorrow is tell people what it’s measuring.
In that Fitbit study, the estimate was designed to approximate your heart rate lying down just after you wake up. Oura shows your lowest and average heart rates during the night. Apple describes how it calculates a daily resting rate from background heart rate readings and motion data, with little detail on how a particular reading makes it into the calculation. You can see why each company made its choice. Now imagine a doctor being visited by patients with all those devices, what should he do? Fitbit study methods · Oura’s resting heart rate documentation · Apple’s heart rate documentation
HRV makes this even easier to see. Apple HealthKit defines its HRV data type as SDNN, the standard deviation of the intervals between normal heartbeats. Android’s Health Connect defines its HRV record as RMSSD, a measure based on how much each interval differs from the one before it. Both come in milliseconds. If the chart just says “HRV,” you can switch devices and change statistics without changing the label. Apple HealthKit · Android Health Connect
You can also keep the same device and get a software update. This is where I thought we’d be able to look at the records arriving at Welltory and give a straightforward answer about what produced each number. We did a few spot checks on November 19, looking at up to ten users in each targeted check. Some Apple Watch records came with the model and watchOS version. Some Fitbit records came with the model. Some integrations sent the phone app version with no device attached. Apple Health has a field for firmware, and we saw it populated by a Withings blood pressure monitor. In the Health Connect device record, there’s a place for the manufacturer, model, and device type. Apple’s device fields · Health Connect’s device fields
The nicest example we found was Oura’s own API, which includes a ring ID and a sleep algorithm version on its sleep records. We didn’t see that version on the records we received through Apple Health. The company had already done the work of labeling the algorithm. We just couldn’t see the label on the other route or other parameters. Oura API documentation
That detail came up in an argument about sleep staging this year. A 2025 study in a clinical sample found that the Oura Ring Gen3 agreed with polysomnography on four-way sleep staging about 53% of the time. Oura took issue with the study, including how it related to the company’s newer algorithm. I think the version is a fair thing to bring up. I’d also like the person reading the study, or looking at a patient’s sleep history, to be able to tell which version they’re dealing with. The sleep study · Oura’s response
And then there’s the question of who gets to check.
Oura’s public developer agreement explicitly mentions benchmarking among the uses restricted without prior written consent.
Fitbit’s platform terms restrict using the platform for benchmarking, and
WHOOP’s API terms require prior written approval for statements that reference WHOOP or its APIs.
These are terms for developers using particular services. They matter because the people who work with data from several devices are often the first ones in a position to notice that something changed. I’d like them to be able to check it and tell the rest of us what they found. Oura’s developer agreement · Fitbit’s platform terms · WHOOP’s API terms
What the cuff had going for it
I think the surgeon was right about how much work it took to make a measurement ordinary.
Labs had to do it with HbA1c. Different methods needed to report results on a common scale, so a standardization program tied those measurements to the reference used in a major diabetes study. Creatinine had a similar problem. Differences between assays made estimates of kidney function less reliable, and labs had to work on calibration. Home blood pressure took a different route. European guidelines give measurements taken at home their own thresholds, based on a defined way of taking and averaging the readings. Home was a useful setting in its own right. It was worth studying what the numbers meant there. HbA1c standardization · Creatinine standardization recommendations · European guidance on home blood pressure
I’d start the wearable conversation with the woman and her ring. Ask her doctor what decision they’d revisit if they could reliably see her heart rate between appointments. Maybe there’s a good reason to measure it near the end of sleep. Maybe a different window would be more useful. Then we can study how much data is enough, how big a change matters, and whether the answer holds for people taking beta-blockers. Until we have a decision in mind, even the question of who should pay to collect and review the data is hard to answer.
Device makers can say which readings go into a metric, when they’re taken, and how they’re summarized, while keeping the code proprietary. They can also attach the device and algorithm version to the records they export. Apple Health and Health Connect can give that information a place to travel. Then, when a study comes out, someone can check whether it applies to the number they’re looking at.
What can you do now?
If you’re the one bringing a graph to an appointment, I’d bring the trend and say how you got it. If you changed devices, mark when. If you’re taking your own readings, a consistent time and position make them easier to compare. With sleep, I’d start with something you can also describe in your own words. “I’ve been waking up an hour earlier for the past month” gives you and your doctor something to talk about before either of you has to make sense of the ring’s REM estimate.
And if you’re a clinician who’s already using wearable data, I’d love to hear about one patient whose data changed a decision you made. What did you need to know about the number before you trusted it enough to act?
In a six-month trial of older adults at elevated risk of stroke, doctors diagnosed atrial fibrillation in 21 people in the Apple Watch group and five in the usual-care group, out of about 220 people in each. Twelve of the 21 had no symptoms. There was nothing for them to remember and bring up at their next appointment. That’s what makes these devices worth taking seriously. Many people already wear something that can record what happens in the weeks and months a doctor usually has to reconstruct from a conversation. I want us to finish the work of figuring out which of those measurements a doctor can trust, and what they can do for a patient once they have them. The EQUAL trial


