Three Smartwatches Tested in Low Oxygen

Most smartwatch oxygen tests look at one watch at a time. A 2023 study from the Czech Technical University in Prague put three through the same test: an Apple Watch 8, a Samsung Galaxy Watch 5 and a Withings ScanWatch. Volunteers breathed low-oxygen air, and each watch was compared with a hospital finger oximeter. It was a small randomised crossover study in a lab.

Who was studied?

• 18 healthy white volunteers aged 21 to 26: 14 men and 4 women.
• Each did the test three times, once per watch, in random order, with at least 2 hours between.
• They sat still with the watch on the left wrist and the reference sensor, a Masimo Radical-7, on the left middle finger.
• They breathed room air for 2 minutes, then air with 14%, 12% and 10% oxygen for 2.5 minutes each, then room air again.
• The reference fell from 98% to about 78% on average.
• The researchers read both devices by hand every 40 to 50 seconds: 16 times per test, 288 attempts per watch.

How close were the watches?

The study's main measure is the root mean square error (Arms): the lower it is, the closer the watch was to the reference.

• The standards. ISO 80601-2-61 asks for an Arms of 4.0 or less across the 70% to 100% range, and the FDA asks for 3.5 or less from reflectance sensors, the kind a watch uses. The authors say all three met the ISO limit, and the Apple and Withings watches met the FDA one.
• Samsung read low. Its average was statistically significantly below the reference at all 16 time points, by 1.8 to 3.2 points. The gap shrank as oxygen fell.
• Withings lagged. At one time point its average was 8.3 points below the reference. The authors say this may come from its 30-second measurement, against 15 seconds for the Apple Watch and 12 to 17 for the Samsung.

The Arms by oxygen range, from the paper's Table 2:

• 91% to 100%. Apple 2.1, Samsung 4.1, Withings 3.6.
• 81% to 90%. Apple 2.2, Samsung 3.7, Withings 3.5.
• 80% and under. Apple 2.5, Samsung 2.7, Withings 2.9.

The paper's text says every Arms was under 4, but its Table 2 prints 4.1 for Samsung in the 91% to 100% range.

How often did a watch give no reading?

• Apple gave a reading on 274 of 288 attempts (95%).
• Samsung on 283 (98%).
• Withings on 238 (83%).

That left 795 paired readings. 112 of them (14%) were at 80% or under.

Did they agree on readings under 90%?

The study counted a reference reading under 90% as low oxygen and worked out how often each watch matched it. The paper doesn't state the cutoff applied to the watch reading.

• Reference under 90%, watch matched (sensitivity). Apple 91%, Samsung 97%, Withings 92%.
• Reference 90% or over, watch matched (specificity). Apple 95%, Samsung 76%, Withings 86%.
• All readings together (accuracy). Apple 93%, Samsung 84%, Withings 89%. The Apple figure was statistically significantly higher than the Samsung one.

The authors link Samsung's pattern to its low readings: the highest sensitivity of the three and the lowest specificity. The authors say they'd rather a watch read low than high.

What can't it tell you?

• Other people. Everyone was young, healthy and white. The authors say results may differ in older patients and with other skin tones. See pulse oximeters and skin tone.
• Real blood oxygen. The reference was another pulse oximeter, not a blood sample. For a test against blood samples, see Apple Watch vs a finger oximeter in low oxygen.
• Overnight or everyday use. People sat still with a hand on a table, and the watch position was checked first. The authors expect more failed readings in routine use.
• Steady low oxygen. Oxygen was falling during the test, and the authors say they couldn't fully separate measurement error from the delay in each watch's reading.
• Very low oxygen. 112 of the 795 readings were at 80% or under.

The authors report no conflicts of interest. They also say smartwatches aren't meant for clinical oxygen measurement, and that their slow readings and sensitivity to movement limit that use.

What can you do with this?

• Read a watch's oxygen number as an estimate that can sit several points from a finger oximeter.
• Expect some attempts to give no number at all.
• Look at the trend over many nights, not one reading.
• If you feel unwell, or your readings worry you, talk to a doctor.

More on this: smartwatch SpO2 vs a pulse oximeter and smartwatch SpO2 accuracy at altitude.

Not a medical device. Does not diagnose, treat or prevent any condition. Discuss results and changes with a clinician.

References

1. Walzel S, Mikus R, Rafl-Huttova V, Rozanek M, Bachman TE, Rafl J. Evaluation of Leading Smartwatches for the Detection of Hypoxemia: Comparison to Reference Oximeter. Sensors. 2023;23(22):9164. doi:10.3390/s23229164