Follow Datanami:
April 6, 2021

University of Washington Researchers Develop ML-Enabled Telehealth Tool

For clear reasons, telehealth has boomed during the pandemic, and its popularity remains even as vaccinations become more accessible. With this paradigm shift in treatment, some researchers are examining how to enable patients to measure more of their vitals in a home setting, decreasing the necessity of many in-person medical appointments. At the University of Washington (UW), a team has developed a machine learning-enabled system that they hope will allow users to accurately measure their pulses and respiratory rates using only a smartphone or computer camera.

In contrast to tools like Google Fit’s smartphone camera-driven pulse and respiratory rate measurement tool (which requires you to place your finger directly over the camera), the UW tool works by using a device’s camera to capture images of a user’s face. The machine learning elements of the tool then analyze the reflections of light across that face, using minute fluctuations to determine the patient’s vitals.

To develop the initial machine learning model, the researchers trained it with a dataset of videos of faces, paired with information on those individuals’ vital signs over the course of those videos. The first version of this system was presented by the team in December. Relative to that prior work, the new system is better-able to accommodate different cameras, lighting, facial features, and skin colors.

“Every person is different, so this system needs to be able to quickly adapt to each person’s unique physiological signature, and separate this from other variations, such as what they look like and what environment they are in,” explained lead author Xin Liu, a doctoral student at UW, in an interview with UW’s Sarah McQuate. Liu did acknowledge that the system still has deficits. “We acknowledge that there is still a trend toward inferior performance when the subject’s skin type is darker. This is in part because light reflects differently off of darker skin, resulting in a weaker signal for the camera to pick up. Our team is actively developing new methods to solve this limitation.”

While the system is still in development, the researchers are working on collaborations with physicians to begin initial testing and are hopeful that the eventual product will improve telehealth for many patients.

“Any ability to sense pulse or respiration rate remotely provides new opportunities for remote patient care and telemedicine. This could include self-care, follow-up care or triage, especially when someone doesn’t have convenient access to a clinic,” said senior author Shwetak Patel, a professor at UW. “It’s exciting to see academic communities working on new algorithmic approaches to address this with devices that people have in their homes.”

Datanami