How do virtual doctors get my heart rate through the camera?

The evolution of virtual care from a simple video conferencing tool to a clinically robust modality hinges on data. While a conversation can provide subjective information, objective clinical data has traditionally required in-person visits or patient-owned peripheral devices. Today, that is changing. Health systems are now using a technology that uses the patient's own device, a smartphone or laptop, to perform a vital measurement. The ability to capture a patient's heart rate using only their camera is a prime example of this shift, turning a standard televisit into a data-rich clinical encounter through camera based clinical vitals.

"In a clinical validation study involving cardiovascular disease (CVD) patients, rPPG-derived pulse rate showed a mean absolute error (MAE) of 1.061 bpm and a Pearson correlation of 0.962 when compared to a medical-grade electrocardiogram (ECG)." - Peng, et al. (2021)

How camera-based clinical vitals analysis works

The technology that enables a virtual doctor to get a heart rate through a camera is called remote photoplethysmography (rPPG). It is an optical measurement technique that works by detecting subtle changes in light reflected from human skin. These changes, imperceptible to the human eye, correspond to the ebb and flow of blood in the vessels just below the surface. As the heart pumps, the volume of blood in the microvasculature changes, which in turn alters the amount of light that is absorbed and reflected. A standard RGB video camera can record these alterations.

The process for extracting a heart rate from a video stream involves several sophisticated steps:

1. Face Detection and Region of Interest (ROI) Selection: An algorithm first identifies the patient's face in the video feed. It then selects one or more ROIs, typically areas with good blood perfusion and minimal noise, such as the forehead and cheeks.
2. Signal Extraction: The system analyzes the video frames to track the average color values within the selected ROIs over time. The green channel of the RGB color model is often most effective, as hemoglobin absorbs green light, making the pulsatile signal stronger in this part of the spectrum.
3. Noise Filtering: This is a critical step. The raw signal contains noise from various sources, including patient motion (talking, shifting), changes in ambient lighting, and camera sensor noise. Advanced algorithms use signal processing and machine learning techniques to filter out these artifacts and isolate the underlying cardiac signal.
4. Pulse Calculation: Once the clean signal is extracted, a Fast Fourier Transform (FFT) or similar frequency analysis method is applied. This converts the time-based signal into the frequency domain, revealing a clear peak at the frequency corresponding to the patient's heart rate. This frequency is then converted into beats per minute (BPM).

Comparison of Remote (rPPG) vs. Contact (PPG) Measurement

For clinical informatics teams and virtual care directors, understanding the differences between camera-based methods and traditional contact-based sensors is essential for workflow design and technology evaluation.

| Feature | Camera-Based rPPG | Contact-Based PPG (e.g., Pulse Oximeter) | | :--- | :--- | :--- | | Methodology | Optical measurement of light reflection from the skin via a standard video camera. | Optical measurement of light transmission or reflection via a dedicated sensor in direct contact with the skin (e.g., fingertip, earlobe). | | Hardware Required | Patient's existing smartphone or computer with a standard RGB camera. | A dedicated medical or consumer-grade device (pulse oximeter, smartwatch). | | Patient Experience | Seamless and frictionless; no action or hardware required from the patient. Measurement occurs passively during the virtual visit. | Requires the patient to own, find, and correctly use a separate piece of hardware during the visit. | | Key Limitations | Sensitive to poor lighting conditions, significant patient motion, and certain skin tones. Requires advanced signal processing. | Can be affected by poor circulation, nail polish, or incorrect sensor placement. Requires physical hardware. | | Typical Use Case | Spot-checking vital signs during synchronous virtual visits to inform clinical decisions in real-time. | Continuous or spot-check monitoring, often in a clinical setting or for remote patient monitoring (RPM) programs. |

Industry applications for health systems

The integration of camera based clinical vitals into telehealth platforms is not just a technical curiosity; it has direct applications that can enhance clinical workflows and improve the quality of care.

Automated triage and intake

• Before a patient even speaks to a provider, their heart rate can be measured as they wait in the virtual "waiting room." This baseline data can be automatically populated into the EHR, giving the provider immediate context and helping triage nurses prioritize patients more effectively.

Chronic disease management

• For patients with conditions like hypertension or atrial fibrillation, regular heart rate monitoring is crucial. Camera-based measurements during routine follow-up televisits provide a valuable data point for medication management and treatment adjustments without requiring the patient to purchase or use a separate device.

Behavioral health and stress monitoring

• Heart Rate Variability (HRV), a measure derived from the beat-to-beat changes in heart rate, is a key indicator of autonomic nervous system function and stress. Some advanced rPPG systems can estimate HRV, providing objective data to support behavioral health screenings and interventions during virtual consultations.

Current research and evidence

The foundational work on rPPG was conducted over a decade ago. A seminal study by Ming-Zher Poh, Daniel McDuff, and Rosalind Picard at the MIT Media Lab in 2010 demonstrated the feasibility of measuring cardiac pulse from video recordings of the human face. Since then, research has accelerated to improve the accuracy and robustness of the technology.

More recent studies have focused on clinical validation. For instance, a 2021 study by Peng et al. evaluated an rPPG-based software application against medical-grade ECGs in patients with cardiovascular disease. Their findings showed a high degree of correlation, confirming the potential of rPPG for clinical use cases. The primary challenges that researchers are actively working to solve are improving performance across all skin tones, increasing resilience to motion artifacts, and expanding the range of measurable vitals to include respiration rate, blood pressure, and oxygen saturation. Deep learning and advanced AI models are at the forefront of these efforts, learning to better distinguish the cardiac signal from environmental noise.

The future of contactless vitals in virtual care

The trajectory for camera-based clinical vitals is moving toward a comprehensive, multi-parameter assessment. As algorithms become more sophisticated and validated against clinical standards, the virtual visit will transform from a conversation into a comprehensive, data-driven assessment. For health system CIOs and virtual care program directors, this technology represents a significant opportunity to enhance the clinical depth of telehealth encounters, improve provider decision-making, and create a more seamless and equitable experience for patients, regardless of whether they own peripheral medical devices.

The ability to capture objective measurements is a key step in solidifying the role of virtual care as a permanent and powerful component of the healthcare delivery system. It closes a critical data gap that has limited virtual encounters since their inception.

Frequently asked questions

Q: How accurate is getting a heart rate through a camera? A: The accuracy of camera-based heart rate measurement is dependent on factors like lighting, patient motion, and the quality of the underlying algorithm. Leading commercial solutions are validated against clinical-grade devices like ECGs and have been shown to achieve a high degree of accuracy (e.g., mean absolute error of 1-3 BPM) under typical virtual visit conditions.

Q: Does this work for patients of all skin tones? A: This is a critical area of research and development. Melanin, which is more present in darker skin tones, absorbs more light, which can make the underlying blood flow signal fainter. However, modern algorithms are specifically designed and trained on diverse datasets to mitigate this bias and ensure reliable performance across the full range of human skin tones.

Q: What does the patient or provider need to do to use this? A: For the most integrated solutions, the process is designed to be seamless. The measurement is initiated by the provider within their telehealth platform and runs automatically. The patient simply needs to remain relatively still and well-lit for a short period (typically 30-60 seconds) while the analysis is performed. There is no separate app to download or device to configure.

Q: Can other vitals besides heart rate be measured this way? A: Yes, the technology is evolving rapidly. Heart rate and respiration rate are the most established measurements. Research and development are actively underway to reliably measure blood pressure, oxygen saturation (SpO2), and heart rate variability (HRV) using the same camera-based approach.

As health systems look to standardize and scale their virtual care programs, the ability to capture objective data is critical. Circadify is at the forefront of addressing this need, developing solutions to embed seamless, camera-based clinical vitals into existing telehealth workflows. To learn more about how this technology can be integrated into your health system's virtual care strategy, explore our clinical workflow solutions at circadify.com/solutions/telehealth.