Knowledge Base

What Vital Signs Can You Measure with FaceReader?

With just a regular webcam, FaceReader lets you detect key physiological signals like heart rate and heart rate variability. These vital signs are an important dimension to consider in human-centric experiments and support the integration of physiological data into the analysis.

Without the need for extra hardware, you can measure heart rate, breathing rate, and heart rate variability (HRV).

• Heart rate is a strong indicator of cardiovascular effort and offers a practical way to approximate real-time energy use.
• Breathing rate can be used to estimate metabolic demand and highlight shifts in emotional arousal, e.g., distinguishing relaxed breathing from stress-induced hyperventilation.
• Heart rate variability is often used to assess autonomic nervous system balance, providing an indirect measure of overall recovery status and cumulative fatigue.

This white paper discusses the methodology and expected performance of FaceReader's vital sign measurement techniques, including validation examples and practical tips for optimal results.

How Does It Work?

FaceReader measures vital signs with two underlying technologies: one to estimate blood volume changes for heart rate and heart rate variability; and another to detect respiratory motion for measuring breathing rate.

At their core, both breathing and heart rate estimation methods work on the principle of extracting and cleaning an information-dense signal and detecting the relevant physiological information from this signal.

Estimating Heart Rate and Heart Rate Variability

Estimation of heart rate via photoplethysmography (PPG) is based on the principle that changes in the blood volume result in changes in the light reflectance of the skin. The visual color of the face changes with each heartbeat due to increase or decrease in blood volume within the skin's capillaries. While these changes are invisible to the naked eye, digital cameras are able to capture them remotely. When tracking these color changes in a patch of skin over time, the variations form a signal called the blood volume pulse. Extracting blood volume pulse changes using a camera is known as remote photoplethysmography or RPPG.

The RPPG algorithm processes video data in the following steps:

• Face detection and tracking: The RPPG algorithm starts by detecting and tracking the face to extract color changes from the cheeks and forehead. It reduces noise in the signal by excluding the eye and mouth regions.
• Noise reduction: After enough signal has been gathered during a brief calibration period, noise from movement and other sources is filtered from the signal. This is done by an initial aggregation step that removes movement information based on intensity changes in different color channels.
• Signal filtering: The signal is then further refined using a two-stage filter. A Fourier-based filter first removes all frequencies outside of the heart rate range and those caused by rhythmic head movements. This is followed by a time-based filter that produces a signal with clear peaks and valleys.
• Heartbeat detection: Each peak in this signal represents a heartbeat. The time differences between individual beats, the inter beat intervals (IBIs), are used to calculate heart rate and heart rate variability.

The RPPG pipeline in FaceReader is based on the work done in [1].

Heart Rate Calculation

The heart rate in beats per minute (bpm) is calculated as follows:

Where w represents a time window of 8.5 seconds, N represents the number of IBIs within that window, and 60 is the number of seconds per minute.

Heart Rate Variability Calculation

FaceReader supports the calculation of two HRV measures: the Root Mean Square of Successive Differences (RMSSD) and the Standard Deviation of Normal-to-Normal intervals (SDNN).

The calculation of both short-term (5 min window) and ultra-short term (<5 min window) heart rate variability is supported.

Estimating Breathing Rate

The breathing rate algorithm processes video data in the following steps:

• Chest motion detection: Breathing rate is estimated by detecting key points on the chest and shoulders and tracking their motion across video frames.
• Motion signal extraction: After removing irrelevant points not correlated with breathing, the changes in motion of individual points are combined into a single motion signal.
• Signal filtering: This signal is cleaned using a two-stage filter, similar to how it works with heart rate. A Fourier-based filter first removes all frequencies from the signal outside of the breathing rate range. This is followed by a time-based filter that creates a signal with clear peaks and valleys, from which breathing rate can be calculated.

Breathing Rate Calculation

The breathing rate in breaths per minute (bpm) is calculated from the average of inter-breath intervals (IBIs):

Where w represents a time window of 15-30 seconds, N represents the number of IBIs within that window, and 60 is the number of seconds per minute.

How Well Does FaceReader Measure Vital Signs?

To demonstrate the effectiveness of FaceReader's vital sign analysis, performance was evaluated and benchmarked on a collection of established and proprietary RPPG datasets. This collection consists of approximately 200 front-facing videos of approximately 50 participants, and includes ground truth signals for heart rate, heart rate variability, and breathing rate under various recording conditions, making them suitable for objective validation and comparison with existing methods.

For this evaluation, the data was divided into two subsets:

• A high-compression/low-movement set, where the participants show minimal movement; these videos are highly compressed.
• A low-compression/high-movement set, wherein participants make certain head/body movements; these videos are uncompressed.

State-of-the-Art Accuracy

As video compression can introduce unwanted noise, all heart rate related evaluations were performed on the low-compression/high-movement subset.

For heart rate estimation, a mean absolute error (MAE) of 0.21 bpm was achieved (Table 1), demonstrating a state-of-the-art result. Compared to previously published methods, this represents a significant improvement in accuracy.

In terms of HRV, results indicate improved performance over earlier methods as well, with a lower MAE of 14 ms for RMSSD and 6 ms for SDNN, compared to 16.8 ms and 8.1 ms reported by Finzgar [8].

Based on the HRV literature and considering that the average human heart rate variability is in the range of 19-75 ms RMSSD, error rates of approximately 30 ms or lower RMSSD are generally considered acceptable for distinguishing between broad HRV level groups. These results fall well within this range.

Table 1. Heart rate estimation performance of FaceReader, in comparison with prior methods.

Method	MAE	STD
LiCVPR [3]	28.2	-
ICA [4]	24.1	30.9
NMD-HR [5]	8.7	24.1
2SR [6]	2.4	-
CHROM [7]	2.07	-
FaceReader 10	0.21	0.35

Table 2. Heart Rate Variability performance of FaceReader, in comparison with prior methods.

Method	HRV Metric	MAE	STD
Finzgar [8]	RMSSD	16.8	-
FaceReader 10	RMSSD	14.6	13.2
Finzgar [8]	SDNN	8.1	-
FaceReader 10	SDNN	6.1	6

These results show that FaceReader can estimate heart rate and HRV with sufficient accuracy even under movement conditions when the video quality is sufficiently high.

How Compression Noise Impacts Accuracy

Heart rate estimation is based on the extraction of a faint signal that is invisible to the naked eye. Video compression algorithms, which are commonly used to reduce file size, preserve visual quality but remove the invisible heart rate information contained in a video, introducing what is referred to as compression noise. This can drastically reduce the quality of the extracted signal and thereby reduce the accuracy of the estimated heart rate.

Despite high-compression videos having relatively low movement, their high video compression noise level results in noticeably worse performance. In contrast, low-compression videos, which have more motion but no compression, achieve higher accuracy - showing the extent of performance degradation caused by video compression noise.

Highly Accurate Measurement of Breathing Rate

To evaluate the accuracy of the breathing rate estimation method, performance was benchmarked against existing approaches using the low-movement subset. On these videos, a mean absolute error (MAE) of 0.55 breaths per minute was achieved, showing state-of-the-art performance compared to other published methods for breathing rate evaluation.

Table 3. Breathing rate performance of FaceReader, in comparison with prior methods.

Method	MAE	STD
Tarassenko et al. [9]	9.0	11.8
Poh et al. [10]	4.2	5.2
Mehta et al. [11]	3.6	4.6
Massaroni et al. [12]	2.4	5.7
OPOIRES [2]	0.62	1.4
FaceReader 10	0.55	0.73

These results show that FaceReader is able to estimate the breathing rate accurately in limited-movement scenarios where the observed chest movement is not significantly distorted by movement noise.

Using FaceReader for Your Research

The results demonstrate that measuring vital signs such as heart rate, breathing rate, and heart rate variability (HRV) through a standard webcam is both practical and accessible for researchers using FaceReader.

Provided that lighting and participant positioning are adequate, individual differences and technical variations have minimal impact on measurement accuracy.

As with any physiological assessment, there is a trade-off between ecological validity and measurement precision. For maximum precision, a controlled laboratory environment with specialized equipment may be preferable. However, measuring vital signs with FaceReader is ideal for studies that prioritize natural behavior and require scalable, non-intrusive data collection.

Practical Tips for Optimal Vital Signs Recordings

When planning a video recording setup for analyses that include vital signs, the following guidelines can help to ensure optimal signal quality.

• Heart rate and HRV: When recording videos, make sure to use a lossless compression method to reduce compression noise in the video. Ensure that the participant's forehead and upper face are clearly visible, and avoid strong specular reflections (e.g., bright white spots on the forehead). Note that heart rate and heart rate variability estimations may not work for participants wearing heavy makeup.
• Breathing rate: For the cleanest breathing rate signal, minimize participants' upper-body movement. This works best in scenarios where participants are sitting calmly, such as when watching a stimulus or engaging in passive interaction.

Please refer to the Guidelines for an optimal RPPG measurement page in the FaceReader manual for more specific and detailed recommendations.

References

1. A. Gudi, M. Bittner, and J. van Gemert (2020). Real-time webcam heart-rate and variability estimation with clean ground truth for evaluation. Applied Sciences, vol. 10 (23), p. 8630, doi: 10.3390/app10238630.
2. A. Deo Mehta and H. Sharma. (2023). OPOIRES: A robust non-contact respiratory rate extraction based on optimal points-of-interest selection from an RGB camera. Biomedical Signal Processing and Control, vol. 86, p. 105193, doi: 10.1016/j.bspc.2023.105193.
3. X. Li, J. Chen, G. Zhao, and M. Pietikainen. (2014). Remote heart rate measurement from face videos under realistic situations, in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4264-4271.
4. M.-Z. Poh, D. J. McDuff, and R. W. Picard. (2010). Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express, vol. 18 (10), pp. 10762-10774.
5. H. Demirezen and C. E. Erdem. (2018). Remote photoplethysmography using nonlinear mode decomposition, in 2018 IEEE international conference on acoustics, speech and signal processing, IEEE, pp. 1060-1064.
6. W. Wang, S. Stuijk, and G. De Haan. (2015). A novel algorithm for remote photoplethysmography: Spatial subspace rotation, IEEE transactions on Biomedical Engineering, vol. 63 (9), pp. 1974-1984.
7. G. De Haan and V. Jeanne (2013). Robust pulse rate from chrominance-based rPPG, IEEE Transactions on Biomedical Engineering, vol. 60 (10), pp. 2878-2886.
8. M. Finzgar and P. Podrzaj (2020). Feasibility of assessing ultra-short-term pulse rate variability from video recordings, PeerJ, vol. 8, p. e8342.
9. L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. A. Clifton, and C. Pugh. (2014). Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiological Measurement, 35 (5), 807. https://doi.org/10.1088/0967-3334/35/5/807.
10. M.-Z. Poh, D. J. McDuff, and R. W. Picard. (2010). Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering, 58 (1), 7-11.
11. A. D. Metha and H. Sharma. (2020). Tracking nostril movement in facial video for respiratory rate estimation. In 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 1-6, https://doi.org/10.1109/ICCCNT49239.2020.9225464.
12. C. Massaroni, E. Schena, S. Silvestri, and M. Soumyajyoti. (2019). Comparison of two methods for estimating respiratory waveforms from videos without contact. In IEEE International Symposium on Medical Measurements and Applications (MeMeA), 1-6. https://doi.org/10.1109/MeMeA.2019.8802167.

---

Download the original PDF: White paper - FaceReader 10 - Vital signs (PDF)