Knowledge Base

Vital Signs Performance Evaluation

To demonstrate the effectiveness of FaceReader's voice analysis, we evaluated and benchmarked their performance on a collection of established and proprietary RPPG datasets. This collection consists of approximately 200 front-facing videos of approximately 50 participants, and includes ground truth signals for heart rate, heart rate variability and breathing rate under various recording conditions. This makes them suitable for objective validation and comparison with existing methods.

For this evaluation, we divided the data into two subsets:

• A high-compression/low-movement set, where the participants show minimal movement; these videos are highly compressed.
• A low-compression/high-movement set, wherein participants make certain head/body movements; these videos are uncompressed.

State-of-the-Art Accuracy

As video compression can introduce unwanted noise, all heart rate related evaluations were performed on the low-compression/high-movement subset.

For heart rate estimation, we report a mean absolute error (MAE) of 0.21 bpm (Table 1), demonstrating a state-of-the-art result. Compared to previously published methods, our method shows a significant improvement in accuracy.

In terms of HRV, our results indicate improved performance over earlier methods as well, with a lower MAE of 14 ms for RMSSD and 6 ms for SDNN, compared to 16.8 ms and 8.1 ms reported by Finžgar [8].

Based on the HRV literature and considering that the average human heart rate variability is in the range of 19–75 ms RMSSD, error rates of approximately 30 ms or lower RMSSD are generally considered acceptable for distinguishing between broad HRV level groups. Our results fall well within this range.

Table 1: Heart Rate Estimation Performance

Heart rate estimation performance of FaceReader, in comparison with prior methods.

Table 2: Heart Rate Variability Performance

Heart rate variability performance of FaceReader, in comparison with prior methods.

These results show that FaceReader can estimate heart rate and HRV with sufficient accuracy even under movement conditions when the video quality is sufficiently high.

How Compression Noise Impacts Accuracy

Heart rate estimation is based on the extraction of a faint signal that is invisible to the naked eye.

Video compression algorithms, which are commonly used to reduce file size, preserve visual quality but remove the invisible heart rate information contained in a video, introducing what we refer to as compression noise. This can drastically reduce the quality of the extracted signal and thereby reduce the accuracy of the estimated heart rate.

The effect of compression noise can clearly be seen when comparing heart rate estimation accuracy on low-compression and high-compression video subsets. Despite the high-compression videos having relatively low movement, their high video compression noise level results in noticeably worse performance. In contrast, the low-compression videos, which have more motion but no compression, achieve higher accuracy — showing the extent of performance degradation caused by video compression noise.

Highly Accurate Measurement of Breathing Rate

To evaluate the accuracy of our breathing rate estimation method, we benchmarked its performance against existing approaches using the low-movement subset. On these videos, we achieved a mean absolute error (MAE) of 0.55 breaths per minute, showing state-of-the-art performance compared to other published methods for breathing rate evaluation.

Table 3: Breathing Rate Performance

Breathing rate performance of FaceReader, in comparison with prior methods.

These results show that FaceReader is able to estimate the breathing rate accurately in limited-movement scenarios where the observed chest movement is not significantly distorted by movement noise.

For Your Research

The results demonstrate that measuring vital signs such as heart rate, breathing rate, and heart rate variability (HRV) through a standard webcam is both practical and accessible for researchers using FaceReader.

Provided that lighting and participant positioning are adequate, individual differences and technical variations have minimal impact on measurement accuracy.

As with any physiological assessment, there is a trade-off between ecological validity and measurement precision. For maximum precision, a controlled laboratory environment with specialized equipment may be preferable. However, measuring vital signs with FaceReader is ideal for studies that prioritize natural behavior and require scalable, non-intrusive data collection.

Recordings

When planning a video recording setup for analyses that include vital signs, the following guidelines can help to ensure optimal signal quality.

• Heart Rate and HRV: When recording videos, make sure to use a lossless compression method to reduce compression noise in the video. Ensure that the participant's forehead and upper face are clearly visible, and avoid strong specular reflections (e.g., bright white spots on the forehead). Note that heart rate and heart rate variability estimations may not work for participants wearing heavy makeup.
• Breathing Rate: For the cleanest breathing rate signal, minimize participants' upper-body movement. This works best in scenarios where participants are sitting calmly, such as when watching a stimulus or engaging in passive interaction.

Please refer to the Guidelines for an Optimal RPPG Measurement page in the FaceReader manual for more specific and detailed recommendations.

Source: EthoVision XT 18 - THC - Trial and Hardware Control, Noldus Information Technology