Gerard de Haan

Robust Pulse Rate From Chrominance-Based rPPG

Remote photoplethysmography (rPPG) enables contactless monitoring of the blood volume pulse using a regular camera. Recent research focused on improved motion robustness, but the proposed blind source separation techniques (BSS) in RGB color space show limited success. We present an analysis of the motion problem, from which far superior chrominance-based methods emerge. For a population of 117 stationary subjects, we show our methods to perform in 92% good agreement (±1.96σ) with contact PPG, with RMSE and standard deviation both a factor of 2 better than BSS-based methods. In a fitness setting using a simple spectral peak detector, the obtained pulse-rate for modest motion (bike) improves from 79% to 98% correct, and for vigorous motion (stepping) from less than 11% to more than 48% correct. We expect the greatly improved robustness to considerably widen the application scope of the technology.

Exploiting Spatial Redundancy of Image Sensor for Motion Robust rPPG

Remote photoplethysmography (rPPG) techniques can measure cardiac activity by detecting pulse-induced color variations on human skin using an RGB camera. State-of-the-art rPPG methods are sensitive to subject body motions (e.g., motion-induced color distortions). This study proposes a novel framework to improve the motion robustness of rPPG. The basic idea of this paper originates from the observation that a camera can simultaneously sample multiple skin regions in parallel, and each of them can be treated as an independent sensor for pulse measurement. The spatial redundancy of an image sensor can thus be exploited to distinguish the pulse signal from motion-induced noise. To this end, the pixel-based rPPG sensors are constructed to estimate a robust pulse signal using motion-compensated pixel-to-pixel pulse extraction, spatial pruning, and temporal filtering. The evaluation of this strategy is not based on a full clinical trial, but on 36 challenging benchmark videos consisting of subjects that differ in gender, skin types, and performed motion categories. Experimental results show that the proposed method improves the SNR of the state-of-the-art rPPG technique from 3.34 to 6.76 dB, and the agreement (±1.96σ) with instantaneous reference pulse rate from 55% to 80% correct. ANOVA with post hoc comparison shows that the improvement on motion robustness is significant. The rPPG method developed in this study has a performance that is very close to that of the contact-based sensor under realistic situations, while its computational efficiency allows real-time processing on an off-the-shelf computer.

Low Cost Robust Blur Estimator

In this paper a novel local blur estimation method is presented. The focal blur process is usually modeled as a Gaussian low-pass filtering and then the problem of blur estimation is to identify the Gaussian blur kernel. In the proposed method, the blurred input image is first re-blurred by Gaussian blur kernels with different blur radii. Then the difference ratios between the multiple re-blurred images and the input image are used to determine the unknown blur radius. We show that the proposed method does not require edge detection preprocessing and can estimate a wide range of blur radius. Experimental results of the proposed method on both synthetic and natural images and a comparison with a state-of-the-art method are presented.

A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation

In this paper, we propose a conceptually novel algorithm, namely “Spatial Subspace Rotation” (2SR), that improves the robustness of remote photoplethysmography. Based on the assumption of 1) spatially redundant pixel-sensors of a camera, and 2) a well-defined skin mask, our core idea is to estimate a spatial subspace of skin-pixels and measure its temporal rotation for pulse extraction, which does not require skin-tone or pulse-related priors in contrast to existing algorithms. The proposed algorithm is thoroughly assessed on a benchmark dataset containing 54 videos, which includes challenges of various skin-tones, body-motions in complex illuminance conditions, and pulse-rate recovery after exercise. The experimental results show that given a well-defined skin mask, 2SR outperforms the popular ICA-based approach and two state-of-the-art algorithms (CHROM and PBV). When comparing the pulse frequency spectrum, 2SR improves on average the SNR of ICA by 2.22 dB, CHROM by 1.56 dB, and PBV by 1.95 dB. When comparing the instant pulse-rate, 2SR improves on average the Pearson correlation and precision of ICA by 47% and 65%, CHROM by 22% and 23%, and PBV by 21% and 39%. ANOVA confirms the significant improvement of 2SR in peak-to-peak accuracy. The proposed 2SR algorithm is very simple to use and extend, i.e., the implementation only requires a few lines MATLAB code.

Algorithmic Principles of Remote PPG

This paper introduces a mathematical model that incorporates the pertinent optical and physiological properties of skin reflections with the objective to increase our understanding of the algorithmic principles behind remote photoplethysmography (rPPG). The model is used to explain the different choices that were made in existing rPPG methods for pulse extraction. The understanding that comes from the model can be used to design robust or application-specific rPPG solutions. We illustrate this by designing an alternative rPPG method, where a projection plane orthogonal to the skin tone is used for pulse extraction. A large benchmark on the various discussed rPPG methods shows that their relative merits can indeed be understood from the proposed model.

Smoothness Constraints in Recursive Search Motion Estimation for Picture Rate Conversion

Many motion compensation algorithms are based on block matching. The quality of the block correlation depends on the validity of the brightness constancy assumption and the assumption of fixed translational motion within a block. These assumptions are invalid in areas with texture changes, noise, lighting changes, and rapid deformations. Smoothness priors should enforce stable estimates in these regions by propagating neighboring estimates, while preserving hard object boundaries (piecewise smoothness). Most motion estimation algorithms that successfully implement these constraints are computationally complex. In this paper, we show an intuitive and computationally efficient way to implement them within the framework of (real-time) recursive search, targeting consumer-market embedded devices with limited resources.

Adaptive image restoration based on local robust blur estimation

This paper presents a novel non-iterative method to restore the out-of-focus part of an image. The proposed method first applies a robust local blur estimation to obtain a blur map of the image. The estimation uses the maximum of difference ratio between the original image and its two digitally re-blurred versions to estimate the local blur radius. Then adaptive least mean square filters based on the local blur radius and the image structure are applied to restore the image and to eliminate the sensor noise. Experimental results have shown that despite its low complexity the proposed method has a good performance at reducing spatially varying blur.

Content-adaptive video up-scaling for high definition displays

The introduction of HDTV asks for spatial up-conversion techniques enabling the display of standard resolution material. Recently, X. Li and M. Orchard proposed the New Edge-Directed Interpolation (NEDI) algorithm for high quality up-scaling of natural images. We shall show that the method, although it generally behaves well, introduces annoying artifacts in fine-textured areas. Based on an analysis of these artifacts and taking advantage of the temporal correlation between video images, we propose an improved NEDI algorithm. In our evaluation, we compare the performance of the original and the improved NEDI method on a significant set of test images. We conclude from both subjective and objective measures that the proposed modifications improve the overall performance of NEDI.

Ballistocardiographic Artifacts in PPG Imaging

Objective: Photoplethysmography (PPG) is a noninvasive technique to measure the blood-volume pulse and derive various vital signs. Camera-based PPG imaging was recently proposed for clinical microvascular assessment, but motion robustness is still an issue for this technique. Our study aims to quantify cardiac-related, i.e., ballistocardiographic (BCG), motion as a source of artifacts in PPG imaging. Methods: In this paper, using the human head as a relevant region of interest, the amplitude of BCG-artifacts was modeled for a Lambertian surface illuminated by a light source. To derive peak-to-peak head displacements for the model, we recorded, on 54 subjects, PPG and inertial sensor data at the pulse and cranial vertex. We simulated the effect of light source location at a mesh representation of a human face and conducted additional experiments on a real subject. Results: Under nonorthogonal illumination, the relative strength of the BCG artifacts is strong enough, compared to the amplitude of PPG signals, to compromise PPG imaging in realistic scenarios. Particularly affected are the signals obtained in the nongreen part of the spectrum and/or when the incident angle at the skin surface exceeds 45°. Conclusion: From the model and an additional experiment conducted on real skin, we were able to prove that homogenous and orthogonal illumination is a means to minimize the problem. Significance: Our illumination recommendation provides a simple and effective means to improve the validity of remote PPG-imagers. We hope that it helps to prevent mistakes currently seen in many publications on remote PPG.

Robust respiration detection from remote photoplethysmography

Continuous monitoring of respiration is essential for early detection of critical illness. Current methods require sensors attached to the body and/or are not robust to subject motion. Alternative camera-based solutions have been presented using motion vectors and remote photoplethysmography. In this work, we present a non-contact camera-based method to detect respiration, which can operate in both visible and dark lighting conditions by detecting the respiratory-induced colour differences of the skin. We make use of the close similarity between skin colour variations caused by the beating of the heart and those caused by respiration, leading to a much improved signal quality compared to single-channel approaches. Essentially, we propose to find the linear combination of colour channels which suppresses the distortions best in a frequency band including pulse rate, and subsequently we use this same linear combination to extract the respiratory signal in a lower frequency band. Evaluation results obtained from recordings on healthy subjects which perform challenging scenarios, including motion, show that respiration can be accurately detected over the entire range of respiratory frequencies, with a correlation coefficient of 0.96 in visible light and 0.98 in infrared, compared to 0.86 with the best-performing non-contact benchmark algorithm. Furthermore, evaluation on a set of videos recorded in a Neonatal Intensive Care Unit (NICU) shows that this technique looks promising as a future alternative to current contact-sensors showing a correlation coefficient of 0.87.