1.

Introduction

Pulse rate is widely recognized as an important cardio-vascular parameter for assessing a patient’s health. Current methods of measuring pulse rate rely on the placement of electrodes on patient’s skin. Continuous monitoring using commercial pulse oximetry sensors attached to a finger or earlobe pose discomfort, susceptible to infections and could lead to irritation of the skin. In addition, the sensors must be placed and read by a qualified medical professional, resulting in potential bottlenecks when attending to patients at healthcare facilities. Remote nonobtrusive monitoring is clearly an attractive alternative, provided that accurate measurements are obtained. Past attempts for remote nonobtrusive monitoring of the pulse rate include the use of photoplethysmography (PPG). In PPG, a dedicated light source is used to capture the blood volume pulse (BVP) by observing variations in reflected light due to pulsatile blood volume. Spectral estimation is applied to the BVP to extract the pulse rate.

Recent work focused on obtaining BVP using ambient light rather than a dedicated light source.^1–4 The works in Refs. 1 and 2 pioneered an approach to extract pulse rate measurements¹ and PPG signals² using digital RGB cameras. The work in Ref. 3 improved on the accuracy of such methods by applying blind source separation using independent component analysis (ICA) and incorporating face tracking to automatically capture the face of a single or multiple patients. The work in Ref. 3 used a webcam to capture the video and was recently expanded to include measurements of other parameters such as the heart rate variability (HRV) based on the ICA approach.⁴

Although ICA is a fairly new technique, it was successfully used in many source separation problems. It assumes the observed signals are the result of a linear mixture of independent sources, a.k.a. independent components (ICs). The number of sources is equal to the number of observations, i.e., the linear mixture model is represented by a square matrix. ICA extracts the ICs responsible for the observed signals.⁵

In the framework of pulse rate measurements, we are interested in extracting the periodic variations in color due to the pulsating blood flow beneath the skin surface controlled by the heart. Other sources are mixed with the pulsating heart, such as fast variations in ambient light and movement of the subject. These act as interference in our measurements.

Standard ICA techniques suffer from the sorting problem: the independent components are not ordered, meaning that the source signal of interest could be present in any of the ICA outputs. The work in Refs. 3 and 4 recognized the sorting problem and resolved it either by always selecting the second IC,³ or by selecting the IC for which the peak frequency has the highest power.⁴ In the latter case, the selected component varies from measurement to measurement. Results in Refs. 3 and 4 were obtained from a dataset of 12 videos.

In this contribution, we propose the use of constrained ICA (cICA) to improve the accuracy of BVP measurements using a webcam. cICA is a useful extension of ICA and is applicable in cases where prior knowledge about the underlying sources is available.^6–8 Using cICA, a single source closely related to the reference signal is extracted in the optimization process. cICA has been successfully used in the past to extract an underlying signal of interest. A relevant example can be found in Ref. 6, where the weak signal of a fetal electrocardiogram (ECG) was extracted after removing the dominant maternal signal using a cICA approach. In the framework of pulse rate measurement, cICA can help avoid the sorting problem and extract the BVP as the source of interest even when other sources exist.

In what follows, we design, implement and test a novel cICA-based algorithm for accurately measuring the pulse rate. We demonstrate how the sorting problem hinders performance of the ICA based approach and how the proposed cICA algorithm solves the problem and improves performance. In addition, we perform a comparative study between the ICA based algorithms of Refs. 3 and 4 and the proposed cICA algorithm over a dataset of 45 videos.

2.

Proposed Algorithm

In one form of cICA a single output is forced and optimized to match a reference signal embodying prior knowledge about an underlying IC.^7,8 Since the sources considered in source separation usually have nonGaussian distributions, negentropy ($\mathcal{J}$) is used to separate the sources. To this end, $\mathcal{J}$ is estimated using approximations as described in Ref. 7 and our cICA problem is modeled as follows:

Minimize:

Subject to:

Initially, $\xi$ is chosen to be very small so as to avoid any local minima. It is then gradually increased within its range to converge to the global minima. Optimization of Eq. (1) with equality and inequality constraints in Eq. (2) would yield a single IC at the output which is closest to the input reference signal.

The proposed cICA algorithm is depicted in Fig. 1 using a functional block scheme. A video is captured using an RGB webcam along with face tracking for locating the region of interest (RoI) for pulse detection (entire face region). The RGB traces from the RoI are pre-processed per trace by band-limiting to the expected pulse rate range, whitening to uncorrelate the traces by linear transformation, shifting to zero mean and normalizing to unit variance to simplify the separation. The three processed traces are fed as input signals to cICA optimization as described in Eqs. (1)–(3). The constrained optimization problem is solved using Newton-like learning⁷ to incorporate prior information and additional requirements in the form of constraints. The reference signal fed to the cICA algorithm is a single tone with frequency corresponding to a possible heart rate in beats per minute (bpm). The peak frequency in the spectrum of the cICA output is compared to the frequency of the reference signal and the error is recorded as the absolute difference in frequencies.

Fig. 1

Functional block scheme of proposed algorithm.

This process is repeated while sweeping through frequencies of the reference signal within the expected range of the corresponding pulse. The reference frequency which results in the minimal absolute error is selected as the estimated pulse rate.

3.

Comparative Study

Forty-five participants between the ages of 18 to 45 years and from various nationalities were enrolled in a study. The study was approved by the Internal Review Committee for Protecting Human Subjects at the Rochester Institute of Technology, and a corresponding consent form was signed by participants. One minute video recordings were taken using an off-the-shelf, 2 MP, RGB Logitech camera at 15 frames per second with pixel resolution of $320\times 240$ saved in WMV/AVI format. During the video recording, participants were asked to maintain a relaxed sitting position. At the same time a reference pulse rate was measured using a BVP finger probe oximeter. All videos were processed offline using MATLAB.

An automatic face tracker was used to detect the faces and select the RoI within each video frame. We used the face detection toolbox (version 0.21) using local binary patterns and haar features.⁹ The face detection code was mainly written in C and wrapped with a MATLAB interface. The RoI was then separated into three RGB channels by averaging all the pixels in a frame to obtain a single red, blue, and green point for each video frame and form the raw traces. Each video resulted in a block of three vectors, where each vector has 900 numbers. The raw traces were whitened, shifted to zero mean, normalized to unit variance and band-pass filtered within the range of 0.75 to 4 Hz, under the assumption that the pulse lies within 45 to 240 bpm. The filtered traces were fed to the cICA algorithm with reference signal sweep as previously described to extract the BVP with a sweep resolution of 0.5 bpm. The peak frequency was extracted using the MATLAB periodogram function with a hamming window for power spectral density estimation. All video streams were also subjected to the ICA pulse rate measurement algorithms described in Refs. 3 and 4 to form a basis for comparison with state-of-the-art algorithms.

4.

Results

We start by analyzing a representative video. Figure 2 depicts the error as a function of the reference signal frequency in bpm. Note that there is a single frequency (very close to the true pulse of 84 bpm) for which the error is very close to zero and that the error increases linearly as we move away from the true pulse. We observed the same behavior for all other videos. This implies that the algorithm does not suffer from ambiguities associated with convergence to local minima. Figure 3 presents the cICA output signal when the minimum error is achieved, and a PPG signal obtained via the MP36 Biopac System with a sensor attached to the earlobe. The close resemblance between the signals is evident. The cICA reference signal is presented as well.

Fig. 2

Error vs. frequency sweep in beats per minute (bpm).

Fig. 3

Photoplethysmography (PPG) sensor signal compared to constrained independent component analysis (cICA) output signal.

We now demonstrate the sorting problem of the ICA based approach and how it is resolved using our cICA approach. Table 1 presents pulse detection results along with the heart rate measured by the finger probe oximeter (true pulse) for a representative set of 10 videos. For the ICA approach the detected pulse from each ICA component is presented. For each video a different component provided a result closer to the true pulse. In most videos, the cICA algorithm provided a result similar in accuracy to the best result across all ICA components. For videos 8–10, cICA performed better than all ICA components.

Table 1

Pulse rate samples from data in bpm.

Figure 4 summarizes results for the entire data set using Bland-Altman plots for the ICA based algorithms depicted in Refs. 3 and 4, including the impractical scenario where the ICA component providing a pulse closest to the true pulse is chosen, and the proposed cICA algorithm. Comparing the ICA algorithms of Refs. 3 and 4 [Fig. 4(a)] with the impractical a-posterior best IC selection [Fig. 4(b)], it is clear that truly solving the sorting problem provides much better results. Comparing Fig. 4(b) with the proposed cICA algorithm [Fig. 4(c)], it is evident that the cICA algorithm furthers estimation accuracy beyond solving the sorting problem. The root mean square error (RMSE) for the ICA algorithm across the set is 20.6 bpm when selecting the second IC³ and 9.5 bpm when selecting the IC with maximum spectral peak.⁴ The RMSE for the proposed cICA algorithm is 3.5 bpm. Such inaccuracy is within the expected margin of error of the true pulse reference measurement, implying that the proposed algorithm provides measurement accuracy similar to the BVP finger probe oximeter.

Fig. 4

Bland-Altman plots: (a) independent component analysis (ICA) algorithms, (b) best component of ICA, (c) constrained ICA (cICA) algorithm.

We observed that motion of subjects during video capture affects the accuracy of all tested algorithms for pulse rate measurement. Face tracking was efficient in locating the face within a RoI selected for processing; however, the positioning of the face within the RoI would alter slightly causing jitter in the processed RGB traces. We were able to compensate for this effect by reducing the rate at which face tracking is performed. Since subjects were asked to sit still, the effect of such motion artifacts on presented results is minimal. In a real-world application, nonobtrusive pulse rate measurement algorithms should be augmented with existing methods of motion artifacts cancellation.

The cICA approach requires more processing time than the ICA approach. This is due to the sweep performed over the reference signal. We compared the runtime between the two algorithms and found that the cICA algorithm takes 30 times longer to run. Recall that our sweep was defined with intervals of 0.5 bpm. The processing time ratio is reduced to 15 if the sweep is performed using an interval of 1 bpm. Implementing the algorithms in a different programming language such as C would most likely reduce the processing time ratio, since iterations would run faster compared to MATLAB. Since pulse measurement is not expected to be performed continuously, we do not expect processing time to be a limited factor in practical implementations.

5.

Conclusion

A cICA algorithm for nonobtrusive pulse rate measurement using a webcam was proposed and evaluated using a comparative study with state-of-the-art algorithms over a large data set. The results demonstrated how the proposed algorithm extracts a PPG-equivalent signal and how it resolves the ICA sorting problem. The estimation error of the proposed algorithm was shown within the estimation error of a commercially available finger-probe oximeter, suggesting nonobtrusive pulse rate measurements could replace existing practices. Although the focus of this contribution was on pulse rate measurement, the proposed cICA algorithm is likely useful for estimating other physiological parameters where an ICA sorting problem is evident and prior knowledge exists on the parameter being estimated.