Ioannis T. Pavlidis

Contact-Free Measurement of Cardiac Pulse Based on the Analysis of Thermal Imagery

We have developed a novel method to measure human cardiac pulse at a distance. It is based on the information contained in the thermal signal emitted from major superficial vessels. This signal is acquired through a highly sensitive thermal imaging system. Temperature on the vessel is modulated by pulsative blood flow. To compute the frequency of modulation (pulse), we extract a line-based region along the vessel. Then, we apply fast Fourier transform (FFT) to individual points along this line of interest to capitalize on the pulses thermal propagation effect. Finally, we use an adaptive estimation function on the average FFT outcome to quantify the pulse. We have carried out experiments on a data set of 34 subjects and compared the pulse computed from our thermal signal analysis method to concomitant ground-truth measurements obtained through a standard contact sensor (piezo-electric transducer). The performance of the new method ranges from 88.52% to 90.33% depending on the clarity of the vessels thermal imprint. To the best of our knowledge, it is the first time that cardiac pulse has been measured several feet away from a subject with passive means.

Urban surveillance systems: from the laboratory to the commercial world

Research in the surveillance domain was confined for years in the military domain. Recently, as military spending for this kind of research was reduced and the technology matured, the attention of the research and development community turned to commercial applications of surveillance. In this paper we describe a state-of-the-art monitoring system developed by a corporate R&D lab in cooperation with the corresponding security business units. It represents a sizable effort to transfer some of the best results produced by computer vision research into a viable commercial product. Our description spans both practical and technical issues. From the practical point of view we analyze the state of the commercial security market, typical cultural differences between the research team and the business team and the perspective of the potential users of the technology. These are important issues that have to be dealt with or the surveillance technology will remain in the lab for a long time. From the technical point of view we analyze our algorithmic and implementation choices. We describe the improvements we introduced to the original algorithms reported in the literature in response to some problems that arose during field testing. We also provide extensive experimental results that highlight the strong points and some weaknesses of the prototype system.

Physiology-Based Face Recognition in the Thermal Infrared Spectrum

The current dominant approaches to face recognition rely on facial characteristics that are on or over the skin. Some of these characteristics have low permanency can be altered, and their phenomenology varies significantly with environmental factors (e.g., lighting). Many methodologies have been developed to address these problems to various degrees. However, the current framework of face recognition research has a potential weakness due to its very nature. We present a novel framework for face recognition based on physiological information. The motivation behind this effort is to capitalize on the permanency of innate characteristics that are under the skin. To establish feasibility, we propose a specific methodology to capture facial physiological patterns using the bioheat information contained in thermal imagery. First, the algorithm delineates the human face from the background using the Bayesian framework. Then, it localizes the superficial blood vessel network using image morphology. The extracted vascular network produces contour shapes that are characteristic to each individual. The branching points of the skeletonized vascular network are referred to as thermal minutia points (TMPs) and constitute the feature database. To render the method robust to facial pose variations, we collect for each subject to be stored in the database five different pose images (center, midleft profile, left profile, midright profile, and right profile). During the classification stage, the algorithm first estimates the pose of the test image. Then, it matches the local and global TMP structures extracted from the test image with those of the corresponding pose images in the database. We have conducted experiments on a multipose database of thermal facial images collected in our laboratory, as well as on the time-gap database of the University of Notre Dame. The good experimental results show that the proposed methodology has merit, especially with respect to the problem of low permanence over time. More importantly, the results demonstrate the feasibility of the physiological framework in face recognition and open the way for further methodological and experimental research in the area

Face recognition by fusing thermal infrared and visible imagery

Thermal infrared (IR) imagery offers a promising alternative to visible imagery for face recognition due to its relative insensitive to variations in face appearance caused by illumination changes. Despite its advantages, however, thermal IR has several limitations including that it is opaque to glass. The focus of this study is on the sensitivity of thermal IR imagery to facial occlusions caused by eyeglasses. Specifically, our experimental results illustrate that recognition performance in the IR spectrum degrades seriously when eyeglasses are present in the probe image but not in the gallery image and vice versa. To address this serious limitation of IR, we propose fusing IR with visible imagery. Since IR and visible imagery capture intrinsically different characteristics of the observed faces, intuitively, a better face description could be found by utilizing the complimentary information present in the two spectra. Two different fusion schemes have been investigated in this study. The first one is pixelbased and operates in the wavelet domain, while the second one is feature-based and operates in the eigenspace domain. In both cases, we employ a simple and general framework based on Genetic Algorithms (GAs) to find an optimum fusion strategy. We have evaluated our approaches through extensive experiments using the Equinox face database and the eigenface recognition methodology. Our results illustrate significant performance improvements in recognition, suggesting that IR and visible fusion is a viable approach that deserves further consideration. q 2006 Elsevier B.V. All rights reserved.

NEAT-o-Games: blending physical activity and fun in the daily routine

This article describes research that aims to encourage physical activity through a novel pervasive gaming paradigm. Data from a wearable accelerometer are logged wirelessly to a cell phone and control the animation of an avatar that represents the player in a virtual race game with other players over the cellular network. Winners are declared every day and players with an excess of activity points can spend some to get hints in mental games of the suite, like Sudoku. The racing game runs in the background throughout the day and every little move counts. As the gaming platform is embedded in the daily routine of players, it may act as a strong behavioral modifier and increase everyday physical activity other than volitional sporting exercise. Such physical activity (e.g., taking the stairs), is termed NEAT and was shown to play a major role in obesity prevention and intervention. A pilot experiment demonstrates that players are engaged in NEAT-o-Games and become more physically active while having a good dosage of fun.

Thermal image analysis for polygraph testing

We have designed, developed, and tested a very promising thermal image analysis method for polygraph testing. The method achieved a correct classification rate of CCR= 84% on the test population to our avail. This method, once refined, can serve as an additional channel for increasing the reliability and accuracy of traditional polygraph examination. We extract subtle facial temperature fluctuation patterns through nonlinear heat transfer modeling. The modeling transforms raw thermal data to blood flow rate information. Then, we use the slope of the average periorbital blood flow rate as the feature of a binary classification scheme. The results come to support our previous laboratory findings about the importance of periorbital blood flow in anxious states.

Human behaviour: Seeing through the face of deception

We have developed a high-definition thermal-imaging technique that can detect attempted deceit by recording the thermal patterns from peoples faces. This technique has an accuracy comparable to that of polygraph examination by experts and has potential for application in remote and rapid security screening, without the need for skilled staff or physical contact.

Imaging Facial Physiology for the Detection of Deceit

Previous work has demonstrated the correlation of increased blood perfusion in the orbital muscles and stress levels for human beings. It has also been suggested that this periorbital perfusion can be quantified through the processing of thermal video. The idea has been based on the fact that skin temperature is heavily modulated by superficial blood flow. Proof of this concept was established for two different types of stress inducing experiments: startle experiments and mock-crime polygraph interrogations. However, the polygraph interrogation scenarios were simplistic and highly constrained. In the present paper, we report results derived from a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. Additionally, we propose a new methodology to compute the mean periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem CONDENSATION tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (Deceptive) from non-stressful (Non-Deceptive) subjects based on a comparative measure between the entire interrogation signal (baseline) and a critical subsection of it (transient response). The successful classification rate is 87.2% for 39 subjects. This is on par with the success rate achieved by highly trained psycho-physiological experts and opens the way for automating lie detection in realistic settings.

The imaging issue in an automatic face/disguise detection system

Automatic face recognition systems have made great strides. They still, however cannot cope with changes due to lighting and cannot detect disguises, Both of these issues are critical for the employment of face recognition systems in high security applications, such as embassy perimeters, federal plazas, mid the like. We propose novel imaging solutions that address these difficult problems. We demonstrate with theoretical and experimental arguments that a dual-band fusion system in the near infrared can segment human faces much more accurately than traditional visible band face detection systems. Face detection is useful by itself as an early warning method in certain surveillance applications. Accurate face delineation can also improve the performance of face recognition systems in certain difficult scenarios, particularly in outside environments. We also demonstrate with theoretical and experimental arguments that the upper band of the near infrared (1.4-2.4 /spl mu/m) is particularly advantageous for disguise detection purposes. This is attributable to the unique and universal properties of the human skin in this sub-band. We conclude the paper with a description of our ongoing and future efforts.

Interacting with human physiology

We propose a novel system that incorporates physiological monitoring as part of the human-computer interface. The sensing element is a thermal camera that is employed as a computer peripheral. Through bioheat modeling of facial imagery almost the full range of vital signs can be extracted, including localize blood flow, cardiac pulse, and breath rate. This physiological information can then be used to draw inferences about a variety of health symptoms and psychological states. Our research aims to realize the notion of desktop health monitoring and create truly collaborative interactions in which humans and machines are both observing and responding.