Jonathan Dowdall

Imaging Facial Physiology for the Detection of Deceit

Previous work has demonstrated the correlation of increased blood perfusion in the orbital muscles and stress levels for human beings. It has also been suggested that this periorbital perfusion can be quantified through the processing of thermal video. The idea has been based on the fact that skin temperature is heavily modulated by superficial blood flow. Proof of this concept was established for two different types of stress inducing experiments: startle experiments and mock-crime polygraph interrogations. However, the polygraph interrogation scenarios were simplistic and highly constrained. In the present paper, we report results derived from a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. Additionally, we propose a new methodology to compute the mean periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem CONDENSATION tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (Deceptive) from non-stressful (Non-Deceptive) subjects based on a comparative measure between the entire interrogation signal (baseline) and a critical subsection of it (transient response). The successful classification rate is 87.2% for 39 subjects. This is on par with the success rate achieved by highly trained psycho-physiological experts and opens the way for automating lie detection in realistic settings.

Interacting with human physiology

We propose a novel system that incorporates physiological monitoring as part of the human-computer interface. The sensing element is a thermal camera that is employed as a computer peripheral. Through bioheat modeling of facial imagery almost the full range of vital signs can be extracted, including localize blood flow, cardiac pulse, and breath rate. This physiological information can then be used to draw inferences about a variety of health symptoms and psychological states. Our research aims to realize the notion of desktop health monitoring and create truly collaborative interactions in which humans and machines are both observing and responding.

Face Detection in the Near-IR Spectrum

Abstract Face detection is an important prerequisite step for successful face recognition. The performance of previous face detection methods reported in the literature is far from perfect and deteriorates ungracefully where lighting conditions cannot be controlled. We propose a method that outperforms state-of-the-art face detection methods in environments with stable lighting. In addition, our method can potentially perform well in environments with variable lighting conditions. The approach capitalizes upon our near-IR skin detection method reported elsewhere [Proceedings IEEE Workshop on Computer Vision beyond the Visible Spectrum: Methods and Applications; 2000, IEEE Trans. Int. Trans. Sys.; vol. 1; 72–85]. It ascertains the existence of a face within the skin region by finding the eyes and eyebrows. The eye–eyebrow pairs are determined by extracting appropriate features from multiple near-IR bands. Very successful feature extraction is achieved by simple algorithmic means like integral projections and template matching. This is because processing is constrained in the skin region and aided by the near-IR phenomenology. The effectiveness of our method is substantiated by comparative experimental results with the Identix face detector [ http://www.faceit.com ].

Coalitional tracking

We propose a novel tracking method that uses a network of independent particle filter trackers whose interactions are modeled using coalitional game theory. Our tracking method is general, it maintains pixel level accuracy, and can negotiate surface deformations and occlusions. We tested our method on a substantial video set featuring non-trivial motion from over 40 objects in both the infrared and visual spectra. The coalitional tracker demonstrated fault tolerant behavior that exceeds by far the performance of single particle filter trackers. Our method represents a shift from the typical tracking paradigms and may find application in demanding imaging problems across the electromagnetic spectrum.

Face detection in the near-IR spectrum

Face detection is an important prerequisite step for successful face recognition. Face detection methods reported in the literature are far from perfect and deteriorate ungracefully where lighting conditions cannot be controlled. We propose a method that could potentially outperform state-of-the-art face detection methods in environments with dynamic lighting conditions. The approach capitalizes upon our near-IR skin and face detection methods reported elsewhere. It ascertains the existence of a face within a skin region by finding the eyes and eyebrows. The eye-eyebrow pairs are determined by extracting appropriate features from multiple near-IR bands. In this paper we introduce a novel feature extraction method we call dynamic integral projection. The method is relatively simple but highly effective because the processing is constrained within the skin region and aided by the near-IR phenomenology.

Automatic thermal monitoring system (ATHEMOS) for deception detection

Previous work has demonstrated the correlation of periorbital perfusion and stress levels in human beings. In this paper, we report results on a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. We propose a new methodology to compute the average periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem condensation tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (deceptive) from non-stressful (non-deceptive) subjects based on a comparative measure between the interrogation signal (baseline) and portions thereof (transient response).

Coalitional Tracking in Facial Infrared Imaging and Beyond

We propose a novel facial tracking method that uses a distributed network of individual trackers whose interactions are modeled using coalitional game theory. Our tracking method maintains a high level of accuracy and can negotiate surface deformations and occlusions. We tested the method on a substantial video set featuring non-trivial face motion from over 40 subjects in both the infrared and visual spectra. The coalitional tracker demonstrated fault tolerant behavior that exceeds by far the performance of single condensation trackers. Our method represents a shift from the typical tracking paradigms and may find broader application in demanding imaging problems across the electromagnetic spectrum.

Coalitional Tracker for Deception Detection in Thermal Imagery

We propose a novel tracking method that uses a network of independent particle filter trackers whose interactions are modeled using coalitional game theory. Our tracking method is general; it maintains pixel-level accuracy, and can negotiate surface deformations and occlusions. We tested our method in a substantial video set featuring nontrivial motion from over 40 objects in both the infrared and vi sual spectra. The coalitional tracker demonstrated fault-tolerant behavior that far exceeds the performance of single-particle filter trackers. Our method represents a shift from the typical tracking paradigms and may find application in demanding imaging problems across the electromagnetic spectrum.

Thermal image analysis for detecting facemask leakage

Due to the modern advent of near ubiquitous accessibility to rapid international transportation the epidemiologic trends of highly communicable diseases can be devastating. With the recent emergence of diseases matching this pattern, such as Severe Acute Respiratory Syndrome (SARS), an area of overt concern has been the transmission of infection through respiratory droplets. Approved facemasks are typically effective physical barriers for preventing the spread of viruses through droplets, but breaches in a mask’s integrity can lead to an elevated risk of exposure and subsequent infection. Quality control mechanisms in place during the manufacturing process insure that masks are defect free when leaving the factory, but there remains little to detect damage caused by transportation or during usage. A system that could monitor masks in real-time while they were in use would facilitate a more secure environment for treatment and screening. To fulfill this necessity, we have devised a touchless method to detect mask breaches in real-time by utilizing the emissive properties of the mask in the thermal infrared spectrum. Specifically, we use a specialized thermal imaging system to detect minute air leakage in masks based on the principles of heat transfer and thermodynamics. The advantage of this passive modality is that thermal imaging does not require contact with the subject and can provide instant visualization and analysis. These capabilities can prove invaluable for protecting personnel in scenarios with elevated levels of transmission risk such as hospital clinics, border check points, and airports.