Benjamin S. Riggan

A Polarimetric Thermal Database for Face Recognition Research

We present a polarimetric thermal face database, the first of its kind, for face recognition research. This database was acquired using a polarimetric longwave infrared imager, specifically a division-of-time spinning achromatic retarder system. A corresponding set of visible spectrum imagery was also collected, to facilitate crossspectrum (also referred to as heterogeneous) face recognition research. The database consists of imagery acquired at three distances under two experimental conditions: neutral/baseline condition, and expressions condition. Annotations (spatial coordinates of key fiducial points) are provided for all images. Cross-spectrum face recognition performance on the database is benchmarked using three techniques: partial least squares, deep perceptual mapping, and coupled neural networks.

Optimal feature learning and discriminative framework for polarimetric thermal to visible face recognition

A face recognition system capable of day- and night-time operation is highly desirable for surveillance and reconnaissance. Polarimetric thermal imaging is ideal for such applications, as it acquires emitted radiation from skin tissue. However, polarimetric thermal facial imagery must be matched to visible face images for interoperability with existing biometric databases. This work proposes a novel framework for polarimetric thermal-to-visible face recognition, where polarimetric features are optimally combined to facilitate training of a discriminant classifier. We evaluate its performance on imagery collected under different expressions and at different ranges, and compare with recent deep perceptual mapping, coupled neural network, and partial least squares techniques for cross-spectrum face matching.

Coupled Auto-Associative Neural Networks for Heterogeneous Face Recognition

Several models have been previously suggested for learning correlated representations between source and target modalities. In this paper, we propose a novel coupled autoassociative neural network for learning a target-to-source image representation for heterogenous face recognition. This coupled network is unique, because a cross-modal transformation is learned by forcing the hidden units (latent features) of two neural networks to be as similar as possible, while simultaneously preserving information from the input. The effectiveness of this model is demonstrated using multiple existing heterogeneous face recognition databases. Moreover, the empirical results show that the learned image representation-common latent features-by the coupled auto-associative produces competitive cross-modal face recognition results. These results are obtained by training a softmax classifier using only the latent features from the source domain and testing using only the latent features from the target domain.

Estimation of visible spectrum faces from polarimetric thermal faces

A method for synthesizing visible spectrum face imagery from polarimetric-thermal face imagery is presented. This work extends recent within-spectrum (i.e., visible-to-visible) reconstruction techniques for image representation understanding using convolutional neural networks. Despite the challenging task, we effectively demonstrate the ability to produce a visible image from a probe polarimetric-thermal image. Moreover, we are able to demonstrate the same capability with conventional thermal imagery, but we report a significant improvement by incorporating polarization-state information. These reconstructions, or estimates, can be used to aid human examiners performing one-to-one verification of matches retrieved from automated cross-spectrum face recognition algorithms.

An accumulative fusion architecture for discriminating people and vehicles using acoustic and seismic signals

In this paper, we develop new multiclass classification algorithms for detecting people and vehicles by fusing data from a multimodal, unattended ground sensor node. The specific types of sensors that we apply in this work are acoustic and seismic sensors. We investigate two alternative approaches to multiclass classification in this context — the first is based on applying Dempster-Shafer Theory to perform score-level fusion, and the second involves the accumulation of local similarity evidences derived from a feature-level fusion model that combines both modalities. We experiment with the proposed algorithms using different datasets obtained from acoustic and seismic sensors in various outdoor environments, and evaluate the performance of the two algorithms in terms of receiver operating characteristic and classification accuracy. Our results demonstrate overall superiority of the proposed new feature-level fusion approach for multiclass discrimination among people, vehicles and noise.

Heterogeneous Face Recognition: Recent Advances in Infrared-to-Visible Matching

An emerging topic in face recognition is matching between facial images acquired from different sensing modalities, referred to as heterogeneous face recognition. Heterogeneous face recognition has the potential to provide key capabilities for the commercial sector as well as for law enforcement, intelligence gathering, and the military, especially in challenging unconstrained settings. However, the difficulty in heterogeneous face recognition is compounded by phenomenology differences between modalities, giving rise to significant facial appearance variations due to the modality gap. In this paper, we focus on a subset of heterogeneous face recognition and present a succinct review of recent work on infrared-to-visible face recognition.

An Order Preserving Bilinear Model for Person Detection in Multi-Modal Data

We propose a new order preserving bilinear framework that exploits low-resolution video for person detection in a multi-modal setting using deep neural networks. In this setting cameras are strategically placed such that less robust sensors, e.g. geophones that monitor seismic activity, are located within the field of views (FOVs) of cameras. The primary challenge is being able to leverage sufficient information from videos where there are less than 40 pixels on targets, while also taking advantage of less discriminative information from other modalities, e.g. seismic. Unlike state-of-the-art methods, our bilinear framework retains spatio-temporal order when computing the vector outer products between pairs of features. Despite the high dimensionality of these outer products, we demonstrate that our order preserving bilinear framework yields better performance than recent orderless bilinear models and alternative fusion methods. Code is available at https://github.com/oulutan/OP-Bilinear-Model

Overview of polarimetric thermal imaging for biometrics

This paper presents an overview of polarimetric thermal imaging for biometrics, focusing on face recognition, with a short discussion on fingerprints and iris. Face recognition has been and continues to be an active area of biometrics research, with most of the research dedicated to recognition in the visible spectrum. However, face recognition in the visible spectrum is not practical for discrete surveillance in low-light and nighttime scenarios. Polarimetric thermal imaging represents an ideal modality for acquiring the naturally emitted thermal radiation from the human face, providing additional geometric and textural details not available in conventional thermal imagery. One of the main challenges lies in matching the acquired polarimetric thermal facial signature to gallery databases containing only visible facial signature, for interoperability with existing government biometric repositories. This paper discusses approaches and algorithms to exploit polarization information, as represented by the Stokes vectors, through feature extraction and nonlinear regression to enable polarimetric thermal-to-visible face recognition. In addition to cross-spectrum feature based approaches, crossspectrum image synthesis methods are discussed that seek to reconstruct a visible-like image given a polarimetric thermal face image input. Beyond facial biometrics, this paper presents an initial exploration of polarimetric thermal imaging for latent fingerprint acquisition. Latent prints are formed when the oils and sweat from the finger are deposited onto another surface through contact, and are typically collected by first dusting with powder before being imaged and then lifted with adhesive tape. This paper presents polarimetric thermal imagery of latent prints from a nonporous glass surface, acquired without the dusting process. A brief discussion of the utility of polarimetric thermal imaging for iris recognition is also presented.