Andrea Selinger

A comparative analysis of face recognition performance with visible and thermal infrared imagery

We present a comprehensive performance analysis of multiple appearance-based face recognition methodologies, on visible and thermal infrared imagery. We compare algorithms within and between modalities in terms of recognition performance, false alarm rates and requirements to achieve specified performance levels. The effect of illumination conditions on recognition performance is emphasized, as it underlines the relative advantage of radiometrically calibrated thermal imagery for face recognition.

Face recognition with visible and thermal infrared imagery

We present a comprehensive performance study of multiple appearance-based face recognition methodologies, on visible and thermal infrared imagery. We compare algorithms within the same imaging modality as well as between them. Both identification and verification scenarios are considered, and appropriate performance statistics reported for each case. Our experimental design is aimed at gaining full understanding of algorithm performance under varying conditions, and is based on Monte Carlo analysis of performance measures. This analysis reveals that under many circumstances, using thermal infrared imagery yields higher performance, while in other cases performance in both modalities is equivalent. Performance increases further when algorithms on visible and thermal infrared imagery are fused. Our study also provides a partial explanation for the multiple contradictory claims in the literature regarding performance of various algorithms on visible data sets.

Thermal face recognition in an operational scenario

We present results on the latest advances in thermal infrared face recognition, and its use in combination with visible imagery. Previous research by the authors has shown high performance under very controlled conditions, or questionable performance under a wider range of conditions. This paper shows results on the use of thermal infrared and visible imagery for face recognition in operational scenarios. In particular, we show performance statistics for outdoor face recognition and recognition across multiple sessions. Our results support the conclusion that face recognition performance with thermal infrared imagery is stable over multiple sessions, and that fusion of modalities increases performance. As measured by the number of images and number of subjects, this is the largest ever reported study on thermal face recognition.

A cubist approach to object recognition

We describe an appearance-based object recognition system using a keyed, multi-level contest representation reminiscent of certain aspects of cubist art. Specifically, we utilize distinctive intermediate-level features in this case automatically extracted 2-D boundary fragments, as keys, which are then verified within a local contest, and assembled within a loose global contest to evoke an overall percept. This system demonstrates good recognition of a variety of 3-D shapes, ranging from sports cars and fighter planes to snakes and lizards with full orthographic invariance. We report the results of large-scale tests, involving over 2000 separate test images, that evaluate performance with increasing number of items in the database, in the presence of clutter, background change, and occlusion, and also the results of some generic classification experiments where the system is tested on objects never previously seen or modeled. To our knowledge, the results we report are the best in the literature for full-sphere tests of general shapes with occlusion and clutter resistance.

A Perceptual Grouping Hierarchy for Appearance-Based 3D Object Recognition

In this paper we consider the problem of 3D object recognition and the role that perceptual grouping processes must play. In particular, we argue that reliance on a single level of perceptual grouping is inadequate, since it is responsible for the specific weaknesses of several well-known recognition techniques. Instead, recognition must use a hierarchy of perceptual grouping processes. We describe an appearance-based system that uses four distinct levels of perceptual grouping to represent 3D objects in a form that allows not only recognition, but reasoning about 3D manipulation of a sort that has been supported in the past only by 3D geometric models. The results of the algorithms have been previously reported, and the main contribution of this paper is the development of the perceptual organization hierarchy.

Thermal face recognition over time

We present a comparative study of face recognition performance with visible and thermal infrared imagery, emphasizing the influence of time-lapse between enrollment and testing images. Most previous research in this area, with few exceptions, focused on results obtained when enrollment and testing images were acquired in the same session. We show that the performance difference between visible and thermal recognition in a time-lapse scenario is smaller than previously believed, and in fact is not statistically significant on existing data sets.

Appearance-based object recognition using multiple views

Object recognition from a single view fails when the available features are not sufficient to determine the identity of a single object, either because of similarity with another object or because of feature corruption due to clutter and occlusion. Active object recognition systems have addressed this problem successfully, but they require complicated systems with adjustable viewpoints that are not always available. In this paper we investigate the performance gain available by combining the results of a single view object recognition system applied to imagery obtained from multiple fixed cameras. In particular, we address performance in cluttered scenes with varying degrees of information about relative camera pose. We argue that a property common to many computer vision recognition systems, which we term a weak target error, is responsible for two interesting limitations of multi-view performance enhancement: the lack of significant improvement in systems whose single-view performance is weak, and the plateauing of performance improvement as additional multi-view constraints are added.

Large-scale tests of a keyed, appearance-based 3-D object recognition system

We describe and analyze an appearance-based 3-D object recognition system that avoids some of the problems of previous appearance-based schemes. We describe various large-scale performance tests and report good performance for full-sphere/hemisphere recognition of up to 24 complex, curved objects, robustness against clutter and occlusion, and some intriguing generic recognition behavior. We also establish a protocol that permits performance in the presence of quantifiable amounts of clutter and occlusion to be predicted on the basis of simple score statistics derived from clean test images and pure clutter images.

Face Recognition in the Dark

Previous research has established thermal infrared imagery of faces as a valid biometric and has shown high recognition performance in a wide range of scenarios. However, all these results have been obtained using eye locations that were either manually marked, or automatically detected in a coregistered visible image, making the realistic use of thermal infrared imagery alone impossible. In this paper we present the results of an eye detector on thermal infrared imagery and we analyze its impact on recognition performance. Our experiments show that although eyes cannot be detected as reliably in thermal images as in visible ones, some face recognition algorithms can still achieve adequate performance.

Improving appearance-based object recognition in cluttered backgrounds

Appearance-based object recognition systems are currently the most successful approach for dealing with 3D recognition of arbitrary objects in the presence of clutter and occlusion. However, no current system seems directly scalable to human performance levels in this domain. We describe a series of experiments on a previously described object recognition system that try to see, if any, which design axes of such systems hold the greatest potential for improving performance. We look at the potential effect of different design modifications, and conclude that the greatest leverage lies at the level of intermediate feature construction.