Mark J. Burge

Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A

Rapid progress in unconstrained face recognition has resulted in a saturation in recognition accuracy for current benchmark datasets. While important for early progress, a chief limitation in most benchmark datasets is the use of a commodity face detector to select face imagery. The implication of this strategy is restricted variations in face pose and other confounding factors. This paper introduces the IARPA Janus Benchmark A (IJB-A), a publicly available media in the wild dataset containing 500 subjects with manually localized face images. Key features of the IJB-A dataset are: (i) full pose variation, (ii) joint use for face recognition and face detection benchmarking, (iii) a mix of images and videos, (iv) wider geographic variation of subjects, (v) protocols supporting both open-set identification (1:N search) and verification (1:1 comparison), (vi) an optional protocol that allows modeling of gallery subjects, and (vii) ground truth eye and nose locations. The dataset has been developed using 1,501,267 million crowd sourced annotations. Baseline accuracies for both face detection and face recognition from commercial and open source algorithms demonstrate the challenge offered by this new unconstrained benchmark.

Handbook of Iris Recognition

The definitive work on iris recognition technology, this comprehensive handbook presents a broad overview of the state of the art in this exciting and rapidly evolving field. Revised and updated from the highly-successful original, this second edition has also been considerably expanded in scope and content, featuring four completely new chapters. Features: provides authoritative insights from an international selection of preeminent researchers from government, industry, and academia; reviews issues covering the full spectrum of the iris recognition process, from acquisition to encoding; presents surveys of topical areas, and discusses the frontiers of iris research, including cross-wavelength matching, iris template aging, and anti-spoofing; describes open source software for the iris recognition pipeline and datasets of iris images; includes new content on liveness detection, correcting off-angle iris images, subjects with eye conditions, and implementing software systems for iris recognition.

Face Recognition Performance: Role of Demographic Information

This paper studies the influence of demographics on the performance of face recognition algorithms. The recognition accuracies of six different face recognition algorithms (three commercial, two nontrainable, and one trainable) are computed on a large scale gallery that is partitioned so that each partition consists entirely of specific demographic cohorts. Eight total cohorts are isolated based on gender (male and female), race/ethnicity (Black, White, and Hispanic), and age group (18-30, 30-50, and 50-70 years old). Experimental results demonstrate that both commercial and the nontrainable algorithms consistently have lower matching accuracies on the same cohorts (females, Blacks, and age group 18-30) than the remaining cohorts within their demographic. Additional experiments investigate the impact of the demographic distribution in the training set on the performance of a trainable face recognition algorithm. We show that the matching accuracy for race/ethnicity and age cohorts can be improved by training exclusively on that specific cohort. Operationally, this leads to a scenario, called dynamic face matcher selection, where multiple face recognition algorithms (each trained on different demographic cohorts) are available for a biometric system operator to select based on the demographic information extracted from a probe image. This procedure should lead to improved face recognition accuracy in many intelligence and law enforcement face recognition scenarios. Finally, we show that an alternative to dynamic face matcher selection is to train face recognition algorithms on datasets that are evenly distributed across demographics, as this approach offers consistently high accuracy across all cohorts.

Principles of Digital Image Processing

This easy-to-follow textbook provides a modern, algorithmic introduction to digital image processing, designed to be used both by learners desiring a firm foundation on which to build, and practitioners in search of critical analysis and modern implementations of the most important techniques. It compiles the key elements of digital image processing, starting from the basic concepts and elementary properties of digital images through simple statistics and point operations, fundamental filtering techniques, localization of edges and contours, and basic operations on color images. This reader-friendly text concentrates on practical applications and working implementations, and presents the important formal details and mathematics necessary for a deeper understanding of the algorithms. Implementations are all based on Java and ImageJ. This concise yet comprehensive, reader-friendly text is ideal for undergraduates studying foundation courses as well as ideal for self-study.

Multispectral iris fusion for enhancement, interoperability, and cross wavelength matching

Traditionally, only a narrow band of the Near-Infrared (NIR) spectrum (700-900nm) is utilized for iris recognition since this alleviates any physical discomfort from illumination, reduces specular reflections and increases the amount of texture captured for some iris colors. However, previous research has shown that matching performance is not invariant to iris color and can be improved by imaging outside of the NIR spectrum. Building on this research, we demonstrate that iris texture increases with the frequency of the illumination for lighter colored sections of the iris and decreases for darker sections. Using registered visible light and NIR iris images captured using a single-lens multispectral camera, we illustrate how physiological properties of the iris (e.g., the amount and distribution of melanin) impact the transmission, absorbance, and reflectance of different portions of the electromagnetic spectrum and consequently affect the quality of the imaged iris texture. We introduce a novel iris code, Multispectral Enhanced irisCode (MEC), which uses pixel-level fusion algorithms to exploit texture variations elicited by illuminating the iris at different frequencies, to improve iris matcher performance and reduce Failure-To-Enroll (FTE) rates. Finally, we present a model for approximating an NIR iris image using features derived from the color and structure of a visible light iris image. The simulated NIR images generated by this model are designed to improve the interoperability between legacy NIR iris images and those acquired under visible light by enabling cross wavelength matching of NIR and visible light iris images.

Assessment of H.264 Video Compression on Automated Face Recognition Performance in Surveillance and Mobile Video Scenarios

We assess the impact of the H.264 video codec on the match performance of automated face recognition in surveillance and mobile video applications. A set of two hundred access control (90 pixel inter-pupilary distance) and distance surveillance (45 pixel inter-pupilary distance) videos taken under non-ideal imaging and facial recognition (e.g., pose, illumination, and expression) conditions were matched using two commercial face recognition engines in the studies. The first study evaluated automated face recognition performance on access control and distance surveillance videos at CIF and VGA resolutions using the H.264 baseline profile at nine bitrates rates ranging from 8kbs to 2048kbs. In our experiments, video signals were able to be compressed up to 128kbs before a significant drop face recognition performance occurred. The second study evaluated automated face recognition on mobile devices at QCIF, iPhone, and Android resolutions for each of the H.264 PDA profiles. Rank one match performance, cumulative match scores, and failure to enroll rates are reported.

The Discrete Cosine Transform DCT

The Fourier transform and the DFT are designed for processing complex-valued signals, and they always produce a complex-valued spectrum even in the case where the original signal was strictly realvalued. The reason is that neither the real nor the imaginary part of the Fourier spectrum alone is sufficient to represent (i.e., reconstruct) the signal completely. In other words, the corresponding cosine (for the real part) or sine functions (for the imaginary part) alone do not constitute a complete set of basis functions.

Multispectral Iris Fusion and Cross-Spectrum Matching

Traditionally, only a narrow band of the Near-Infrared (NIR) spectrum (700–900 nm) is utilized for iris recognition since this alleviates any physical discomfort from illumination, reduces specular reflections and increases the amount of texture captured for some iris colors. However, previous research has shown that matching performance is not invariant to iris color and can be improved by imaging outside of the NIR spectrum. Building on this research, we demonstrate that iris texture increases with the frequency of the illumination for lighter colored sections of the iris and decreases for darker sections. Using registered visible light and NIR iris images captured using a single-lens multispectral camera, we illustrate how physiological properties of the iris (e.g., the amount and distribution of melanin) impact the transmission, absorbance, and reflectance of different portions of the electromagnetic spectrum and consequently affect the quality of the imaged iris texture. We introduce a novel iris code, Multispectral Enhanced irisCode (MEC), which uses pixel-level fusion algorithms to exploit texture variations elicited by illuminating the iris at different frequencies to improve iris matcher performance and reduce Failure-To-Enroll (FTE) rates. Finally, we present a model for approximating an NIR iris image using features derived from the color and structure of a visible light iris image. The simulated NIR images generated by this model are designed to improve the interoperability between legacy NIR iris images and those acquired under visible light by enabling cross wavelength matching of NIR and visible light iris images.

Scale-Invariant Feature Transform (SIFT)

Many real applications require the localization of reference positions in one or more images, for example, for image alignment, removing distortions, object tracking, 3D reconstruction, etc. We have seen that corner points1 can be located quite reliably and independent of orientation. However, typical corner detectors only provide the position and strength of each candidate point, they do not provide any information about its characteristic or “identity” that could be used for matching. Another limitation is that most corner detectors only operate at a particular scale or resolution, since they are based on a rigid set of filters.

Edge-Preserving Smoothing Filters

Noise reduction in images is a common objective in image processing, not only for producing pleasing results for human viewing but also to facilitate easier extraction of meaningful information in subsequent steps, for example, in segmentation or feature detection. Simple smoothing filters, such as the Gaussian filter and the filters discussed in Chapter 3 of this volume effectively perform low-pass filtering and thus remove high-frequency noise. However, they also tend to suppress high-rate intensity variations that are part of the original signal, thereby destroying image structures that are visually important. The filters described in this chapter are “edge preserving” in the sense that they change their smoothing behavior adaptively depending upon the local image structure. In general, maximum smoothing is performed over “flat” (uniform) image regions, while smoothing is reduced near or across edge-like structures, typically characterized by high intensity gradients.