Lacey Best-Rowden

Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection

As face recognition applications progress from constrained sensing and cooperative subjects scenarios (e.g., drivers license and passport photos) to unconstrained scenarios with uncooperative subjects (e.g., video surveillance), new challenges are encountered. These challenges are due to variations in ambient illumination, image resolution, background clutter, facial pose, expression, and occlusion. In forensic investigations where the goal is to identify a person of interest, often based on low quality face images and videos, we need to utilize whatever source of information is available about the person. This could include one or more video tracks, multiple still images captured by bystanders (using, for example, their mobile phones), 3-D face models constructed from image(s) and video(s), and verbal descriptions of the subject provided by witnesses. These verbal descriptions can be used to generate a face sketch and provide ancillary information about the person of interest (e.g., gender, race, and age). While traditional face matching methods generally take a single media (i.e., a still face image, video track, or face sketch) as input, this paper considers using the entire gamut of media as a probe to generate a single candidate list for the person of interest. We show that the proposed approach boosts the likelihood of correctly identifying the person of interest through the use of different fusion schemes, 3-D face models, and incorporation of quality measures for fusion and video frame selection.

Video-to-video face matching: Establishing a baseline for unconstrained face recognition

Face recognition in video is becoming increasingly important due to the abundance of video data captured by surveillance cameras, mobile devices, Internet uploads, and other sources. Given the aggregate of facial information contained in a video (i.e., a sequence of face images or frames), video-based face recognition solutions can potentially alleviate classic challenges caused by variations in pose, illumination, and expression. However, with this increased focus on the development of algorithms specifically crafted for video-based face recognition, it is important to establish a baseline for the accuracy using state-of-the-art still image matchers. Note that most commercial-off-the-shelf (COTS) offerings are still limited to single frame matching. In order to measure the accuracy of COTS face recognition systems on video data, we first investigate the effectiveness of multi-frame score-level fusion and analyze the consistency across three COTS face matchers. We demonstrate that all three COTS matchers individually are superior to previously published face recognition results on the unconstrained YouTube Faces database. Further, fusion of scores from the three COTS matchers achieves a 20% improvement in accuracy over previously published results. We encourage the use of these results as a competitive baseline for video-to-video face matching on the YouTube Faces database.

Unconstrained face recognition: Establishing baseline human performance via crowdsourcing

Research focus in face recognition has shifted towards recognition of faces “in the wild” for both still images and videos which are captured in unconstrained imaging environments and without user cooperation. Due to confounding factors of pose, illumination, and expression, as well as occlusion and low resolution, current face recognition systems deployed in forensic and security applications operate in a semi-automatic manner; an operator typically reviews the top results from the face recognition system to manually determine the final match. For this reason, it is important to analyze the accuracies achieved by both the matching algorithms (machines) and humans on unconstrained face recognition tasks. In this paper, we report human accuracy on unconstrained faces in still images and videos via crowd-sourcing on Amazon Mechanical Turk. In particular, we report the first human performance on the YouTube Faces database and show that humans are superior to machines, especially when videos contain contextual cues in addition to the face image. We investigate the accuracy of humans from two different countries (United States and India) and find that humans from the United States are more accurate, possibly due to their familiarity with the faces of the public figures in the YouTube Faces database. A fusion of recognitions made by humans and a commercial-off-the-shelf face matcher improves performance over humans alone.

Fingerprint Recognition of Young Children

In 1899, Galton first captured ink-on-paper fingerprints of a single child from birth until the age of 4.5 years, manually compared the prints, and concluded that “the print of a child at the age of 2.5 years would serve to identify him ever after.” Since then, ink-on-paper fingerprinting and manual comparison methods have been superseded by digital capture and automatic fingerprint comparison techniques, but only a few feasibility studies on child fingerprint recognition have been conducted. Here, we present the first systematic and rigorous longitudinal study that addresses the following questions: 1) Do fingerprints of young children possess the salient features required to uniquely recognize a child? 2) If so, at what age can a child’s fingerprints be captured with sufficient fidelity for recognition? 3) Can a child’s fingerprints be used to reliably recognize the child as he ages? For this paper, we collected fingerprints of 309 children (0–5 years old) four different times over a one year period. We show, for the first time, that fingerprints acquired from a child as young as 6-h old exhibit distinguishing features necessary for recognition, and that state-of-the-art fingerprint technology achieves high recognition accuracy (98.9% true accept rate at 0.1% false accept rate) for children older than six months. In addition, we use mixed-effects statistical models to study the persistence of child fingerprint recognition accuracy and show that the recognition accuracy is not significantly affected over the one year time lapse in our data. Given rapidly growing requirements to recognize children for vaccination tracking, delivery of supplementary food, and national identification documents, this paper demonstrates that fingerprint recognition of young children (six months and older) is a viable solution based on available capture and recognition technology.

Giving Infants an Identity: Fingerprint Sensing and Recognition

There is a growing demand for biometrics-based recognition of children for a number of applications, particularly in developing countries where children do not have any form of identification. These applications include tracking child vaccination schedules, identifying missing children, preventing fraud in food subsidies, and preventing newborn baby swaps in hospitals. Our objective is to develop a fingerprint-based identification system for infants (age range: 0-12 months)1. Our ongoing research has addressed the following issues: (i) design of a compact, comfortable, high-resolution (>1,000 ppi) fingerprint reader; (ii) image enhancement algorithms to improve quality of infant fingerprint images; and (iii) collection of longitudinal infant fingerprint data to evaluate identification accuracy over time. This collaboration between Michigan State University, Dayalbagh Educational Institute, Saran Ashram Hospital, Agra, India and NEC Corporation, has demonstrated the feasibility of recognizing infants older than 4 weeks using fingerprints.

A longitudinal study of automatic face recognition

With the deployment of automatic face recognition systems for many large-scale applications, it is crucial that we gain a thorough understanding of how facial aging affects the recognition performance, particularly across a large population. Because aging is a complex process involving genetic and environmental factors, some faces “age well” while the appearance of others changes drastically over time. This heterogeneity (inter-subject variability) suggests the need for a subject-specific aging analysis. In this paper, we conduct such an analysis using a longitudinal database of 147,784 operational mug shots of 18,007 repeat criminal offenders, where each subject has at least five face images acquired over a minimum of five years. By fitting multilevel statistical models to genuine similarity scores from two commercial-off-the-shelf (COTS) matchers, we quantify (i) the population average rate of change in genuine scores with respect to the elapsed time between two face images, and (ii) how closely the subject-specific rates of change follow the population average. Longitudinal analysis of the scores from the more accurate COTS matcher shows that despite decreasing genuine scores over time, the average subject can still be correctly verified at a false accept rate (FAR) of 0.01% across all 16 years of elapsed time in our database. We also investigate (i) the effects of several other covariates (gender, race, face quality), and (ii) the probability of true acceptance over time.

Face Recognition Performance under Aging

With the integration of face recognition technology into important identity applications, it is imperative that the effects of facial aging on face recognition performance are thoroughly understood. As face recognition systems evolve and improve, they should be periodically re-evaluated on large-scale longitudinal face datasets. In our study, we evaluate the performance of two state-of-the-art commercial off the shelf (COTS) face recognition systems on two large-scale longitudinal datasets of mugshots of repeat offenders. The largest of these two datasets has 147,784 images of 18,007 subjects with an average of 8 images per subject over an average time span of 8.5 years. We fit multilevel statistical models to genuine comparison scores (similarity between images of the same face) from the two COTS face matchers. This allows us to analyze the degradation in recognition performance due to elapsed time between a probe (query) and its enrollment (gallery) image. We account for face image quality to obtain a better estimate of trends due to aging, and analyze whether longitudinal trends in genuine scores differ by subject gender and race. Based on the results of our statistical model, we infer that the state-of-the-art COTS matchers can verify 99% of the subjects at a false accept rate (FAR) of 0.01% for up to 10.5 and 8.5 years of elapsed time. Beyond this time lapse of 8.5 years, there is a significant loss in face recognition accuracy. This study extends and confirms the findings of earlier longitudinal studies on face recognition.

Automatic Face Recognition of Newborns, Infants, and Toddlers: A Longitudinal Evaluation

A number of emerging applications requiring reliable identification of children have called attention to whether biometric traits can be utilized as a solution. While biometric traits based on friction ridge patterns (e.g. fingerprints, footprints) have been evaluated to some extent, to our knowledge, no effort has been made to evaluate the efficacy of automatic face recognition of young children over useful durations of time. Additionally, there are some applications where only the face images of a child are available, such as identification of missing or abducted children and children shown in sexually exploitive media sequestered by law enforcement. In this paper, we introduce the Newborns, Infants, and Toddlers Longitudinal (NITL) face image database, which was collected by the authors during four different sessions over a period of one year (March 2015 to March 2016) at the Saran Ashram Hospital, Dayalbagh, India. The NITL database contains 314 subjects in total in the age range of 0 to 4 years old. The aim of this paper is to provide a comprehensive evaluation of a state-of-the-art commercial-off- the-shelf (COTS) face matcher on the NITL face image database to investigate the feasibility of face recognition of children as they age. Experimental results show that while available face recognition technology is not yet ready to reliably recognize very young children, face recognition enrolled at 3 years of age or older may be feasible.