Alice J. O’Toole

The perception of face gender: the role of stimulus structure in recognition and classification.

The perception of face gender was examined in the context of extending “face space” models of human face representations to include the perceptual categories defined by male and female faces. We collected data on the recognizability, gender classifiability (reaction time to classify a face as male/female), attractiveness, and masculinity/femininity of individual male and female faces. Factor analyses applied separately to the data for male and female faces yielded the following results. First, for both male and female faces, the recognizability and gender classifiability of faces were independent—a result inconsistent with the hypothesis that both recognizability and gender classifiability depend on a face’s “distance” from the subcategory gender prototype. Instead, caricatured aspects of gender (femininity/masculinity ratings) related to the gender classifiability of the faces. Second, facial attractiveness related inversely to face recognizability for male, but not for female, faces—a result that resolves inconsistencies in previous studies. Third, attractiveness and femininity for female faces were nearly equivalent, but attractiveness and masculinity for male faces were not equivalent. Finally, we applied principal component analysis to the pixel-coded face images with the aim of extracting measures related to the gender classifiability and recognizability of individual faces. We incorporated these model-derived measures into the factor analysis with the human rating and performance measures.

The neural processing of familiar and unfamiliar faces: A review and synopsis

Familiar faces are represented with rich visual, semantic, and emotional codes that support nearly effortless perception and recognition of these faces. Unfamiliar faces pose a greater challenge to human perception and memory systems. The established behavioural disparities for familiar and unfamiliar faces undoubtedly stem from differences in the quality and nature of their underlying neural representations. In this review, our goal is to characterize what is known about the neural pathways that respond to familiar and unfamiliar faces using data from functional neuroimaging studies. We divide our presentation by type of familiarity (famous, personal, and visual familiarity) to consider the distinct neural underpinnings of each. We conclude with a description of a recent model of person information proposed by Gobbini and Haxby (2007) and a list of open questions and promising directions for future research.

Unaware Person Recognition From the Body When Face Identification Fails

How does one recognize a person when face identification fails? Here, we show that people rely on the body but are unaware of doing so. State-of-the-art face-recognition algorithms were used to select images of people with almost no useful identity information in the face. Recognition of the face alone in these cases was near chance level, but recognition of the person was accurate. Accuracy in identifying the person without the face was identical to that in identifying the whole person. Paradoxically, people reported relying heavily on facial features over noninternal face and body features in making their identity decisions. Eye movements indicated otherwise, with gaze duration and fixations shifting adaptively toward the body and away from the face when the body was a better indicator of identity than the face. This shift occurred with no cost to accuracy or response time. Human identity processing may be partially inaccessible to conscious awareness.

Recognizing People in Motion

Natural movements of the face and body, as well as voice, provide converging cues to a persons identity. To date, person recognition has been studied primarily with static images of faces. Face recognition, however, is part of a larger system, whose preeminent goal is to efficiently recognize dynamic familiar people in unconstrained environments. We present a comprehensive framework for understanding person recognition as it happens in the real world. In this framework, dynamic information plays the central role in binding multi-modal information from the face, body, and the voice to achieve robust and highly accurate recognition. The superior temporal sulcus (STS) integrates multisensory, dynamic information from the whole person for recognition, thereby complementing its role in social cognition.

The face typicality-recognizability relationship: Encoding or retrieval locus?

Using a crossover recognition memory testing paradigm, we tested whether the effects on face recognition of the memorability component of face typicality (Vokey & Read, 1992, 1995) are due primarily to the encoding process occurring during study or to the retrieval process occurring at test. At study, faces were either veridical in form or at moderate (Experiment 1) or extreme (Experiment 2) levels of caricature. The variable of degree of facial caricature at study was crossed with the degree of caricature at test. The primary contribution of increased memorability to increased hit rate was through increased distinctiveness at study. Increased distinctiveness at test contributed to substantial reductions in the false alarm rate, too. Signal detection analyses confirmed that the mirror effects obtained were primarily stimulus/memory-based, rather than decision-based. Contrary to the conclusion of Vokey and Read (1992), we found that increments in face memorability produced increments in face recognition that were due at least as much to enhanced encoding of studied faces as they were to increased rejection of distractor faces.

Competence Judgments Based on Facial Appearance Are Better Predictors of American Elections Than of Korean Elections

Competence judgments based on facial appearance predict election results in Western countries, which indicates that these inferences contribute to decisions with social and political consequence. Because trait inferences are less pronounced in Asian cultures, such competence judgments should predict Asian election results less accurately than they do Western elections. In the study reported here, we compared Koreans’ and Americans’ competence judgments from face-to-trait inferences for candidates in U.S. Senate and state gubernatorial elections and Korean Assembly elections. Perceived competence was a far better predictor of the outcomes of real elections held in the United States than of elections held in Korea. When deciding which of two candidates to vote for in hypothetical elections, however, Koreans and Americans both voted on the basis of perceived competence inferred from facial appearance. Combining actual and hypothetical election results, we conclude that for Koreans, competence judgments from face-to-trait inferences are critical in voting only when other information is unavailable. However, in the United States, such competence judgments are substantially important, even in the presence of other information.

An X Windows tool for synthesizing face images from eigenvectors

An X Windows software tool for the construction of faces with a weighted combination of eigenvectors is described. The eigenvectors were extracted from an autoassociative matrix that comprised 100 face images. The program input consists of eigenvectors and sets of weights that describe individual faces and combines these to create face images. The tool creates a panel of buttons that permits the display of individual eigenvectors and the display of an average face as well. Facilities for on-line changes to the intensity of individual eigenvectors can be used to change the appearance of a face. Previously, O’Toole, Abdi, Deffenbacher, and Bartlett (1991) have shown that the intensity of certain individual eigenvectors contains reliable information for determining the sex and race of the face.

Creating Body Shapes From Verbal Descriptions by Linking Similarity Spaces

Brief verbal descriptions of people’s bodies (e.g., “curvy,” “long-legged”) can elicit vivid mental images. The ease with which these mental images are created belies the complexity of three-dimensional body shapes. We explored the relationship between body shapes and body descriptions and showed that a small number of words can be used to generate categorically accurate representations of three-dimensional bodies. The dimensions of body-shape variation that emerged in a language-based similarity space were related to major dimensions of variation computed directly from three-dimensional laser scans of 2,094 bodies. This relationship allowed us to generate three-dimensional models of people in the shape space using only their coordinates on analogous dimensions in the language-based description space. Human descriptions of photographed bodies and their corresponding models matched closely. The natural mapping between the spaces illustrates the role of language as a concise code for body shape that captures perceptually salient global and local body features.

Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms

Significance This study measures face identification accuracy for an international group of professional forensic facial examiners working under circumstances that apply in real world casework. Examiners and other human face “specialists,” including forensically trained facial reviewers and untrained superrecognizers, were more accurate than the control groups on a challenging test of face identification. Therefore, specialists are the best available human solution to the problem of face identification. We present data comparing state-of-the-art face recognition technology with the best human face identifiers. The best machine performed in the range of the best humans: professional facial examiners. However, optimal face identification was achieved only when humans and machines worked in collaboration. Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.

Human Factors in Forensic Face Identification

Facial identification by forensic examiners is a core component of criminal investigations and convictions. These identifications are often done in challenging circumstances that require experts to match identity across images and videos taken at a various distances, under different illumination conditions, and across a wide range of poses. Until recently, laboratory studies of human face identification have concentrated, almost exclusively, on face identification by untrained (naive) observers, with only a handful of studies focusing directly on the accuracy of expert forensic facial examiners. Over the last two decades, DNA-based exonerations of convicted criminals in the United States have revealed weaknesses in the forensic identification process due to human factors. In this paper, we review and analyze the factors known to impact facial identification accuracy for both naive participants and trained experts. Combined, these studies point to a set of challenges that impact accuracy for both groups of participants. They also lead to an understanding of the specific conditions under which forensic facial examiners can surpass naive observers at the task of face identification. Finally, we consider the role that computer-based face recognition systems can play in the future of forensic facial identification. These systems have made remarkable strides in recent years, raising new questions about how human and machine strengths at face identification can be combined to achieve optimum accuracy.