Rob Jenkins

Variability in photos of the same face

Psychological studies of face recognition have typically ignored within-person variation in appearance, instead emphasising differences between individuals. Studies typically assume that a photograph adequately captures a persons appearance, and for that reason most studies use just one, or a small number of photos per person. Here we show that photographs are not consistent indicators of facial appearance because they are blind to within-person variability. Crucially, this within-person variability is often very large compared to the differences between people. To investigate variability in photos of the same face, we collected images from the internet to sample a realistic range for each individual. In Experiments 1 and 2, unfamiliar viewers perceived images of the same person as being different individuals, while familiar viewers perfectly identified the same photos. In Experiment 3, multiple photographs of any individual formed a continuum of good to bad likeness, which was highly sensitive to familiarity. Finally, in Experiment 4, we found that within-person variability exceeded between-person variability in attractiveness. These observations are critical to our understanding of face processing, because they suggest that a key component of face processing has been ignored. As well as its theoretical significance, this scale of variability has important practical implications. For example, our findings suggest that face photographs are unsuitable as proof of identity.

Robust representations for face recognition: the power of averages.

We are able to recognise familiar faces easily across large variations in image quality, though our ability to match unfamiliar faces is strikingly poor. Here we ask how the representation of a face changes as we become familiar with it. We use a simple image-averaging technique to derive abstract representations of known faces. Using Principal Components Analysis, we show that computational systems based on these averages consistently outperform systems based on collections of instances. Furthermore, the quality of the average improves as more images are used to derive it. These simulations are carried out with famous faces, over which we had no control of superficial image characteristics. We then present data from three experiments demonstrating that image averaging can also improve recognition by human observers. Finally, we describe how PCA on image averages appears to preserve identity-specific face information, while eliminating non-diagnostic pictorial information. We therefore suggest that this is a good candidate for a robust face representation.

Separate Coding of Different Gaze Directions in the Superior Temporal Sulcus and Inferior Parietal Lobule

Summary Electrophysiological recording in the anterior superior temporal sulcus (STS) of monkeys has demonstrated separate cell populations responsive to direct and averted gaze [1, 2]. Human functional imaging has demonstrated posterior STS activation in gaze processing, particularly in coding the intentions conveyed by gaze [3–6], but to date has provided no evidence of dissociable coding of different gaze directions. Because the spatial resolution typical of group-based fMRI studies (∼6–10 mm) exceeds the size of cellular patches sensitive to different facial characteristics (1–4 mm in monkeys), a more sensitive technique may be required. We therefore used fMRI adaptation, which is considered to offer superior resolution [7], to investigate whether the human anterior STS contains representations of different gaze directions, as suggested by non-human primate research. Subjects viewed probe faces gazing left, directly ahead, or right. Adapting to leftward gaze produced a reduction in BOLD response to left relative to right (and direct) gaze probes in the anterior STS and inferior parietal cortex; rightward gaze adaptation produced a corresponding reduction to right gaze probes. Consistent with these findings, averted gaze in the adapted direction was misidentified as direct. Our study provides the first human evidence of dissociable neural systems for left and right gaze.

I Thought You Were Looking at Me Direction-Specific Aftereffects in Gaze Perception

Gaze direction is an important social signal in humans and other primates. In this study, we used an adaptation paradigm to investigate the functional organization of gaze perception in humans. Adaptation to consistent leftward or rightward gaze produced a powerful illusion that virtually eliminated observers perception of gaze in the adapted direction; gaze to that side was seen as pointing straight ahead, though perception of gaze to the opposite side was unimpaired. This striking dissociation held even when retinotopic mapping between adaptation and test stimuli was disrupted by changes in size or head orientation, suggesting that our findings do not reflect adaptation to low-level visual properties. Moreover, adaptation to averted gaze did not affect judgments of line bisection, illustrating that our findings do not reflect a general spatial bias. Our findings provide evidence that humans have distinct populations of neurons that are selectively responsive to particular directions of seen gaze.

Faces retain attention

In the present study, we investigated whether faces have an advantage in retaining attention over other stimulus categories. In three experiments, subjects were asked to focus on a central go/no-go signal before classifying a concurrently presented peripheral line target. In Experiment 1, the go/no-go signal could be superimposed on photographs of upright famous faces, matching inverted faces, or meaningful objects. Experiments 2 and 3 tested upright and inverted unfamiliar faces, printed names, and another class of meaningful objects in an identical design. A fourth experiment provided a replication of Experiment 1, but with a 1,000-msec stimulus onset asynchrony between the onset of the central face/nonface stimuli and the peripheral targets. In all the experiments, the presence of an upright face significantly delayed target response times, in comparison with each of the other stimulus categories. These results suggest a general attentional bias, so that it is particularly difficult to disengage processing resources from faces.

100% Accuracy in Automatic Face Recognition

Accurate face recognition is critical for many security applications. Current automatic face-recognition systems are defeated by natural changes in lighting and pose, which often affect face images more profoundly than changes in identity. The only system that can reliably cope with such variability is a human observer who is familiar with the faces concerned. We modeled human familiarity by using image averaging to derive stable face representations from naturally varying photographs. This simple procedure increased the accuracy of an industry standard face-recognition algorithm from 54% to 100%, bringing the robust performance of a familiar human to an automated system.

Mental representations of familiar faces

The Bruce and Young (1986) framework makes a number of important distinctions between the types of representation needed to recognize a familiar face. Here, we return to these, focussing particularly on face recognition units. We argue that such representations need to incorporate idiosyncratic within-person variability, asking questions such as What counts as a picture of Harrison Ford?. We describe a mechanism for achieving this, and discuss the relation between image variability and episodic face memories, in the context of behavioural and neurophysiological data.

Stable face representations.

Photographs are often used to establish the identity of an individual or to verify that they are who they claim to be. Yet, recent research shows that it is surprisingly difficult to match a photo to a face. Neither humans nor machines can perform this task reliably. Although human perceivers are good at matching familiar faces, performance with unfamiliar faces is strikingly poor. The situation is no better for automatic face recognition systems. In practical settings, automatic systems have been consistently disappointing. In this review, we suggest that failure to distinguish between familiar and unfamiliar face processing has led to unrealistic expectations about face identification in applied settings. We also argue that a photograph is not necessarily a reliable indicator of facial appearance, and develop our proposal that summary statistics can provide more stable face representations. In particular, we show that image averaging stabilizes facial appearance by diluting aspects of the image that vary between snapshots of the same person. We review evidence that the resulting images can outperform photographs in both behavioural experiments and computer simulations, and outline promising directions for future research.

Passport Officers' Errors in Face Matching

Photo-ID is widely used in security settings, despite research showing that viewers find it very difficult to match unfamiliar faces. Here we test participants with specialist experience and training in the task: passport-issuing officers. First, we ask officers to compare photos to live ID-card bearers, and observe high error rates, including 14% false acceptance of ‘fraudulent’ photos. Second, we compare passport officers with a set of student participants, and find equally poor levels of accuracy in both groups. Finally, we observe that passport officers show no performance advantage over the general population on a standardised face-matching task. Across all tasks, we observe very large individual differences: while average performance of passport staff was poor, some officers performed very accurately – though this was not related to length of experience or training. We propose that improvements in security could be made by emphasising personnel selection.

Visual representation of eye gaze is coded by a nonopponent multichannel system.

To date, there is no functional account of the visual perception of gaze in humans. Previous work has demonstrated that left gaze and right gaze are represented by separate mechanisms. However, these data are consistent with either a multichannel system comprising separate channels for distinct gaze directions (e.g., left, direct, and right) or an opponent-coding system in which all gaze directions are coded by just 2 pools of cells, one coding left gaze and the other right, with direct gaze represented as a neutral point reflecting equal activation of both left and right pools. In 2 experiments, the authors used adaptation procedures to investigate which of these models provides the optimal account. Both experiments supported multichannel coding. Previous research has shown that facial identity is coded by an opponent-coding system; hence, these results also demonstrate that gaze is coded by a different representational system to facial identity.