Jason M. Gold

Inversion Leads to Quantitative, Not Qualitative, Changes in Face Processing

Humans are remarkably adept at recognizing objects across a wide range of views. A notable exception to this general rule is that turning a face upside down makes it particularly difficult to recognize. This striking effect has prompted speculation that inversion qualitatively changes the way faces are processed. Researchers commonly assume that configural cues strongly influence the recognition of upright, but not inverted, faces. Indeed, the assumption is so well accepted that the inversion effect itself has been taken as a hallmark of qualitative processing differences. Here, we took a novel approach to understand the inversion effect. We used response classification to obtain a direct view of the perceptual strategies underlying face discrimination and to determine whether orientation effects can be explained by differential contributions of nonlinear processes. Inversion significantly impaired performance in our face discrimination task. However, surprisingly, observers utilized similar, local regions of faces for discrimination in both upright and inverted face conditions, and the relative contributions of nonlinear mechanisms to performance were similar across orientations. Our results suggest that upright and inverted face processing differ quantitatively, not qualitatively; information is extracted more efficiently from upright faces, perhaps as a by-product of orientation-dependent expertise.

Signal but not noise changes with perceptual learning.

Perceptual discrimination improves with practice. This ‘perceptual learning’ is often specific to the stimuli presented during training, indicating that practice may alter the response characteristics of cortical sensory neurons. Although much is known about how learning modifies cortical circuits, it remains unclear how these changes relate to behaviour. Different theories assume that practice improves discrimination by enhancing the signal, diminishing internal noise or both. Here, to distinguish among these alternatives, we fashioned sets of faces and textures whose signal strength could be varied, and we trained observers to identify these patterns embedded in noise. Performance increased by up to 400% across several sessions over several days. Comparisons of human performance to that of an ideal discriminator showed that learning increased the efficiency with which observers encoded task-relevant information. Observer response consistency, measured by a double-pass technique in which identical stimuli are shown twice in each experimental session, did not change during training, showing that learning had no effect on internal noise. These results indicate that perceptual learning may enhance signal strength, and provide important constraints for theories of learning.

Identification of band-pass filtered letters and faces by human and ideal observers

To better understand how the visual system makes use of information across spatial scales when identifying different kinds of complex patterns, we measured human and ideal contrast identification thresholds to estimate identification efficiency for 1- and 2-octave wide band-pass filtered letters and faces embedded in 2-D dynamic Gaussian noise. Varying stimulus center frequency from 1 to 70 c/object had different effects on letter and face identification efficiency. In the 2-octave conditions, identification efficiencies decreased by 0.25-0.5 log units for letters and 0.5-1.2 log units for faces as center frequency increased from 6.2 to 49.5 c/object, but only letters were identifiable at center frequencies below 6.2 c/object. In the 1-octave conditions, letter identification efficiencies increased by about 0.5 log units as center frequency increased from 1.1 to 2.2 c/object, and were nearly constant from 2.2 to 35 c/object. Letters were unidentifiable by human observers at 70 c/object. Surprisingly, face identification was impossible for human observers at all center frequencies except 8.8 c/object for one observer, and 8.8 and 17.5 c/object for a second observer. Ideal observer thresholds were obtained for both letters and faces in all conditions, so information was always available to perform the task. Thus, the failure to identify faces reflects constraints on visual processing rather than a lack of stimulus information. Selective spatial sampling may account for some of the differences between letter and face identification efficiencies.

The effect of the physical characteristics of cues and targets on facilitation and inhibition

The present experiment was conducted in order to examine the role of cue—target discriminability on early occurring attentional cuing effects and late occurring inhibition of return (IOR). The experiment used a single target stimulus in conjunction with three different cue stimuli. The cues were the same as the target, different in color, shape, and luminance to the target, or did not spatially overlap with the target. At shorter stimulus onset asynchronies (SOAs; 100 and 200 msec), attentional cuing effects were only found with the nonoverlapping cues. However, at longer SOAs (400 and 800 msec), approximately equal IOR effects were found with all three types of cues. The results indicated that the physical characteristics of the cues and targets affected the pattern of reaction times at the shorter SOAs but not at the longer SOAs. The conclusion is that the biphasic pattern of early facilitation and late inhibition following a peripheral cue should not be considered the definitive signature of the peripheral cuing paradigm.

The Perception of a Face Is No More Than the Sum of Its Parts

When you see a person’s face, how do you go about combining his or her facial features to make a decision about who that person is? Most current theories of face perception assert that the ability to recognize a human face is not simply the result of an independent analysis of individual features, but instead involves a holistic coding of the relationships among features. This coding is thought to enhance people’s ability to recognize a face beyond what would be expected if each feature were shown in isolation. In the study reported here, we explicitly tested this idea by comparing human performance on facial-feature integration with that of an optimal Bayesian integrator. Contrary to the predictions of most current notions of face perception, our findings showed that human observers integrate facial features in a manner that is no better than would be predicted by their ability to use each individual feature when shown in isolation. That is, a face is perceived no better than the sum of its individual parts.

The response of face-selective cortex with single face parts and part combinations

A critical issue in object recognition research is how the parts of an object are analyzed by the visual system and combined into a perceptual whole. However, most of the previous research has examined how changes to object parts influence recognition of the whole, rather than recognition of the parts themselves. This is particularly true of the research on face recognition, and especially with questions related to the neural substrates. Here, we investigated patterns of BOLD fMRI brain activation with internal face parts (features) presented singly and in different combinations. A preference for single features over combinations was found in the occipital face area (OFA) as well as a preference for the two-eyes combination stimulus over other combination stimulus types. The fusiform face area (FFA) and lateral occipital cortex (LO) showed no preferences among the single feature and combination stimulus types. The results are consistent with a growing view that the OFA represents processes involved in early, feature-based analysis.

Visual Memory Decay Is Deterministic

After observers see an object or pattern, their visual memory of what they have seen decays slowly over time. Nearly all current theories of vision assume that decay of short-term memory occurs because visual representations are progressively and randomly corrupted as time passes. We tested this assumption using psychophysical noise-masking methods, and we found that visual memory decays in a completely deterministic fashion. This surprising finding challenges current ideas about visual memory and sets a goal for future memory research: to characterize the deterministic “forgetting function” that describes how memories decay over time.

The efficiency of biological motion perception.

Humans can readily perceive biological motion from point-light (PL) animations, which create an image of a moving human figure by tracing the trajectories of a small number of light points affixed to a moving human body. We have applied ideal observer analysis to a standard biological motion discrimination task involving either full-figure or PL displays. Contrary to current dogma, we find that PL animations can be rich in potential stimulus information but that human observers are remarkably inefficient at exploiting this information. Although our findings do not discount the utility of PL animation, they do provide a realistic measure of the computational challenge posed by biological motion perception.

Verbal labeling, gradual decay, and sudden death in visual short-term memory.

Zhang and Luck (Psychological Science, 20, 423–428, 2009) found that perceptual memories are lost over time via sudden death rather than gradual decay. However, they acknowledged that participants may have instead lost memory for the locations of objects. We required observers to recall only a single object. Although the paradigm eliminated the need to maintain object–location bindings, the possibility that observers would use verbal labels increased. To measure the precision of verbal labeling, we included explicit verbal-labeling and label-matching trials. We applied a model that measured the contributions of sudden death, gradual decay, and verbal labeling to recall. Our model-based evidence pointed to sudden death as the primary vehicle by which perceptual memories were lost. Crucially, however, the sudden-death hypothesis was favored only when the verbal-labeling component was included as part of the modeling. The results underscore the importance of taking into account the potential role of verbal-labeling processes in investigations of perceptual memory.

Inversion effects in face-selective cortex with combinations of face parts

The face inversion effect has been used as a basis for claims about the specialization of face-related perceptual and neural processes. One of these claims is that the fusiform face area (FFA) is the site of face-specific feature-based and/or configural/holistic processes that are responsible for producing the face inversion effect. However, the studies on which these claims were based almost exclusively used stimulus manipulations of whole faces. Here, we tested inversion effects using single, discrete features and combinations of multiple discrete features, in addition to whole faces, using both behavioral and fMRI measurements. In agreement with previous studies, we found behavioral inversion effects with whole faces and no inversion effects with a single eye stimulus or the two eyes in combination. However, we also found behavioral inversion effects with feature combination stimuli that included features in the top and bottom halves (eyes-mouth and eyes-nose-mouth). Activation in the FFA showed an inversion effect for the whole-face stimulus only, which did not match the behavioral pattern. Instead, a pattern of activation consistent with the behavior was found in the bilateral inferior frontal gyrus, which is a component of the extended face-preferring network. The results appear inconsistent with claims that the FFA is the site of face-specific feature-based and/or configural/holistic processes that are responsible for producing the face inversion effect. They are more consistent with claims that the FFA shows a stimulus preference for whole upright faces.