Nadine Kloth

Auditory Adaptation in Voice Perception

Perceptual aftereffects following adaptation to simple stimulus attributes (e.g., motion, color) have been studied for hundreds of years. A striking recent discovery was that adaptation also elicits contrastive aftereffects in visual perception of complex stimuli and faces [1-6]. Here, we show for the first time that adaptation to nonlinguistic information in voices elicits systematic auditory aftereffects. Prior adaptation to male voices causes a voice to be perceived as more female (and vice versa), and these auditory aftereffects were measurable even minutes after adaptation. By contrast, crossmodal adaptation effects were absent, both when male or female first names and when silently articulating male or female faces were used as adaptors. When sinusoidal tones (with frequencies matched to male and female voice fundamental frequencies) were used as adaptors, no aftereffects on voice perception were observed. This excludes explanations for the voice aftereffect in terms of both pitch adaptation and postperceptual adaptation to gender concepts and suggests that contrastive voice-coding mechanisms may routinely influence voice perception. The role of adaptation in calibrating properties of high-level voice representations indicates that adaptation is not confined to vision but is a ubiquitous mechanism in the perception of nonlinguistic social information from both faces and voices.

Effects of personal familiarity on early neuromagnetic correlates of face perception

This study investigated effects of familiarity and orientation on face processing by means of magnetoencephalography. Participants were presented with photographs of personally familiar, famous and unfamiliar faces in both upright and inverted orientation. They had to decide whether faces were familiar by means of manual yes/no responses. Independent of orientation, we observed a clear modulation of the M170 by familiarity, with personally familiar faces evoking larger amplitudes than unknown faces. The M170 was also sensitive to orientation, with larger amplitudes for inverted than upright faces. Moreover, the M170 exhibited larger amplitudes over the right than over the left hemisphere, but this asymmetry was present for upright faces only. The present data suggest that at least for personally familiar faces, neural correlates of identification start no later than ∼ 170 ms, and underline a special role of the right hemisphere for faces in their typical upright orientation.

Hearing facial identities: Brain correlates of face–voice integration in person identification

Audiovisual integration (AVI) is a well-known aspect of speech perception, but integration of facial and vocal information is also important for speaker recognition. We recently demonstrated AVI in the recognition of familiar (but not unfamiliar) speakers. Specifically, systematic behavioural benefits and costs in recognizing a familiar voice occur when the voice is combined with a time-synchronised articulating face of corresponding or noncorresponding speaker identity, respectively (Schweinberger et al., 2007; Robertson and Schweinberger, 2010). Here we report an experiment assessing event-related brain potentials (ERPs) in this novel paradigm, while participants recognized familiar speakers presented in (1) Voice only, (2) voice with identity-corresponding and (3) noncorresponding time-synchronised speaking faces, as well as (4) Face only conditions. Audiovisual speaker identity correspondence influenced only later ERPs around 250-600 msec, with increased negativity for noncorresponding identities at central electrodes. Strikingly, when compared with the ERPs from both unimodal conditions, both audiovisual conditions led to a much earlier onset of frontocentral negativity, with maximal differences around 50-80 msec. Moreover, audiovisual stimuli elicited larger N170 responses than Face only stimuli. These findings suggest that the perception of a voice and a time-synchronised articulating face triggers remarkably early and mandatory mechanisms of audiovisual processing, although the correspondence or discrepancy in audiovisual speaker identity may only be computed ∼200msec later.

How distinct is the coding of face identity and expression? Evidence for some common dimensions in face space

Traditional models of face perception emphasize distinct routes for processing face identity and expression. These models have been highly influential in guiding neural and behavioural research on the mechanisms of face perception. However, it is becoming clear that specialised brain areas for coding identity and expression may respond to both attributes and that identity and expression perception can interact. Here we use perceptual aftereffects to demonstrate the existence of dimensions in perceptual face space that code both identity and expression, further challenging the traditional view. Specifically, we find a significant positive association between face identity aftereffects and expression aftereffects, which dissociates from other face (gaze) and non-face (tilt) aftereffects. Importantly, individual variation in the adaptive calibration of these common dimensions significantly predicts ability to recognize both identity and expression. These results highlight the role of common dimensions in our ability to recognize identity and expression, and show why the high-level visual processing of these attributes is not entirely distinct.

Combined Effects of Inversion and Feature Removal on N170 Responses Elicited by Faces and Car Fronts.

The face-sensitive N170 is typically enhanced for inverted compared to upright faces. Itier, Alain, Sedore, and McIntosh (2007) recently suggested that this N170 inversion effect is mainly driven by the eye region which becomes salient when the face configuration is disrupted. Here we tested whether similar effects could be observed with non-face objects that are structurally similar to faces in terms of possessing a homogeneous within-class first-order feature configuration. We presented upright and inverted pictures of intact car fronts, car fronts without lights, and isolated lights, in addition to analogous face conditions. Upright cars elicited substantial N170 responses of similar amplitude to those evoked by upright faces. In strong contrast to face conditions however, the car-elicited N170 was mainly driven by the global shape rather than the presence or absence of lights, and was dramatically reduced for isolated lights. Overall, our data confirm a differential influence of the eye region in upright and inverted faces. Results for car fronts do not suggest similar interactive encoding of eye-like features and configuration for non-face objects, even when these objects possess a similar feature configuration as faces.

How well do computer-generated faces tap face expertise?

The use of computer-generated (CG) stimuli in face processing research is proliferating due to the ease with which faces can be generated, standardised and manipulated. However there has been surprisingly little research into whether CG faces are processed in the same way as photographs of real faces. The present study assessed how well CG faces tap face identity expertise by investigating whether two indicators of face expertise are reduced for CG faces when compared to face photographs. These indicators were accuracy for identification of own-race faces and the other-race effect (ORE)–the well-established finding that own-race faces are recognised more accurately than other-race faces. In Experiment 1 Caucasian and Asian participants completed a recognition memory task for own- and other-race real and CG faces. Overall accuracy for own-race faces was dramatically reduced for CG compared to real faces and the ORE was significantly and substantially attenuated for CG faces. Experiment 2 investigated perceptual discrimination for own- and other-race real and CG faces with Caucasian and Asian participants. Here again, accuracy for own-race faces was significantly reduced for CG compared to real faces. However the ORE was not affected by format. Together these results signal that CG faces of the type tested here do not fully tap face expertise. Technological advancement may, in the future, produce CG faces that are equivalent to real photographs. Until then caution is advised when interpreting results obtained using CG faces.

Perceiving age and gender in unfamiliar faces: An fMRI study on face categorization

Efficient processing of unfamiliar faces typically involves their categorization (e.g., into old vs. young or male vs. female). However, age and gender categorization may pose different perceptual demands. In the present study, we employed functional magnetic resonance imaging (fMRI) to compare the activity evoked during age vs. gender categorization of unfamiliar faces. In different blocks, participants performed age and gender classifications for old or young unfamiliar faces (50% female respectively). Both tasks elicited activations in the bilateral fusiform gyri (fusiform face area, FFA) and bilateral inferior occipital gyri (occipital face area, OFA). Importantly, the same stimuli elicited enhanced activation during gender as compared to age categorization. This enhancement was significant in the right FFA and the left OFA, and may be related to increased configural processing. Our findings replicate and extend recent work, and shows that the activation of core components of the face processing network is strongly dependent on task demands.

Electrophysiological correlates of eye gaze adaptation

Recent research shows a strong effect of adaptation on gaze perception: Adaptation to faces with eye gaze constantly diverted in one direction subsequently impairs the perception of that gaze direction. A previous study on the neural correlates of this effect found that N170 amplitudes to test faces were strongly attenuated following adaptation. ERP modulations as a function of gaze direction adaptation, however, were found only later (250-350 ms). Here, we used a new paradigm to study the exact nature of both the gaze direction-invariant N170 attenuation effect and the direction-specific effects in later time windows. We compared the ability to classify gaze direction before and after adaptation to direct gaze (control condition) or to eye gaze diverted to the right (adaptation condition). Behavioral results clearly replicated earlier findings of an impaired perception of eye gaze directed to the adapted side. The ERP analysis confirmed an insensitivity of the N170 to gaze adaptation, suggesting that reported attenuations resulted from adaptation to generic face information irrespective of gaze direction. Occipitotemporal ERPs ∼250-350 ms showed direction-specific modulations with most positive amplitudes in response to stimuli gazing in the direction of adaptation. Finally, there was an effect in the parietal late positive component ∼400-600 ms, which we interpret as a neural correlate of adaptation-induced novelty detection.

Adaptation Effects to Attractiveness of Face Photographs and Art Portraits are Domain-Specific

We studied the neural coding of facial attractiveness by investigating effects of adaptation to attractive and unattractive human faces on the perceived attractiveness of veridical human face pictures (Experiment 1) and art portraits (Experiment 2). Experiment 1 revealed a clear pattern of contrastive aftereffects. Relative to a pre-adaptation baseline, the perceived attractiveness of faces was increased after adaptation to unattractive faces, and was decreased after adaptation to attractive faces. Experiment 2 revealed similar aftereffects when art portraits rather than face photographs were used as adaptors and test stimuli, suggesting that effects of adaptation to attractiveness are not restricted to facial photographs. Additionally, we found similar aftereffects in art portraits for beauty, another aesthetic feature that, unlike attractiveness, relates to the properties of the image (rather than to the face displayed). Importantly, Experiment 3 showed that aftereffects were abolished when adaptors were art portraits and face photographs were test stimuli. These results suggest that adaptation to facial attractiveness elicits aftereffects in the perception of subsequently presented faces, for both face photographs and art portraits, and that these effects do not cross image domains.

Aftereffects support opponent coding of face gender

Many aspects of faces derived from structural information appear to be neurally represented using norm-based opponent coding. Recently, however, Zhao, Seriès, Hancock, and Bednar (2011) have argued that another aspect with a strong structural component, namely face gender, is instead multichannel coded. Their conclusion was based on finding that face gender aftereffects initially increased but then decreased for adaptors with increasing levels of gender caricaturing. Critically, this interpretation rests on the untested assumption that caricaturing the differences between male and female composite faces increases perceived sexual dimorphism (masculinity/femininity) of faces. We tested this assumption in Study 1 and found that it held for male, but not female faces. A multichannel account cannot, therefore, be ruled out, although a decrease in realism of adaptors was observed that could have contributed to the decrease in aftereffects. However, their aftereffects likely reflect low-level retinotopic adaptation, which was not minimized for most of their participants. In Study 2 we minimized low-level adaptation and found that face gender aftereffects were strongly positively related to the perceived sexual dimorphism of adaptors. We found no decrease for extreme adaptors, despite testing adaptors with higher perceived sexual dimorphism levels than those used by Zhao et al. These results are consistent with opponent coding of higher-level dimensions related to the perception of face gender.