Ilja Frissen

A modulatory role for facial expressions in prosopagnosia

Brain-damaged patients experience difficulties in recognizing a face (prosopagnosics), but they can still recognize its expression. The dissociation between these two face-related skills has served as a keystone of models of face processing. We now report that the presence of a facial expression can influence face identification. For normal viewers, the presence of a facial expression influences performance negatively, whereas for prosopagnosic patients, it improves performance dramatically. Accordingly, although prosopagnosic patients show a failure to process the facial configuration in the interest of face identification, that ability returns when the face shows an emotional expression. Accompanying brain-imaging results indicate activation in brain areas (amygdala, superior temporal sulcus, parietal cortex) outside the occipitotemporal areas normally activated for face identification and lesioned in these patients. This finding suggests a modulatory role of these areas in face identification that is independent of occipitotemporal face areas.

The aftereffects of ventriloquism: Are they sound-frequency specific?

Exposing different sense modalities (like sight, hearing or touch) to repeated simultaneous but spatially discordant stimulations generally causes recalibration of localization processes in one or both of the involved modalities, which is manifested through aftereffects. These provide opportunities for determining the extent of the changes induced by the exposure. Taking the so-called ventriloquism situation, in which synchronized sounds and light flashes are delivered in different locations, we examine if auditory recalibration produced by exposing tones of one frequency to attraction by discordant light flashes generalizes to different frequencies. Contrary to an earlier report, generalization was obtained across two octaves. This result did not depend on which modality attention was forced on through catch trials during exposure. Implications concerning the functional site of recalibration are briefly discussed.

Integration of vestibular and proprioceptive signals for spatial updating

Spatial updating during self-motion typically involves the appropriate integration of both visual and non-visual cues, including vestibular and proprioceptive information. Here, we investigated how human observers combine these two non-visual cues during full-stride curvilinear walking. To obtain a continuous, real-time estimate of perceived position, observers were asked to continuously point toward a previously viewed target in the absence of vision. They did so while moving on a large circular treadmill under various movement conditions. Two conditions were designed to evaluate spatial updating when information was largely limited to either proprioceptive information (walking in place) or vestibular information (passive movement). A third condition evaluated updating when both sources of information were available (walking through space) and were either congruent or in conflict. During both the passive movement condition and while walking through space, the pattern of pointing behavior demonstrated evidence of accurate egocentric updating. In contrast, when walking in place, perceived self-motion was underestimated and participants always adjusted the pointer at a constant rate, irrespective of changes in the rate at which the participant moved relative to the target. The results are discussed in relation to the maximum likelihood estimation model of sensory integration. They show that when the two cues were congruent, estimates were combined, such that the variance of the adjustments was generally reduced. Results also suggest that when conflicts were introduced between the vestibular and proprioceptive cues, spatial updating was based on a weighted average of the two inputs.

Walking along curved paths of different angles: the relationship between head and trunk turning.

Walking along a curved path requires coordinated motor actions of the entire body. Here, we investigate the relationship between head and trunk movements during walking. Previous studies have found that the head systematically turns into turns before the trunk does. This has been found to occur at a constant distance rather than at a constant time before a turn. We tested whether this anticipatory head behavior is spatially invariant for turns of different angles. Head and trunk positions and orientations were measured while participants walked around obstacles in 45°, 90°, 135° or 180° turns. The radius of the turns was either imposed or left free. We found that the head started to turn into the direction of the turn at a constant distance before the obstacle (~1.1xa0m) for turn angles up to 135°. During turns, the head was consistently oriented more into the direction of the turn than the trunk. This difference increased for larger turning angles and reached its maximum later in the turn for larger turns. Walking speeds decreased monotonically for increasing turn angles. Imposing fixed turn radii only affected the point at which the trunk started to turn into a turn. Our results support the view that anticipatory head movements during turns occur in order to gather advance visual information about the trajectory and potential obstacles.

The aftereffects of ventriloquism: Patterns of spatial generalization

We examined how visual recalibration of apparent sound location obtained at a particular location generalizes to untrained locations. Participants pointed toward the origin of tone bursts scattered along the azimuth, before and after repeated exposure to bursts in one particular location, synchronized with point flashes of light a constant distance to their left/right. Adapter tones were presented straight ahead in Experiment 1, and in the left or right periphery in Experiment 2. With both arrangements, different generalization patterns were obtained on the visual distractors side of the auditory adapter and on the opposite side. On the distractor side, recalibration generalized following a descending gradient; practically no generalization was observed on the other side. This dependence of generalization patterns on the direction of the discordance imposed during adaptation has not been reported before, perhaps because the experimental designs in use did not allow its observation.

Visual bias of perceived tactile location

The primary source of information to determine where on the body we are being touched is derived from the somatosensory system. However, can visual information influence the perceived location of touch on the body? Ten participants localized a brief air puff (ca. 250ms) applied to the smooth ventral surface of the right forearm somewhere between wrist and elbow. Localization was measured with a 2AFC paradigm in which participants judged the location of the tactile stimulus relative to a visual reference using two opposing 1-up/2-down staircases. Participants‘ task was to indicate whether the air puff was closer or further from the wrist relative to the reference. In one condition the visual reference was a line drawn on the forearm midway between elbow and wrist (AIR ONLY). In another the air puff was accompanied by a temporally synchronous line of laser light projected onto the reference location (AIR+LASER). We expected the synchronous light to facilitate multimodal integration and therefore affect discrimination performance without introducing a bias. In the AIR ONLY condition the PSE was on average on the reference location. Surprisingly, in the AIR+LASER condition we did not find a change in discriminability relative to the AIR ONLY condition. We found, however, a significant shift of the PSE by 0.9cm towards the elbow. This bias was evident in eight out of the ten participants. This demonstration of a visual effect on tactile localization may indicate that judging the location of a visual reference on the body is not free from biases.

Visual bias on sound location modulated by content based processes.

Ventriloquism refers to a perceptual phenomenon in which the apparent location of a sound source is displaced in the direction of a synchronous but spatially disparate visual stimulus. It is generally accepted that spatial and temporal proximity are factors facilitating crossmodal integration. Here we investigate whether content based processes could also play a role. In order to control for strategic factors, a psychophysical staircase method (Bertelson and Aschersleben, 1998) was adopted. Auditory stimuli were digital recordings of vowels (/i/ and /o/). Visual stimuli were digital pictures of talking faces articulating the same vowels, and a scrambled face. We ran eight concurrent staircases. In half of these the auditory stimuli were paired with the corresponding face, and in the other half with the scrambled face. Half the staircases started from the extreme left and the other from the extreme right. Presentation of staircases was randomized. Participants were asked to judge whether the sound was coming from the left, or from the right, of the median plane. On the staircases with a face stimulus, reversals started to occur significantly earlier than with a nonface. Thus, a ‘‘realistic’’ stimulus pairing enhances crossmodal integration.