F. Peronnet

Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study

The aim of this study was (1) to provide behavioral evidence for multimodal feature integration in an object recognition task in humans and (2) to characterize the processing stages and the neural structures where multisensory interactions take place. Event-related potentials (ERPs) were recorded from 30 scalp electrodes while subjects performed a forced-choice reaction-time categorization task: At each trial, the subjects had to indicate which of two objects was presented by pressing one of two keys. The two objects were defined by auditory features alone, visual features alone, or the combination of auditory and visual features. Subjects were more accurate and rapid at identifying multimodal than unimodal objects. Spatiotemporal analysis of ERPs and scalp current densities revealed several auditory-visual interaction components temporally, spatially, and functionally distinct before 200 msec poststimulus. The effects observed were (1) in visual areas, new neural activities (as early as 40 msec poststimulus) and modulation (amplitude decrease) of the N185 wave to unimodal visual stimulus, (2) in the auditory cortex, modulation (amplitude increase) of subcomponents of the unimodal auditory N1 wave around 90 to 110 msec, and (3) new neural activity over the right fronto-temporal area (140 to 165 msec). Furthermore, when the subjects were separated into two groups according to their dominant modality to perform the task in unimodal conditions (shortest reaction time criteria), the integration effects were found to be similar for the two groups over the nonspecific fronto-temporal areas, but they clearly differed in the sensory-specific cortices, affecting predominantly the sensory areas of the nondominant modality. Taken together, the results indicate that multisensory integration is mediated by flexible, highly adaptive physiological processes that can take place very early in the sensory processing chain and operate in both sensory-specific and nonspecific cortical structures in different ways.

Electrophysiological evidence for a shared representational medium for visual images and visual percepts.

Does mental imagery involve the activation of representations in the visual system? Systematic effects of imagery on visual signal detection performance have been used to argue that imagery and the perceptual processing of stimuli interact at some common locus of activity (Farah, 1985). However, such a result is neutral with respect to the question of whether the interaction occurs during modality-specific visual processing of the stimulus. If imagery affects stimulus processing at early, modality-specific stages of stimulus representation, this implies that the shared stimulus representations are visual, whereas if imagery affects stimulus processing only at later, amodal stages of stimulus representation, this implies that imagery involves more abstract, postvisual stimulus representations. To distinguish between these two possibilities, we repeated the earlier imagery-perception interaction experiment while recording event-related potentials (ERPs) to stimuli from 16 scalp electrodes. By observing the time course and scalp distribution of the effect of imagery on the ERP to stimuli, we can put constraints on the locus of the shared representations for imagery and perception. An effect of imagery was seen within 200 ms following stimulus presentation, at the latency of the first negative component of the visual ERP, localized at the occipital and posterior temporal regions of the scalp, that is, directly over visual cortex. This finding provides support for the claim that mental images interact with percepts in the visual system proper and hence that mental images are themselves visual representations.

Mental rotation: An event-related potential study with a validated mental rotation task

Event-related potentials (ERPs) were recorded while subjects performed a validated mental rotation task, taken from the cognitive psychology literature. These ERPs show a late posterior negativity relative to a baseline condition requiring all of the same perceptual and cognitive processes except for the mental rotation itself. Our tentative identification of this posterior negativity with mental rotation is further supported by the finding that it varies systematically with the amount of mental rotation required on a trial by trial basis in the experimental task. We conclude that this late negativity is an ERP marker of the mental rotation process, and that this process engages primarily posterior brain regions.

Coronal topography of human auditory evoked responses

Abstract The coronal topography of the auditory evoked response (AER) was studied in 26 normal subjects and 3 patients with hemispheric lesion which definitely involved Heschls gyrus. The results confirm the auditory cortical origin of the responses, proposed by Vaughan and Ritter (1970). They show a statistically significant interhemispheric difference, which could be explained by predominance of the right hemisphere to pure tones. When they are recorded on the same chain, visual evoked responses (VERs) present a completely different topography than AERs. The, at least partially, specific character of these responses is thus confirmed. The polarity reversal of the responses only appears on the healthy cortex in patients with lesions involving the auditory cortical area. Topographical recording of AERs may thus be considered as an objective and useful test in auditory trouble of central origin, especially unilateral lesions.

A case of cortical deafness: Clinical and electrophysiological data

Abstract A patient suffering from a primary myocardiopathy is suddenly no longer able to recognize speech and environmental sounds, in spite of good peripheral hearing; he remains able to speak, read, and write. The C.T. scan shows lesions involving both supratemporal planes. The cortical sensory deficit documented by the absence of late acoustic evoked potentials leads to a diagnosis of cortical deafness rather than of auditory agnosia.

Sequential colour mapping system of brain potentials

We present a colour mapping system for the visualization of both the spatial scalp distribution and the temporal evolution of brain potentials. The system is applicable to recordings of auditory, visual and somatosensory potentials. It uses a Tektronix 4113 colour terminal connected to a Solar 16 (SEMS) mini-computer. The brain potentials are recorded on up to 16 scalp electrodes. The gain and the baseline are corrected separately on each channel. At each point of the scalp the potential is reconstructed by a linear interpolation of the measured potentials of the four nearest electrodes. Simultaneously n2 (1 less than n less than 8) colour maps can be presented on the screen. This allows the study of the temporal evolution of full scalp evoked potentials. The user chooses the two extreme latencies defining the time window to be explored and the latencies of the maps are regularly time-spaced within this window. In a typical case, in which four maps are desired, the latencies of the maps can be chosen independently. The 16-colour palette is predetermined but the user has three possibilities to establish the correspondence between the electrical potential and the colours. Examples are shown in the visual and somatosensory stimulation modalities. The advantages and limitations of such a representation are discussed.

The temporal component of the auditory evoked potential: A reinterpretation

The auditory evoked potentials in man cannot be explained by a single source even though a strong influence of the primary areas in the supra-temporal plane has been pointed out in different works. In 26 normal adults we mathematically extracted the greater part of the experimental AEPs explicable by such an origin. The residual part obtained by subtracting this first component from the experimental data is in agreement with an origin in the precentral motor cortex.

Evidence for Shared Structures Between Imagery and Perception

The evoked activity mapping during a task in which a mental image interacts with the detection of a stimulus was recorded. We found a systematic effect of imagery on the evoked potentials showing a greater early negativity when the image and the stimulus were the same than when they were different shapes. Further we found that this effect was maximal over the occipital regions of the scalp. These two results converge to indicate that imagery engages perceptual representations in the visual system proper.

Une methode originale d'analyse multivariee, adaptee a la comparaison des potentiels evoques moyens: Le test E2

This paper describes a multivariate statistical method of analysis for the comparison of 2 paired series. For such paired series, this test, called E2, is analogous to the generalized Hotteling T2 test, as epsilon test (pairing method) is analogous to Students t test. Here we give an example of its application to a comparison of curves, in this particular case average auditory evoked potentials, picked up at symmetrical points on the scalp.