Till R. Schneider

Crossmodal binding through neural coherence: implications for multisensory processing

Picture yourself on a crowded sideway with people milling about. The acoustic and visual signals generated by the crowd provide you with complementary information about their locations and motion which needs to be integrated. It is not well understood how such inputs from different sensory channels are combined into unified perceptual states. Coherence of oscillatory neural signals might be an essential mechanism supporting multisensory perception. Evidence is now emerging which indicates that coupled oscillatory activity might serve to link neural signals across uni- and multisensory regions and to express the degree of crossmodal matching of stimulus-related information. These results argue for a new view on multisensory processing which considers the dynamic interplay of neural populations as a key to crossmodal integration.

Semi-automatic identification of independent components representing EEG artifact

OBJECTIVE Independent component analysis (ICA) can disentangle multi-channel electroencephalogram (EEG) signals into a number of artifacts and brain-related signals. However, the identification and interpretation of independent components is time-consuming and involves subjective decision making. We developed and evaluated a semi-automatic tool designed for clustering independent components from different subjects and/or EEG recordings. METHODS CORRMAP is an open-source EEGLAB plug-in, based on the correlation of ICA inverse weights, and finds independent components that are similar to a user-defined template. Component similarity is measured using a correlation procedure that selects components that pass a threshold. The threshold can be either user-defined or determined automatically. CORRMAP clustering performance was evaluated by comparing it with the performance of 11 users from different laboratories familiar with ICA. RESULTS For eye-related artifacts, a very high degree of overlap between users (phi>0.80), and between users and CORRMAP (phi>0.80) was observed. Lower degrees of association were found for heartbeat artifact components, between users (phi<0.70), and between users and CORRMAP (phi<0.65). CONCLUSIONS These results demonstrate that CORRMAP provides an efficient, convenient and objective way of clustering independent components. SIGNIFICANCE CORRMAP helps to efficiently use ICA for the removal EEG artifacts.

Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming

An important step in perceptual processing is the integration of information from different sensory modalities into a coherent percept. It has been suggested that such crossmodal binding might be achieved by transient synchronization of neurons from different modalities in the gamma-frequency range (>30 Hz). Here we employed a crossmodal priming paradigm, modulating the semantic congruency between visual-auditory natural object stimulus pairs, during the recording of the high density electroencephalogram (EEG). Subjects performed a semantic categorization task. Analysis of the behavioral data showed a crossmodal priming effect (facilitated auditory object recognition) in response to semantically congruent stimuli. Differences in event-related potentials (ERP) were found between 250 and 350 ms, which were localized to left middle temporal gyrus (BA 21) using a distributed linear source model. Early gamma-band activity (40-50 Hz) was increased between 120 ms and 180 ms following auditory stimulus onset for semantically congruent stimulus pairs. Source reconstruction for this gamma-band response revealed a maximal increase in left middle temporal gyrus (BA 21), an area known to be related to the processing of both complex auditory stimuli and multisensory processing. The data support the hypothesis that oscillatory activity in the gamma-band reflects crossmodal semantic-matching processes in multisensory convergence sites.

Selective modulation of interhemispheric functional connectivity by HD-tACS shapes perception.

This transcranial stimulation study shows that selective modulation of synchronized neuronal activity between the hemispheres of the brain can affect conscious perception.

Multisensory Identification of Natural Objects in a Two-Way Crossmodal Priming Paradigm

The question of how vision and audition interact in natural object identification is currently a matter of debate. We developed a large set of auditory and visual stimuli representing natural objects in order to facilitate research in the field of multisensory processing. Normative data was obtained for 270 brief environmental sounds and 320 visual object stimuli. Each stimulus was named, categorized, and rated with regard to familiarity and emotional valence by N=56 participants (Study 1). This multimodal stimulus set was employed in two subsequent crossmodal priming experiments that used semantically congruent and incongruent stimulus pairs in a S1-S2 paradigm. Task-relevant targets were either auditory (Study 2) or visual stimuli (Study 3). The behavioral data of both experiments expressed a crossmodal priming effect with shorter reaction times for congruent as compared to incongruent stimulus pairs. The observed facilitation effect suggests that object identification in one modality is influenced by input from another modality. This result implicates that congruent visual and auditory stimulus pairs were perceived as the same object and demonstrates a first validation of the multimodal stimulus set.

Gamma-Band Activity as a Signature for Cross-Modal Priming of Auditory Object Recognition by Active Haptic Exploration

When visual sensory information is restricted, we often rely on haptic and auditory information to recognize objects. Here we examined how haptic exploration of familiar objects affects neural processing of subsequently presented sounds of objects. Recent studies indicated that oscillatory responses, in particular in the gamma band (30–100 Hz), reflect cross-modal processing, but it is not clear which cortical networks are involved. In this high-density EEG study, we measured gamma-band activity (GBA) in humans performing a haptic-to-auditory priming paradigm. Haptic stimuli served as primes, and sounds of objects as targets. Haptic and auditory stimuli were either semantically congruent or incongruent, and participants were asked to categorize the objects represented by the sounds. Response times were shorter for semantically congruent compared with semantically incongruent inputs. This haptic-to-auditory priming effect was associated with enhanced total power GBA (250–350 ms) for semantically congruent inputs and additional effects of semantic congruency on evoked GBA (50–100 ms). Source reconstruction of total GBA using linear beamforming revealed effects of semantic congruency in the left lateral temporal lobe, possibly reflecting matching of information across modalities. For semantically incongruent inputs, total GBA was enhanced in middle frontal cortices, possibly indicating the processing or detection of conflicting information. Our findings demonstrate that semantic priming by haptic object exploration affects processing of auditory inputs in the lateral temporal lobe and suggest an important role of oscillatory activity for multisensory processing.

Noise alters beta-band activity in superior temporal cortex during audiovisual speech processing

Speech recognition is improved when complementary visual information is available, especially under noisy acoustic conditions. Functional neuroimaging studies have suggested that the superior temporal sulcus (STS) plays an important role for this improvement. The spectrotemporal dynamics underlying audiovisual speech processing in the STS, and how these dynamics are affected by auditory noise, are not well understood. Using electroencephalography, we investigated how auditory noise affects audiovisual speech processing in event-related potentials (ERPs) and oscillatory activity. Spoken syllables were presented in audiovisual (AV) and auditory only (A) trials at three different auditory noise levels (no, low, and high). Responses to A stimuli were subtracted from responses to AV stimuli, separately for each noise level, and these responses were subjected to the statistical analysis. Central ERPs differed between the no noise and the two noise conditions from 130 to 150 ms and 170 to 210 ms after auditory stimulus onset. Source localization using the local autoregressive average procedure revealed an involvement of the lateral temporal lobe, encompassing the superior and middle temporal gyrus. Neuronal activity in the beta-band (16 to 32 Hz) was suppressed at central channels around 100 to 400 ms after auditory stimulus onset in the averaged AV minus A signal over the three noise levels. This suppression was smaller in the high noise compared to the no noise and low noise condition, possibly reflecting disturbed recognition or altered processing of multisensory speech stimuli. Source analysis of the beta-band effect using linear beamforming demonstrated an involvement of the STS. Our study shows that auditory noise alters audiovisual speech processing in ERPs localized to lateral temporal lobe and provides evidence that beta-band activity in the STS plays a role for audiovisual speech processing under regular and noisy acoustic conditions.

Functionally specific oscillatory activity correlates between visual and auditory cortex in the blind

Many studies have shown that the visual cortex of blind humans is activated in non-visual tasks. However, the electrophysiological signals underlying this cross-modal plasticity are largely unknown. Here, we characterize the neuronal population activity in the visual and auditory cortex of congenitally blind humans and sighted controls in a complex cognitive task. We recorded magnetoencephalographic responses from participants performing semantic categorization of meaningful sounds that followed the presentation of a semantically related or unrelated haptic object. Source analysis of the spectrally resolved magnetoencephalography data revealed that: (i) neuronal responses to sounds were stronger and longer lasting in the auditory cortex of blind subjects; (ii) auditory stimulation elicited strong oscillatory responses in the visual cortex of blind subjects that closely resembled responses to visual stimulation in sighted humans; (iii) the signal in the gamma frequency range was modulated by semantic congruency between the sounds and the preceding haptic objects; and (iv) signal power in the gamma range was correlated on a trial-by-trial basis between auditory and visual cortex in blind subjects, and the strength of this correlation was modulated by semantic congruency. Our results suggest that specifically oscillatory activity in the gamma range reflects non-visual processing in the visual cortex of blind individuals. Moreover, our results provide evidence that the deprived visual cortex is functionally integrated into a larger network that serves non-visual functions.

Perceptual Integration Deficits in Autism Spectrum Disorders Are Associated with Reduced Interhemispheric Gamma-Band Coherence.

The integration of visual details into a holistic percept is essential for object recognition. This integration has been reported as a key deficit in patients with autism spectrum disorders (ASDs). The weak central coherence account posits an altered disposition to integrate features into a coherent whole in ASD. Here, we test the hypothesis that such weak perceptual coherence may be reflected in weak neural coherence across different cortical sites. We recorded magnetoencephalography from 20 adult human participants with ASD and 20 matched controls, who performed a slit-viewing paradigm, in which objects gradually passed behind a vertical or horizontal slit so that only fragments of the object were visible at any given moment. Object recognition thus required perceptual integration over time and, in case of the horizontal slit, also across visual hemifields. ASD participants were selectively impaired in the horizontal slit condition, indicating specific difficulties in long-range synchronization between the hemispheres. Specifically, the ASD group failed to show condition-related enhancement of imaginary coherence between the posterior superior temporal sulci in both hemispheres during horizontal slit-viewing in contrast to controls. Moreover, local synchronization reflected in occipitocerebellar beta-band power was selectively reduced for horizontal compared with vertical slit-viewing in ASD. Furthermore, we found disturbed connectivity between right posterior superior temporal sulcus and left cerebellum. Together, our results suggest that perceptual integration deficits co-occur with specific patterns of abnormal global and local synchronization in ASD. SIGNIFICANCE STATEMENT The weak central coherence account proposes a tendency of individuals with autism spectrum disorders (ASDs) to focus on details at the cost of an integrated coherent whole. Here, we provide evidence, at the behavioral and the neural level, that visual integration in object recognition is impaired in ASD, when details had to be integrated across both visual hemifields. We found enhanced interhemispheric gamma-band coherence in typically developed participants when communication between cortical hemispheres was required by the task. Importantly, participants with ASD failed to show this enhanced coherence between bilateral posterior superior temporal sulci. The findings suggest that visual integration is disturbed at the local and global synchronization scale, which might bear implications for object recognition in ASD.

Different coupling modes mediate cortical cross-frequency interactions.

Cross-frequency coupling (CFC) has been suggested to constitute a highly flexible mechanism for cortical information gating and processing, giving rise to conscious perception and various higher cognitive functions in humans. In particular, it might provide an elegant tool for information integration across several spatiotemporal scales within nested or coupled neuronal networks. However, it is currently unknown whether low-frequency (theta/alpha) or high-frequency gamma oscillations orchestrate cross-frequency interactions, raising the question of who is master and who is slave. While correlative evidence suggested that at least two distinct CFC modes exist, namely, phase-amplitude-coupling (PAC) and amplitude-envelope correlations (AEC), it is currently unknown whether they subserve distinct cortical functions. Novel non-invasive brain stimulation tools, such as transcranial alternating current stimulation (tACS), now provide the unique opportunity to selectively entrain the low- or high-frequency component and study subsequent effects on CFC. Here, we demonstrate the differential modulation of CFC during selective entrainment of alpha or gamma oscillations. Our results reveal that entrainment of the low-frequency component increased PAC, where gamma power became preferentially locked to the trough of the alpha oscillation, while gamma-band entrainment enhanced AECs and reduced alpha power. These results provide causal evidence for the functional role of coupled alpha and gamma oscillations for visual processing.