Mark T. Wallace

Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli

Visual and auditory cortices traditionally have been considered to be modality-specific. Thus, their activity has been thought to be unchanged by information in other sensory modalities. However, using functional magnetic resonance imaging (fMRI), the present experiments revealed that ongoing activity in the visual cortex could be modulated by auditory information and ongoing activity in the auditory cortex could be modulated by visual information. In both cases, this cross-modal modulation of activity took the form of deactivation. Yet, the deactivation response was not evident in either cortical area during the paired presentation of visual and auditory stimuli. These data suggest that cross-modal inhibitory processes operate within traditional modality-specific cortices and that these processes can be switched on or off in different circumstances.

A revised view of sensory cortical parcellation

Traditional cortical parcellation schemes have emphasized the presence of sharply defined visual, auditory, and somatosensory domains populated exclusively by modality-specific neurons (i.e., neurons responsive to sensory stimuli from a single sensory modality). However, the modality-exclusivity of this scheme has recently been challenged. Observations in a variety of species suggest that each of these domains is subject to influences from other senses. Using the cerebral cortex of the rat as a model, the present study systematically examined the capability of individual neurons in visual, auditory, and somatosensory cortex to be activated by stimuli from other senses. Within the major modality-specific domains, the incidence of inappropriate (i.e., nonmatching) and/or multisensory neurons was very low. However, at the borders between each of these domains a concentration of multisensory neurons was found whose modality profile matched the representations in neighboring cortices and that were able to integrate their cross-modal inputs to give rise to enhanced and/or depressed responses. The results of these studies are consistent with some features of both the traditional and challenging views of cortical organization, and they suggest a parcellation scheme in which modality-specific cortical domains are separated from one another by transitional multisensory zones.

Enhanced multisensory integration in older adults

Information from the different senses is seamlessly integrated by the brain in order to modify our behaviors and enrich our perceptions. It is only through the appropriate binding and integration of information from the different senses that a meaningful and accurate perceptual gestalt can be generated. Although a great deal is known about how such cross-modal interactions influence behavior and perception in the adult, there is little knowledge as to the impact of aging on these multisensory processes. In the current study, we examined the speed of discrimination responses of aged and young individuals to the presentation of visual, auditory or combined visual-auditory stimuli. Although the presentation of multisensory stimuli speeded response times in both groups, the performance gain was significantly greater in the aged. Most strikingly, multisensory stimuli restored response times in the aged to those seen in young subjects to the faster of the two unisensory stimuli (i.e., visual). The current results suggest that despite the decline in sensory processing that accompanies aging, the use of multiple sensory channels may represent an effective compensatory strategy to overcome these unisensory deficits.

Integration of multiple sensory modalities in cat cortex

SummaryThe results of this study show that the different receptive fields of multisensory neurons in the cortex of the cat anterior ectosylvian sulcus (AES) were in spatial register, and it is this register that determined the manner in which these neurons integrated multiple sensory stimuli. The functional properties of multisensory neurons in AES cortex bore fundamental similarities to those in other cortical and subcortical structures. These constancies in the principles of multisensory integration are likely to provide a basis for spatial coherence in information processing throughout the nervous system.

An extended multisensory temporal binding window in autism spectrum disorders

Autism spectrum disorders (ASD) form a continuum of neurodevelopmental disorders, characterized by deficits in communication and reciprocal social interaction, as well as by repetitive behaviors and restricted interests. Sensory disturbances are also frequently reported in clinical and autobiographical accounts. However, surprisingly few empirical studies have characterized the fundamental features of sensory and multisensory processing in ASD. The current study is structured to test for potential differences in multisensory temporal function in ASD by making use of a temporally dependent, low-level multisensory illusion. In this illusion, the presentation of a single flash of light accompanied by multiple sounds often results in the illusory perception of multiple flashes. By systematically varying the temporal structure of the audiovisual stimuli, a “temporal window” within which these stimuli are likely to be bound into a single perceptual entity can be defined. The results of this study revealed that children with ASD report the flash-beep illusion over an extended range of stimulus onset asynchronies relative to children with typical development, suggesting that children with ASD have altered multisensory temporal function. These findings provide valuable new insights into our understanding of sensory processing in ASD and may hold promise for the development of more sensitive diagnostic measures and improved remediation strategies.

Semantic congruence is a critical factor in multisensory behavioral performance

It has repeatedly been demonstrated that the presence of multiple cues in different sensory modalities can enhance behavioral performance by speeding responses, increasing accuracy, and/or improving stimulus detection. Despite an extensive knowledge base as to how the spatial, temporal, and physical (e.g., intensity) characteristics of multisensory stimuli influence such enhancements, little is known about the role of semantic or contextual congruence. Our hypothesis was that semantically congruent multisensory stimuli would result in enhanced behavioral performance, and that semantically incongruent multisensory stimuli would result in either no enhancement or a decrement in behavioral performance. The results from a redundant cue feature discrimination task clearly demonstrate that congruent cross-modal stimulation improves behavioral performance. This effect is specific to the multisensory stimuli, as no improvements are seen in the presence of redundant unimodal stimulus pairs. In contrast, incongruent stimulus pairs result in behavioral decrements for both multisensory and paired unimodal stimuli. These results highlight that in addition to such simple stimulus features as space, time and relative effectiveness, the semantic content of a multisensory stimulus plays a critical role in determining how it is processed by the nervous system.

Altered Auditory and Multisensory Temporal Processing in Autism Spectrum Disorders

Autism spectrum disorders (ASD) are characterized by deficits in social reciprocity and communication, as well as by repetitive behaviors and restricted interests. Unusual responses to sensory input and disruptions in the processing of both unisensory and multisensory stimuli also have been reported frequently. However, the specific aspects of sensory processing that are disrupted in ASD have yet to be fully elucidated. Recent published work has shown that children with ASD can integrate low-level audiovisual stimuli, but do so over an extended range of time when compared with typically developing (TD) children. However, the possible contributions of altered unisensory temporal processes to the demonstrated changes in multisensory function are yet unknown. In the current study, unisensory temporal acuity was measured by determining individual thresholds on visual and auditory temporal order judgment (TOJ) tasks, and multisensory temporal function was assessed through a cross-modal version of the TOJ task. Whereas no differences in thresholds for the visual TOJ task were seen between children with ASD and TD, thresholds were higher in ASD on the auditory TOJ task, providing preliminary evidence for impairment in auditory temporal processing. On the multisensory TOJ task, children with ASD showed performance improvements over a wider range of temporal intervals than TD children, reinforcing prior work showing an extended temporal window of multisensory integration in ASD. These findings contribute to a better understanding of basic sensory processing differences, which may be critical for understanding more complex social and cognitive deficits in ASD, and ultimately may contribute to more effective diagnostic and interventional strategies.

An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection.

Physiological and behavioral studies in animal models have revealed that information from the different senses can be used synergistically to enhance the detection of objects and events. Although a great deal of evidence exists which demonstrates the synergistic use of multisensory cues in human behavior and perception, there is conflicting evidence about whether such interactions can be used to aid in stimulus detection. To examine this issue, we had volunteers indicate the presence or absence of a brief, low-intensity sound that was either presented alone or paired with a simultaneous light in a one-interval signal detection task. In the first experiment, the task-irrelevant light was found to enhance the detectability of the sound, but also to increase the subjects willingness to report the presence of a sound (i.e., response bias). In the second experiment, designed to eliminate this response bias, we found an improvement in stimulus detectability in the absence of any change in bias. These findings demonstrate the presence of a significant multisensory-mediated gain in stimulus detection in human subjects.

Perceptual training narrows the temporal window of multisensory binding

The brains ability to bind incoming auditory and visual stimuli depends critically on the temporal structure of this information. Specifically, there exists a temporal window of audiovisual integration within which stimuli are highly likely to be bound together and perceived as part of the same environmental event. Several studies have described the temporal bounds of this window, but few have investigated its malleability. Here, the plasticity in the size of this temporal window was investigated using a perceptual learning paradigm in which participants were given feedback during a two-alternative forced choice (2-AFC) audiovisual simultaneity judgment task. Training resulted in a marked (i.e., ∼40%) narrowing in the size of the window. To rule out the possibility that this narrowing was the result of changes in cognitive biases, a second experiment using a two-interval forced choice (2-IFC) paradigm was undertaken during which participants were instructed to identify a simultaneously presented audiovisual pair presented within one of two intervals. The 2-IFC paradigm resulted in a narrowing that was similar in both degree and dynamics to that using the 2-AFC approach. Together, these results illustrate that different methods of multisensory perceptual training can result in substantial alterations in the circuits underlying the perception of audiovisual simultaneity. These findings suggest a high degree of flexibility in multisensory temporal processing and have important implications for interventional strategies that may be used to ameliorate clinical conditions (e.g., autism, dyslexia) in which multisensory temporal function may be impaired.

Multisensory Temporal Integration in Autism Spectrum Disorders

The new DSM-5 diagnostic criteria for autism spectrum disorders (ASDs) include sensory disturbances in addition to the well-established language, communication, and social deficits. One sensory disturbance seen in ASD is an impaired ability to integrate multisensory information into a unified percept. This may arise from an underlying impairment in which individuals with ASD have difficulty perceiving the temporal relationship between cross-modal inputs, an important cue for multisensory integration. Such impairments in multisensory processing may cascade into higher-level deficits, impairing day-to-day functioning on tasks, such as speech perception. To investigate multisensory temporal processing deficits in ASD and their links to speech processing, the current study mapped performance on a number of multisensory temporal tasks (with both simple and complex stimuli) onto the ability of individuals with ASD to perceptually bind audiovisual speech signals. High-functioning children with ASD were compared with a group of typically developing children. Performance on the multisensory temporal tasks varied with stimulus complexity for both groups; less precise temporal processing was observed with increasing stimulus complexity. Notably, individuals with ASD showed a speech-specific deficit in multisensory temporal processing. Most importantly, the strength of perceptual binding of audiovisual speech observed in individuals with ASD was strongly related to their low-level multisensory temporal processing abilities. Collectively, the results represent the first to illustrate links between multisensory temporal function and speech processing in ASD, strongly suggesting that deficits in low-level sensory processing may cascade into higher-order domains, such as language and communication.