Terrence R. Stanford

Multisensory integration: current issues from the perspective of the single neuron

For thousands of years science philosophers have been impressed by how effectively the senses work together to enhance the salience of biologically meaningful events. However, they really had no idea how this was accomplished. Recent insights into the underlying physiological mechanisms reveal that, in at least one circuit, this ability depends on an intimate dialogue among neurons at multiple levels of the neuraxis; this dialogue cannot take place until long after birth and might require a specific kind of experience. Understanding the acquisition and usage of multisensory integration in the midbrain and cerebral cortex of mammals has been aided by a multiplicity of approaches. Here we examine some of the fundamental advances that have been made and some of the challenging questions that remain.

Subcortical loops through the basal ganglia

Parallel, largely segregated, closed-loop projections are an important component of cortical-basal ganglia-cortical connectional architecture. Here, we present the hypothesis that such loops involving the neocortex are neither novel nor the first evolutionary example of closed-loop architecture involving the basal ganglia. Specifically, we propose that a phylogenetically older, closed-loop series of subcortical connections exists between the basal ganglia and brainstem sensorimotor structures, a good example of which is the midbrain superior colliculus. Insofar as this organization represents a general feature of brain architecture, cortical and subcortical inputs to the basal ganglia might act independently, co-operatively or competitively to influence the mechanisms of action selection.

On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies

A growing number of brain imaging studies are being undertaken in order to better understand the contributions of multisensory processes to human behavior and perception. Many of these studies are designed on the basis of the physiological findings from single neurons in animal models, which have shown that multisensory neurons have the capacity for integrating their different sensory inputs and give rise to a product that differs significantly from either of the unisensory responses. At certain points these multisensory interactions can be superadditive, resulting in a neural response that exceeds the sum of the unisensory responses. Because of the difficulties inherent in interpreting the results of imaging large neuronal populations, superadditivity has been put forth as a stringent criterion for identifying potential sites of multisensory integration. In the present manuscript we discuss issues related to using the superadditive model in human brain imaging studies, focusing on population responses to multisensory stimuli and the relationship between single neuron measures and functional brain imaging measures. We suggest that the results of brain imaging studies be interpreted with caution in regards to multisensory integration. Future directions for imaging multisensory integration are discussed in light of the ideas presented.

Intracellular Recordings in Response to Monaural and Binaural Stimulation of Neurons in the Inferior Colliculus of the Cat

The inferior colliculus (IC) is a major auditory structure that integrates synaptic inputs from ascending, descending, and intrinsic sources. Intracellular recording in situ allows direct examination of synaptic inputs to the IC in response to acoustic stimulation. Using this technique and monaural or binaural stimulation, responses in the IC that reflect input from a lower center can be distinguished from responses that reflect synaptic integration within the IC. Our results indicate that many IC neurons receive synaptic inputs from multiple sources. Few, if any, IC neurons acted as simple relay cells. Responses often displayed complex interactions between excitatory and inhibitory sources, such that different synaptic mechanisms could underlie similar response patterns. Thus, it may be an oversimplification to classify the responses of IC neurons as simply excitatory or inhibitory, as is done in many studies. In addition, inhibition and intrinsic membrane properties appeared to play key roles in creating de novo temporal response patterns in the IC.

Perceptual decision making in less than 30 milliseconds

In perceptual discrimination tasks, a subjects response time is determined by both sensory and motor processes. Measuring the time consumed by the perceptual evaluation step alone is therefore complicated by factors such as motor preparation, task difficulty and speed-accuracy tradeoffs. Here we present a task design that minimizes these confounding factors and allows us to track a subjects perceptual performance with unprecedented temporal resolution. We find that monkeys can make accurate color discriminations in less than 30 ms. Furthermore, our simple task design provides a tool for elucidating how neuronal activity relates to sensory as opposed to motor processing, as demonstrated with neural data from cortical oculomotor neurons. In these cells, perceptual information acts by accelerating and decelerating the ongoing motor plans associated with correct and incorrect choices, as predicted by a race-to-threshold model, and the time course of these neural events parallels the time course of the subjects choice accuracy.

Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness

Single-neuron studies provide a foundation for understanding many facets of multisensory integration. These studies have used a variety of criteria for identifying and quantifying multisensory integration. While a number of techniques have been used, an explicit discussion of the assumptions, criteria, and analytical methods traditionally used to define the principles of multisensory integration is lacking. This was not problematic when the field was small, but with rapid growth a number of alternative techniques and models have been introduced, each with its own criteria and sets of implicit assumptions to define and characterize what is thought to be the same phenomenon. The potential for misconception prompted this reexamination of traditional approaches in order to clarify their underlying assumptions and analytic techniques. The objective here is to review and discuss traditional quantitative methods advanced in the study of single-neuron physiology in order to appreciate the process of multisensory integration and its impact.

Multisensory Integration Shortens Physiological Response Latencies

Individual superior colliculus (SC) neurons integrate information from multiple sensory sources to enhance their physiological response. The response of an SC neuron to a cross-modal stimulus combination can not only exceed the best component unisensory response but can also exceed their arithmetic sum (i.e., superadditivity). The present experiments were designed to investigate the temporal profile of multisensory integration in this model system. We found that cross-modal stimuli frequently shortened physiological response latencies (mean shift, 6.2 ms) and that response enhancement was greatest in the initial phase of the response (the phenomenon of initial response enhancement). The vast majority of the responses studied evidenced superadditive computations, most often at the beginning of the multisensory response.

Development of multisensory integration from the perspective of the individual neuron

The ability to use cues from multiple senses in concert is a fundamental aspect of brain function. It maximizes the brains use of the information available to it at any given moment and enhances the physiological salience of external events. Because each sense conveys a unique perspective of the external world, synthesizing information across senses affords computational benefits that cannot otherwise be achieved. Multisensory integration not only has substantial survival value but can also create unique experiences that emerge when signals from different sensory channels are bound together. However, neurons in a newborns brain are not capable of multisensory integration, and studies in the midbrain have shown that the development of this process is not predetermined. Rather, its emergence and maturation critically depend on cross-modal experiences that alter the underlying neural circuit in such a way that optimizes multisensory integrative capabilities for the environment in which the animal will function.

The neural basis of multisensory integration in the midbrain: its organization and maturation.

Multisensory integration describes a process by which information from different sensory systems is combined to influence perception, decisions, and overt behavior. Despite a widespread appreciation of its utility in the adult, its developmental antecedents have received relatively little attention. Here we review what is known about the development of multisensory integration, with a focus on the circuitry and experiential antecedents of its development in the model system of the multisensory (i.e., deep) layers of the superior colliculus. Of particular interest here are two sets of experimental observations: (1) cortical influences appear essential for multisensory integration in the SC, and (2) postnatal experience guides its maturation. The current belief is that the experience normally gained during early life is instantiated in the cortico-SC projection, and that this is the primary route by which ecological pressures adapt SC multisensory integration to the particular environment in which it will be used.

Stimulus Selectivity in Dorsal and Ventral Prefrontal Cortex after Training in Working Memory Tasks

The prefrontal cortex is known to represent different types of information in working memory. Contrasting theories propose that the dorsal and ventral regions of the lateral prefrontal cortex are innately specialized for the representation of spatial and nonspatial information, respectively (Goldman-Rakic, 1996), or that the two regions are shaped by the demands of cognitive tasks imposed on them (Miller, 2000). To resolve this issue, we recorded from neurons in the two regions, before and at multiple stages of training monkeys on visual working memory tasks. Before training, substantial functional differences were present between the two regions. Dorsal prefrontal cortex exhibited higher overall responsiveness to visual stimuli and higher selectivity for spatial information. After training, stimulus selectivity generally decreased, although dorsal prefrontal cortex retained higher spatial selectivity regardless of task performed. Ventral prefrontal cortex appeared to be affected to a greater extent by the nature of the task. Our results indicate that regional specialization for stimulus selectivity is present in the primate prefrontal cortex regardless of training. Dorsal areas of the prefrontal cortex are inherently organized to represent spatial information, and training has little influence on this spatial bias. Ventral areas are biased toward nonspatial information, although they are more influenced by training both in terms of activation and changes in stimulus selectivity.