Jennifer K. Bizley

Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex

Recent studies, conducted almost exclusively in primates, have shown that several cortical areas usually associated with modality-specific sensory processing are subject to influences from other senses. Here we demonstrate using single-unit recordings and estimates of mutual information that visual stimuli can influence the activity of units in the auditory cortex of anesthetized ferrets. In many cases, these units were also acoustically responsive and frequently transmitted more information in their spike discharge patterns in response to paired visual-auditory stimulation than when either modality was presented by itself. For each stimulus, this information was conveyed by a combination of spike count and spike timing. Even in primary auditory areas (primary auditory cortex [A1] and anterior auditory field [AAF]), ~15% of recorded units were found to have nonauditory input. This proportion increased in the higher level fields that lie ventral to A1/AAF and was highest in the anterior ventral field, where nearly 50% of the units were found to be responsive to visual stimuli only and a further quarter to both visual and auditory stimuli. Within each field, the pure-tone response properties of neurons sensitive to visual stimuli did not differ in any systematic way from those of visually unresponsive neurons. Neural tracer injections revealed direct inputs from visual cortex into auditory cortex, indicating a potential source of origin for the visual responses. Primary visual cortex projects sparsely to A1, whereas higher visual areas innervate auditory areas in a field-specific manner. These data indicate that multisensory convergence and integration are features common to all auditory cortical areas but are especially prevalent in higher areas.

Interdependent Encoding of Pitch, Timbre, and Spatial Location in Auditory Cortex

Because we can perceive the pitch, timbre, and spatial location of a sound source independently, it seems natural to suppose that cortical processing of sounds might separate out spatial from nonspatial attributes. Indeed, recent studies support the existence of anatomically segregated “what” and “where” cortical processing streams. However, few attempts have been made to measure the responses of individual neurons in different cortical fields to sounds that vary simultaneously across spatial and nonspatial dimensions. We recorded responses to artificial vowels presented in virtual acoustic space to investigate the representations of pitch, timbre, and sound source azimuth in both core and belt areas of ferret auditory cortex. A variance decomposition technique was used to quantify the way in which altering each parameter changed neural responses. Most units were sensitive to two or more of these stimulus attributes. Although indicating that neural encoding of pitch, location, and timbre cues is distributed across auditory cortex, significant differences in average neuronal sensitivity were observed across cortical areas and depths, which could form the basis for the segregation of spatial and nonspatial cues at higher cortical levels. Some units exhibited significant nonlinear interactions between particular combinations of pitch, timbre, and azimuth. These interactions were most pronounced for pitch and timbre and were less commonly observed between spatial and nonspatial attributes. Such nonlinearities were most prevalent in primary auditory cortex, although they tended to be small compared with stimulus main effects.

Neural Ensemble Codes for Stimulus Periodicity in Auditory Cortex

We measured the responses of neurons in auditory cortex of male and female ferrets to artificial vowels of varying fundamental frequency (f0), or periodicity, and compared these with the performance of animals trained to discriminate the periodicity of these sounds. Sensitivity to f0 was found in all five auditory cortical fields examined, with most of those neurons exhibiting either low-pass or high-pass response functions. Only rarely was the stimulus dependence of individual neuron discharges sufficient to account for the discrimination performance of the ferrets. In contrast, when analyzed with a simple classifier, responses of small ensembles, comprising 3-61 simultaneously recorded neurons, often discriminated periodicity changes as well as the animals did. We examined four potential strategies for decoding ensemble responses: spike counts, relative first-spike latencies, a binary “spike or no-spike” code, and a spike-order code. All four codes represented stimulus periodicity effectively, and, surprisingly, the spike count and relative latency codes enabled an equally rapid readout, within 75 ms of stimulus onset. Thus, relative latency codes do not necessarily facilitate faster discrimination judgments. A joint spike count plus relative latency code was more informative than either code alone, indicating that the information captured by each measure was not wholly redundant. The responses of neural ensembles, but not of single neurons, reliably encoded f0 changes even when stimulus intensity was varied randomly over a 20 dB range. Because trained animals can discriminate stimulus periodicity across different sound levels, this implies that ensemble codes are better suited to account for behavioral performance.

An investigation of the role of auditory cortex in sound localization using muscimol-releasing Elvax.

Lesion studies suggest that primary auditory cortex (A1) is required for accurate sound localization by carnivores and primates. In order to elucidate further its role in spatial hearing, we examined the behavioural consequences of reversibly inactivating ferret A1 over long periods, using Elvax implants releasing the GABAA receptor agonist muscimol. Sub‐dural polymer placements were shown to deliver relatively constant levels of muscimol to underlying cortex for >5 months. The measured diffusion of muscimol beneath and around the implant was limited to 1 mm. Cortical silencing was assessed electrophysiologically in both auditory and visual cortices. This exhibited rapid onset and was reversed within a few hours of implant removal. Inactivation of cortical neurons extended to all layers for implants lasting up to 6 weeks and throughout at least layers I–IV for longer placements, whereas thalamic activity in layer IV appeared to be unaffected. Blockade of cortical neurons in the deeper layers was restricted to ≤ 500 µm from the edge of the implant, but was usually more widespread in the superficial layers. In contrast, drug‐free Elvax implants had little discernible effect on the responses of the underlying cortical neurons. Bilateral implants of muscimol–Elvax over A1 produced significant deficits in the localization of brief sounds in horizontal space and particularly a reduced ability to discriminate between anterior and posterior sound sources. The performance of these ferrets gradually improved over the period in which the Elvax was in place and attained that of control animals following its removal. Although similar in nature, these deficits were less pronounced than those caused by cortical lesions and suggest a specific role for A1 in resolving the spatial ambiguities inherent in auditory localization cues.

Physiological and behavioral studies of spatial coding in the auditory cortex.

Despite extensive subcortical processing, the auditory cortex is believed to be essential for normal sound localization. However, we still have a poor understanding of how auditory spatial information is encoded in the cortex and of the relative contribution of different cortical areas to spatial hearing. We investigated the behavioral consequences of inactivating ferret primary auditory cortex (A1) on auditory localization by implanting a sustained release polymer containing the GABA(A) agonist muscimol bilaterally over A1. Silencing A1 led to a reversible deficit in the localization of brief noise bursts in both the horizontal and vertical planes. In other ferrets, large bilateral lesions of the auditory cortex, which extended beyond A1, produced more severe and persistent localization deficits. To investigate the processing of spatial information by high-frequency A1 neurons, we measured their binaural-level functions and used individualized virtual acoustic space stimuli to record their spatial receptive fields (SRFs) in anesthetized ferrets. We observed the existence of a continuum of response properties, with most neurons preferring contralateral sound locations. In many cases, the SRFs could be explained by a simple linear interaction between the acoustical properties of the head and external ears and the binaural frequency tuning of the neurons. Azimuth response profiles recorded in awake ferrets were very similar and further analysis suggested that the slopes of these functions and location-dependent variations in spike timing are the main information-bearing parameters. Studies of sensory plasticity can also provide valuable insights into the role of different brain areas and the way in which information is represented within them. For example, stimulus-specific training allows accurate auditory localization by adult ferrets to be relearned after manipulating binaural cues by occluding one ear. Reversible inactivation of A1 resulted in slower and less complete adaptation than in normal controls, whereas selective lesions of the descending cortico collicular pathway prevented any improvement in performance. These results reveal a role for auditory cortex in training-induced plasticity of auditory localization, which could be mediated by descending cortical pathways.

Visual influences on ferret auditory cortex.

Multisensory neurons are now known to be widespread in low-level regions of the cortex usually thought of as being responsible for modality-specific processing. The auditory cortex provides a particularly striking example of this, exhibiting responses to both visual and somatosensory stimulation. Single-neuron recording studies in ferrets have shown that each of auditory fields that have been characterized using physiological and anatomical criteria also receives visual inputs, with the incidence of visually-sensitive neurons ranging from 15% to 20% in the primary areas to around 50% or more in higher-level areas. Although some neurons exhibit spiking responses to visual stimulation, these inputs often have subthreshold influences that modulate the responses of the cortical neurons to sound. Insights into the possible role played by the visual inputs can be obtained by examining their sources of origin and the way in which they alter the processing capabilities of neurons in the auditory cortex. These studies suggest that one of the functions of the visual input to auditory cortex is to sharpen the relatively imprecise spatial coding typically found there. Because the extent to which this happens varies between cortical fields, the investigation of multisensory interactions can also help in understanding their relative contributions to auditory perception.

Auditory Cortex Represents Both Pitch Judgments and the Corresponding Acoustic Cues

Summary The neural processing of sensory stimuli involves a transformation of physical stimulus parameters into perceptual features, and elucidating where and how this transformation occurs is one of the ultimate aims of sensory neurophysiology. Recent studies have shown that the firing of neurons in early sensory cortex can be modulated by multisensory interactions [1–5], motor behavior [1, 3, 6, 7], and reward feedback [1, 8, 9], but it remains unclear whether neural activity is more closely tied to perception, as indicated by behavioral choice, or to the physical properties of the stimulus. We investigated which of these properties are predominantly represented in auditory cortex by recording local field potentials (LFPs) and multiunit spiking activity in ferrets while they discriminated the pitch of artificial vowels. We found that auditory cortical activity is informative both about the fundamental frequency (F0) of a target sound and also about the pitch that the animals appear to perceive given their behavioral responses. Surprisingly, although the stimulus F0 was well represented at the onset of the target sound, neural activity throughout auditory cortex frequently predicted the reported pitch better than the target F0.

Responses of Auditory Cortex to Complex Stimuli: Functional Organization Revealed Using Intrinsic Optical Signals

We used optical imaging of intrinsic signals to study the large-scale organization of ferret auditory cortex in response to complex sounds. Cortical responses were collected during continuous stimulation by sequences of sounds with varying frequency, period, or interaural level differences. We used a set of stimuli that differ in spectral structure, but have the same periodicity and therefore evoke the same pitch percept (click trains, sinusoidally amplitude modulated tones, and iterated ripple noise). These stimuli failed to reveal a consistent periodotopic map across the auditory fields imaged. Rather, gradients of period sensitivity differed for the different types of periodic stimuli. Binaural interactions were studied both with single contralateral, ipsilateral, and diotic broadband noise bursts and with sequences of broadband noise bursts with varying level presented contralaterally, ipsilaterally, or in opposite phase to both ears. Contralateral responses were generally largest and ipsilateral responses were smallest when using single noise bursts, but the extent of the activated area was large and comparable in all three aural configurations. Modulating the amplitude in counter phase to the two ears generally produced weaker modulation of the optical signals than the modulation produced by the monaural stimuli. These results suggest that binaural interactions seen in cortex are most likely predominantly due to subcortical processing. Thus our optical imaging data do not support the theory that the primary or nonprimary cortical fields imaged are topographically organized to form consistent maps of systematically varying sensitivity either to stimulus pitch or to simple binaural properties of the acoustic stimuli.

Lesions of the auditory cortex impair azimuthal sound localization and its recalibration in ferrets.

The role of auditory cortex in sound localization and its recalibration by experience was explored by measuring the accuracy with which ferrets turned toward and approached the source of broadband sounds in the horizontal plane. In one group, large bilateral lesions were made of the middle ectosylvian gyrus, where the primary auditory cortical fields are located, and part of the anterior and/or posterior ectosylvian gyrus, which contain higher-level fields. In the second group, the lesions were intended to be confined to primary auditory cortex (A1). The ability of the animals to localize noise bursts of different duration and level was measured before and after the lesions were made. A1 lesions produced a modest disruption of approach-to-target responses to short-duration stimuli (<500 ms) on both sides of space, whereas head orienting accuracy was unaffected. More extensive lesions produced much greater auditory localization deficits, again primarily for shorter sounds. In these ferrets, the accuracy of both the approach-to-target behavior and the orienting responses was impaired, and they could do little more than correctly lateralize the stimuli. Although both groups of ferrets were still able to localize long-duration sounds accurately, they were, in contrast to ferrets with an intact auditory cortex, unable to relearn to localize these stimuli after altering the spatial cues available by reversibly plugging one ear. These results indicate that both primary and nonprimary cortical areas are necessary for normal sound localization, although only higher auditory areas seem to contribute to accurate head orienting behavior. They also show that the auditory cortex, and A1 in particular, plays an essential role in training-induced plasticity in adult ferrets, and that this is the case for both head orienting responses and approach-to-target behavior.

Cortical encoding of pitch: Recent results and open questions

It is widely appreciated that the key predictor of the pitch of a sound is its periodicity. Neural structures which support pitch perception must therefore be able to reflect the repetition rate of a sound, but this alone is not sufficient. Since pitch is a psychoacoustic property, a putative cortical code for pitch must also be able to account for the relationship between the amount to which a sound is periodic (i.e. its temporal regularity) and the perceived pitch salience, as well as limits in our ability to detect pitch changes or to discriminate rising from falling pitch. Pitch codes must also be robust in the presence of nuisance variables such as loudness or timbre. Here, we review a large body of work on the cortical basis of pitch perception, which illustrates that the distribution of cortical processes that give rise to pitch perception is likely to depend on both the acoustical features and functional relevance of a sound. While previous studies have greatly advanced our understanding, we highlight several open questions regarding the neural basis of pitch perception. These questions can begin to be addressed through a cooperation of investigative efforts across species and experimental techniques, and, critically, by examining the responses of single neurons in behaving animals.