Jean Mary Zarate

The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts

Speech contains temporal structure that the brain must analyze to enable linguistic processing. To investigate the neural basis of this analysis, we used sound quilts, stimuli constructed by shuffling segments of a natural sound, approximately preserving its properties on short timescales while disrupting them on longer scales. We generated quilts from foreign speech to eliminate language cues and manipulated the extent of natural acoustic structure by varying the segment length. Using functional magnetic resonance imaging, we identified bilateral regions of the superior temporal sulcus (STS) whose responses varied with segment length. This effect was absent in primary auditory cortex and did not occur for quilts made from other natural sounds or acoustically matched synthetic sounds, suggesting tuning to speech-specific spectrotemporal structure. When examined parametrically, the STS response increased with segment length up to ∼500 ms. Our results identify a locus of speech analysis in human auditory cortex that is distinct from lexical, semantic or syntactic processes.

An acoustical study of vocal pitch matching in congenital amusia

Vocal pitch matching is a foundational skill for singing and is an interesting place to study the relationship between pitch perception and production. To better understand this relationship, we assessed pitch-matching abilities in congenital amusics, who have documented disabilities in pitch perception, and in matched controls under normal, masked, and guided feedback conditions. Their vocal productions were analyzed for fundamental frequency and showed that amusics were significantly less accurate at pitch matching than the controls. However, five of the six amusics showed a significant correlation between their produced pitches and the target pitch. Feedback condition had no effect on pitch-matching accuracy. These results show impaired vocal pitch-matching abilities in amusics but also show a relationship between perceived and produced pitches.

The neural control of singing

Singing provides a unique opportunity to examine music performance—the musical instrument is contained wholly within the body, thus eliminating the need for creating artificial instruments or tasks in neuroimaging experiments. Here, more than two decades of voice and singing research will be reviewed to give an overview of the sensory-motor control of the singing voice, starting from the vocal tract and leading up to the brain regions involved in singing. Additionally, to demonstrate how sensory feedback is integrated with vocal motor control, recent functional magnetic resonance imaging (fMRI) research on somatosensory and auditory feedback processing during singing will be presented. The relationship between the brain and singing behavior will be explored also by examining: (1) neuroplasticity as a function of various lengths and types of training, (2) vocal amusia due to a compromised singing network, and (3) singing performance in individuals with congenital amusia. Finally, the auditory-motor control network for singing will be considered alongside dual-stream models of auditory processing in music and speech to refine both these theoretical models and the singing network itself.

Pitch-interval discrimination and musical expertise: Is the semitone a perceptual boundary?

The ability to discriminate pitch changes (or intervals) is foundational for speech and music. In an auditory psychophysical experiment, musicians and non-musicians were tested with fixed- and roving-pitch discrimination tasks to investigate the effects of musical expertise on interval discrimination. The tasks were administered parametrically to assess performance across varying pitch distances between intervals. Both groups showed improvements in fixed-pitch interval discrimination as a function of increasing interval difference. Only musicians showed better roving-pitch interval discrimination as interval differences increased, suggesting that this task was too demanding for non-musicians. Musicians had better interval discrimination than non-musicians across most interval differences in both tasks. Interestingly, musicians exhibited improved interval discrimination starting at interval differences of 100 cents (a semitone in Western music), whereas non-musicians showed enhanced discrimination at interval differences exceeding 125 cents. Although exposure to Western music and speech may help establish a basic interval-discrimination threshold between 100 and 200 cents (intervals that occur often in Western languages and music), musical training presumably enhances auditory processing and reduces this threshold to a semitone. As musical expertise does not decrease this threshold beyond 100 cents, the semitone may represent a musical training-induced intervallic limit to acoustic processing.

Modulation of Auditory Cortex Response to Pitch Variation Following Training with Microtonal Melodies

We tested changes in cortical functional response to auditory patterns in a configural learning paradigm. We trained 10 human listeners to discriminate micromelodies (consisting of smaller pitch intervals than normally used in Western music) and measured covariation in blood oxygenation signal to increasing pitch interval size in order to dissociate global changes in activity from those specifically associated with the stimulus feature that was trained. A psychophysical staircase procedure with feedback was used for training over a 2-week period. Behavioral tests of discrimination ability performed before and after training showed significant learning on the trained stimuli, and generalization to other frequencies and tasks; no learning occurred in an untrained control group. Before training the functional MRI data showed the expected systematic increase in activity in auditory cortices as a function of increasing micromelody pitch interval size. This function became shallower after training, with the maximal change observed in the right posterior auditory cortex. Global decreases in activity in auditory regions, along with global increases in frontal cortices also occurred after training. Individual variation in learning rate was related to the hemodynamic slope to pitch interval size, such that those who had a higher sensitivity to pitch interval variation prior to learning achieved the fastest learning. We conclude that configural auditory learning entails modulation in the response of auditory cortex to the trained stimulus feature. Reduction in blood oxygenation response to increasing pitch interval size suggests that fewer computational resources, and hence lower neural recruitment, is associated with learning, in accord with models of auditory cortex function, and with data from other modalities.

Mental imagery of speech implicates two mechanisms of perceptual reactivation

Sensory cortices can be activated without any external stimuli. Yet, it is still unclear how this perceptual reactivation occurs and which neural structures mediate this reconstruction process. In this study, we employed fMRI with mental imagery paradigms to investigate the neural networks involved in perceptual reactivation. Subjects performed two speech imagery tasks: articulation imagery (AI) and hearing imagery (HI). We found that AI induced greater activity in frontal-parietal sensorimotor systems, including sensorimotor cortex, subcentral (BA 43), middle frontal cortex (BA 46) and parietal operculum (PO), whereas HI showed stronger activation in regions that have been implicated in memory retrieval: middle frontal (BA 8), inferior parietal cortex and intraparietal sulcus. Moreover, posterior superior temporal sulcus (pSTS) and anterior superior temporal gyrus (aSTG) was activated more in AI compared with HI, suggesting that covert motor processes induced stronger perceptual reactivation in the auditory cortices. These results suggest that motor-to-perceptual transformation and memory retrieval act as two complementary mechanisms to internally reconstruct corresponding perceptual outcomes. These two mechanisms can serve as a neurocomputational foundation for predicting perceptual changes, either via a previously learned relationship between actions and their perceptual consequences or via stored perceptual experiences of stimulus and episodic or contextual regularity.

Cortical Processing of Music

Here’s a commonplace experience: you are walking in a shopping mall when you hear a tune being played in the background. It takes you a moment but then you realize that it is a song that you last heard 20 years ago, which has now been redone—perhaps unfortunately—as an advertising jingle. Although the aesthetic experience associated with this little vignette may not be high, the ease with which our nervous system can carry out this kind of analysis belies the complexity involved. Consider: the music you hear is embedded in a background of irrelevant noise, so you need first to strip it away; you recognize the pattern of sound as the tune you are familiar with, even though none of the actual elements reaching your ear are the same as what you had originally encoded—the tempo, musical key, and instrument timbres may all be different; if the song has lyrics you must also separate the tonal component from the speech component to process each of them; the experience may also lead to retrieval of memories associated with the song; you could also begin to sing along with it, which means you must convert the information contained in the sound waves you hear to a set of motor commands that will produce similar sound waves from your vocal musculature; finally the song may lead you to experience emotion, which could range from annoyance to pleasure. The mechanisms that allow this complex cognitive chain of events to occur are far from being fully understood. This chapter aims to give readers an overview of what is known about the role of auditory cortex in processing and production of musical sounds, and an indication of the many open questions that remain. Understanding the neural and cognitive mechanisms involved in tonal and musical processes will yield insights into fundamental aspects of neural organization and function that would otherwise be difficult to obtain.

Multiple levels of linguistic and paralinguistic features contribute to voice recognition

Voice or speaker recognition is critical in a wide variety of social contexts. In this study, we investigated the contributions of acoustic, phonological, lexical, and semantic information toward voice recognition. Native English speaking participants were trained to recognize five speakers in five conditions: non-speech, Mandarin, German, pseudo-English, and English. We showed that voice recognition significantly improved as more information became available, from purely acoustic features in non-speech to additional phonological information varying in familiarity. Moreover, we found that the recognition performance is transferable between training and testing in phonologically familiar conditions (German, pseudo-English, and English), but not in unfamiliar (Mandarin) or non-speech conditions. These results provide evidence suggesting that bottom-up acoustic analysis and top-down influence from phonological processing collaboratively govern voice recognition.

The Effect of Instrumental Timbre on Interval Discrimination

We tested non-musicians and musicians in an auditory psychophysical experiment to assess the effects of timbre manipulation on pitch-interval discrimination. Both groups were asked to indicate the larger of two presented intervals, comprised of four sequentially presented pitches; the second or fourth stimulus within a trial was either a sinusoidal (or “pure”), flute, piano, or synthetic voice tone, while the remaining three stimuli were all pure tones. The interval-discrimination tasks were administered parametrically to assess performance across varying pitch distances between intervals (“interval-differences”). Irrespective of timbre, musicians displayed a steady improvement across interval-differences, while non-musicians only demonstrated enhanced interval discrimination at an interval-difference of 100 cents (one semitone in Western music). Surprisingly, the best discrimination performance across both groups was observed with pure-tone intervals, followed by intervals containing a piano tone. More specifically, we observed that: 1) timbre changes within a trial affect interval discrimination; and 2) the broad spectral characteristics of an instrumental timbre may influence perceived pitch or interval magnitude and make interval discrimination more difficult.

The cost of brain structure

individual neurons. In Parkinson’s disease, the slow dancer inhibits the faster one from implementing his fancy footwork, therefore limiting their combined ability to generate movement itself (I’m sure some of us can relate). What this means physiologically is that enhanced coupling with prominent beta oscillations prevent higher frequency gamma oscillations in the motor cortex from encoding and ultimately directing appropriate movement. By meticulously parsing out these distinct signals, the authors found that electrical stimulation deep in the STN uncouples the slow beta from fast gamma oscillations in the motor cortex, thereby reducing PAC both at rest and during volitional movement. More remarkably, the degree to which DBS elicited a reduction in oscillatory coupling was closely associated with improvement in certain motor symptoms, such as muscle rigidity. The authors also demonstrated that the temporal relation between PAC reduction and symptom improvement were closely linked. To further confirm that such reduction was not simply a result of wishful thinking by hypervigilant testers, the authors blinded the testers to whether and how much stimulation was being given. Finally, they considered other potential confounding factors, such as a simple reduction in the strength of beta oscillations. However, they observed that the association between slow and fast oscillations held across patients and was largely preserved irrespective of the effect of stimulation on oscillatory strength. Thus, like a good dance coach, DBS appears to disentangle the abnormal relationship between the slow and fast oscillations in the motor cortex. This reduction in coupling takes the brakes off the motor cortex, at least over the short timespans tested here, and allows neurons responsible for volitional motor control to generate movement. Moreover, this effect appears to be specific to these oscillatory couplings, which may be based, in part, on the particular connectivity of the STN within the motor system. This study raises a number of intriguing questions. Although the authors demonstrate a striking correlation between DBS-related reduction in PAC and improvement in motor symptoms, this relation was not universal. For example, the ability of subjects to initiate movement did not appear to be highly correlated with the strength of PAC in the motor cortex, even though the ability to initiate movement is often prominently affected by DBS. One possible explanation for this is the type of behavioral test employed by the authors or area recorded. With some tweaking of the behavioral parameters, future investigation may provide additional understanding of how DBS exerts its effects. Further investigation may also aim to provide insight into how DBS improves parkinsonian symptoms in the long run, over months to years. A second, related question is how to use this new insight to improve DBS treatment. For example, one may imagine recording from cortical sites in real time and using this information to intelligently guide the timing and frequency of DBS in deeper subcortical structures such as the STN. This idea of ‘smart’ closed-loop stimulation is rapidly evolving, and some pioneering teams, such as that of Hagai Bergman, are testing similar concepts in animals7. Rather than applying DBS indiscriminately, it may be possible to dynamically scan for enhanced PAC in the motor cortex. If and when such an event is observed or when the individual is planning to move, stimulation could be delivered in short, temporally focused bursts. With the fundamental insight provided by the present study, such prospective devices could potentially revolutionize the treatment of Parkinson’s disease and related disorders.