Jayaganesh Swaminathan

Experience-dependent enhancement of linguistic pitch representation in the brainstem is not specific to a speech context

Neural representation of pitch is influenced by lifelong experiences with music and language at both cortical and subcortical levels of processing. The aim of this article is to determine whether neural plasticity for pitch representation at the level of the brainstem is dependent upon specific dimensions of pitch contours that commonly occur as part of a native listeners language experience. Brainstem frequency following responses (FFRs) were recorded from Chinese and English participants in response to four Mandarin tonal contours presented in a nonspeech context in the form of iterated rippled noise. Pitch strength (whole contour, 250 msec; 40-msec segments) and pitch-tracking accuracy (whole contour) were extracted from the FFRs using autocorrelation algorithms. Narrow band spectrograms were used to extract spectral information. Results showed that the Chinese group exhibits smoother pitch tracking than the English group in three out of the four tones. Moreover, cross-language comparisons of pitch strength of 40-msec segments revealed that the Chinese group exhibits more robust pitch representation of those segments containing rapidly changing pitch movements across all four tones. FFR spectral data were complementary showing that the Chinese group exhibits stronger representation of multiple pitch-relevant harmonics relative to the English group across all four tones. These findings support the view that at early preattentive stages of subcortical processing, neural mechanisms underlying pitch representation are shaped by particular dimensions of the auditory stream rather than speech per se. Adopting a temporal correlation analysis scheme for pitch encoding, we propose that long-term experience sharpens the tuning characteristics of neurons along the pitch axis with enhanced sensitivity to linguistically relevant variations in pitch.

Experience-dependent neural representation of dynamic pitch in the brainstem

Brainstem frequency-following responses were recorded from Chinese and English participants in response to an iterated rippled noise homologue of Mandarin Tone 2 (T2) and linear and inverted curvilinear variants. Pitch-tracking accuracy and pitch strength analyses showed advantages for the Chinese group over the English in response to T2 only. Pitch strength was larger for the Chinese group in rapidly changing sections of T2 compared with corresponding sections of a linear ramp. We conclude that experience-dependent neural plasticity at subcortical levels of representation is highly sensitive to specific features of pitch patterns in ones native language. Such experience-dependent effects suggest that subcortical sensory encoding interacts with cognitive processing in the cerebral cortex to shape the perceptual systems response to pitch patterns.

Quantifying Envelope and Fine-Structure Coding in Auditory Nerve Responses to Chimaeric Speech

Any sound can be separated mathematically into a slowly varying envelope and rapidly varying fine-structure component. This property has motivated numerous perceptual studies to understand the relative importance of each component for speech and music perception. Specialized acoustic stimuli, such as auditory chimaeras with the envelope of one sound and fine structure of another have been used to separate the perceptual roles for envelope and fine structure. Cochlear narrowband filtering limits the ability to isolate fine structure from envelope; however, envelope recovery from fine structure has been difficult to evaluate physiologically. To evaluate envelope recovery at the output of the cochlea, neural cross-correlation coefficients were developed that quantify the similarity between two sets of spike-train responses. Shuffled auto- and cross-correlogram analyses were used to compute separate correlations for responses to envelope and fine structure based on both model and recorded spike trains from auditory nerve fibers. Previous correlogram analyses were extended to isolate envelope coding more effectively in auditory nerve fibers with low center frequencies, which are particularly important for speech coding. Recovered speech envelopes were present in both model and recorded responses to one- and 16-band speech fine-structure chimaeras and were significantly greater for the one-band case, consistent with perceptual studies. Model predictions suggest that cochlear recovered envelopes are reduced following sensorineural hearing loss due to broadened tuning associated with outer-hair cell dysfunction. In addition to the within-fiber cross-stimulus cases considered here, these neural cross-correlation coefficients can also be used to evaluate spatiotemporal coding by applying them to cross-fiber within-stimulus conditions. Thus, these neural metrics can be used to quantitatively evaluate a wide range of perceptually significant temporal coding issues relevant to normal and impaired hearing.

Pitch encoding in speech and nonspeech contexts in the human auditory brainstem

Frequency-following responses were recorded from Chinese and English participants at the level of the brainstem in response to four Mandarin tonal contours presented in a speech and nonspeech context. Pitch strength analysis of these preattentive brainstem responses showed that the Chinese group exhibited stronger pitch representation than the English group regardless of context. Moreover, the Chinese group exhibited relatively more robust pitch representation of rapidly changing pitch segments. These findings support the view that at early preattentive stages of subcortical processing, neural mechanisms underlying pitch representation are shaped by particular features of the auditory stream rather than speech per se. These findings have implications for optimizing signal-processing strategies for cochlear implant design for speakers of tonal languages.

Psychophysiological Analyses Demonstrate the Importance of Neural Envelope Coding for Speech Perception in Noise

Understanding speech in noisy environments is often taken for granted; however, this task is particularly challenging for people with cochlear hearing loss, even with hearing aids or cochlear implants. A significant limitation to improving auditory prostheses is our lack of understanding of the neural basis for robust speech perception in noise. Perceptual studies suggest the slowly varying component of the acoustic waveform (envelope, ENV) is sufficient for understanding speech in quiet, but the rapidly varying temporal fine structure (TFS) is important in noise. These perceptual findings have important implications for cochlear implants, which currently only provide ENV; however, neural correlates have been difficult to evaluate due to cochlear transformations between acoustic TFS and recovered neural ENV. Here, we demonstrate the relative contributions of neural ENV and TFS by quantitatively linking neural coding, predicted from a computational auditory nerve model, with perception of vocoded speech in noise measured from normal hearing human listeners. Regression models with ENV and TFS coding as independent variables predicted speech identification and phonetic feature reception at both positive and negative signal-to-noise ratios. We found that: (1) neural ENV coding was a primary contributor to speech perception, even in noise; and (2) neural TFS contributed in noise mainly in the presence of neural ENV, but rarely as the primary cue itself. These results suggest that neural TFS has less perceptual salience than previously thought due to cochlear signal processing transformations between TFS and ENV. Because these transformations differ between normal and impaired ears, these findings have important translational implications for auditory prostheses.

Applications of Static and Dynamic Iterated Rippled Noise to Evaluate Pitch Encoding in the Human Auditory Brainstem

This paper presents a new application of the dynamic iterated rippled noise (IRN) algorithm by generating dynamic pitch contours representative of those that occur in natural speech in the context of EEG and the frequency following response (FFR). Besides IRN steady state and linear rising stimuli, curvilinear rising stimuli were modeled after pitch contours of natural productions of Mandarin Tone 2. Electrophysiological data on pitch representation at the level of the brainstem, as reflected in FFR, were evaluated for all stimuli, static or dynamic. Autocorrelation peaks were observed corresponding to the fundamental period (tau) as well as spectral bands at the fundamental and its harmonics for both a low and a high iteration step. At the higher iteration step, both spectral and temporal FFR representations were more robust, indicating that both acoustic properties may be utilized for pitch extraction at the level of the brainstem. By applying curvilinear IRN stimuli to elicit FFRs, we can evaluate the effects of temporal degradation on 1) the neural representation of linguistically-relevant pitch features in a target population (e.g. cochlear implant) and 2) the efficacy of signal processing schemes in conventional hearing aids and cochlear implants to recover these features.

Musical training, individual differences and the cocktail party problem

Are musicians better able to understand speech in noise than non-musicians? Recent findings have produced contradictory results. Here we addressed this question by asking musicians and non-musicians to understand target sentences masked by other sentences presented from different spatial locations, the classical ‘cocktail party problem’ in speech science. We found that musicians obtained a substantial benefit in this situation, with thresholds ~6 dB better than non-musicians. Large individual differences in performance were noted particularly for the non-musically trained group. Furthermore, in different conditions we manipulated the spatial location and intelligibility of the masking sentences, thus changing the amount of ‘informational masking’ (IM) while keeping the amount of ‘energetic masking’ (EM) relatively constant. When the maskers were unintelligible and spatially separated from the target (low in IM), musicians and non-musicians performed comparably. These results suggest that the characteristics of speech maskers and the amount of IM can influence the magnitude of the differences found between musicians and non-musicians in multiple-talker “cocktail party” environments. Furthermore, considering the task in terms of the EM-IM distinction provides a conceptual framework for future behavioral and neuroscientific studies which explore the underlying sensory and cognitive mechanisms contributing to enhanced “speech-in-noise” perception by musicians.

Across-Fiber Coding of Temporal Fine-Structure: Effects of Noise-Induced Hearing Loss on Auditory-Nerve Responses

Recent psychophysical evidence suggests that listeners with sensorineural hearing loss (SNHL) have a reduced ability to use temporal fine-structure cues. These results have renewed an interest in the effects of SNHL on the neural coding of fine structure. The lack of convincing evidence that SNHL affects within-fiber phase locking has led to the hypothesis that degraded across-fiber temporal coding may underlie this perceptual effect. Spike trains were recorded from auditory-nerve (AN) fibers in chinchillas with normal hearing and with noise-induced hearing loss. A spectro-temporal manipulation procedure was used to predict spatiotemporal patterns for characteristic frequencies (CFs) spanning up to an octave range from the responses of individual AN fibers to a stimulus presented with sampling rates spanning an octave range. Shuffled cross-correlogram analyses were used to quantify across-CF fine-structure coding in terms of both a neural cross-correlation coefficient and a characteristic delay. Neural cross-correlation for fine-structure decreased and the estimated traveling-wave delay increased with increases in CF separation for both normal and impaired fibers. However, the range of CF separations over which significant correlated activity existed was wider, and the estimated traveling-wave delay was less for impaired AN fibers. Both of these effects of SNHL on across-CF coding have important implications for spatiotemporal theories of speech coding.

Executive Function, Visual Attention and the Cocktail Party Problem in Musicians and Non-Musicians

The goal of this study was to investigate how cognitive factors influence performance in a multi-talker, “cocktail-party” like environment in musicians and non-musicians. This was achieved by relating performance in a spatial hearing task to cognitive processing abilities assessed using measures of executive function (EF) and visual attention in musicians and non-musicians. For the spatial hearing task, a speech target was presented simultaneously with two intelligible speech maskers that were either colocated with the target (0° azimuth) or were symmetrically separated from the target in azimuth (at ±15°). EF assessment included measures of cognitive flexibility, inhibition control and auditory working memory. Selective attention was assessed in the visual domain using a multiple object tracking task (MOT). For the MOT task, the observers were required to track target dots (n = 1,2,3,4,5) in the presence of interfering distractor dots. Musicians performed significantly better than non-musicians in the spatial hearing task. For the EF measures, musicians showed better performance on measures of auditory working memory compared to non-musicians. Furthermore, across all individuals, a significant correlation was observed between performance on the spatial hearing task and measures of auditory working memory. This result suggests that individual differences in performance in a cocktail party-like environment may depend in part on cognitive factors such as auditory working memory. Performance in the MOT task did not differ between groups. However, across all individuals, a significant correlation was found between performance in the MOT and spatial hearing tasks. A stepwise multiple regression analysis revealed that musicianship and performance on the MOT task significantly predicted performance on the spatial hearing task. Overall, these findings confirm the relationship between musicianship and cognitive factors including domain-general selective attention and working memory in solving the “cocktail party problem”.

Determining the energetic and informational components of speech-on-speech masking

Identification of target speech was studied under masked conditions consisting of two or four independent speech maskers. In the reference conditions, the maskers were colocated with the target, the masker talkers were the same sex as the target, and the masker speech was intelligible. The comparison conditions, intended to provide release from masking, included different-sex target and masker talkers, time-reversal of the masker speech, and spatial separation of the maskers from the target. Significant release from masking was found for all comparison conditions. To determine whether these reductions in masking could be attributed to differences in energetic masking, ideal time-frequency segregation (ITFS) processing was applied so that the time-frequency units where the masker energy dominated the target energy were removed. The remaining target-dominated “glimpses” were reassembled as the stimulus. Speech reception thresholds measured using these resynthesized ITFS-processed stimuli were the same for the reference and comparison conditions supporting the conclusion that the amount of energetic masking across conditions was the same. These results indicated that the large release from masking found under all comparison conditions was due primarily to a reduction in informational masking. Furthermore, the large individual differences observed generally were correlated across the three masking release conditions.