Bonnie K. Lau

Perception of missing fundamental pitch by 3- and 4-month-old human infants

A hallmark of complex pitch perception is that the pitch of a harmonic complex is the same whether or not the fundamental frequency is present. By 7 months, infants appear to discriminate on the basis of the pitch of the missing fundamental (MF). Although electrophysiological cortical responses to MF pitch changes have been recorded in infants younger than 7 months, no psychophysical studies have been published. This study investigated the ability of 3- and 4-month-olds to perceive the pitch of MF harmonic complexes based on fundamentals of 160 Hz and 200 Hz using an observer-based method. In experiment I, to demonstrate MF pitch discrimination, 3- and 4-month-olds were required to ignore spectral changes in complexes with the same fundamental and to respond only when the fundamental changed. In experiment II, a 60-260 Hz noise was presented with complexes to mask combination tones at the fundamental frequency. In experiment III, complexes were bandpass filtered with a -12 dB/octave slope to limit use of spectral edge cues and presented with a pink noise to mask all distortion products. Nearly all infants tested categorized complexes by MF pitch in these experiments, suggesting perception of the missing fundamental at 3 months.

Temporal alignment of pupillary response with stimulus events via deconvolution

Analysis of pupil dilation has been used as an index of attentional effort in the auditory domain. Previous work has modeled the pupillary response to attentional effort as a linear time-invariant system with a characteristic impulse response, and used deconvolution to estimate the attentional effort that gives rise to changes in pupil size. Here it is argued that one parameter of the impulse response (the latency of response maximum, t(max)) has been mis-estimated in the literature; a different estimate is presented, and it is shown how deconvolution with this value of t(max) yields more intuitively plausible and informative results.

Pupillometry shows the effort of auditory attention switchinga)

Successful speech communication often requires selective attention to a target stream amidst competing sounds, as well as the ability to switch attention among multiple interlocutors. However, auditory attention switching negatively affects both target detection accuracy and reaction time, suggesting that attention switches carry a cognitive cost. Pupillometry is one method of assessing mental effort or cognitive load. Two experiments were conducted to determine whether the effort associated with attention switches is detectable in the pupillary response. In both experiments, pupil dilation, target detection sensitivity, and reaction time were measured; the task required listeners to either maintain or switch attention between two concurrent speech streams. Secondary manipulations explored whether switch-related effort would increase when auditory streaming was harder. In experiment 1, spatially distinct stimuli were degraded by simulating reverberation (compromising across-time streaming cues), and target-masker talker gender match was also varied. In experiment 2, diotic streams separable by talker voice quality and pitch were degraded by noise vocoding, and the time alloted for mid-trial attention switching was varied. All trial manipulations had some effect on target detection sensitivity and/or reaction time; however, only the attention-switching manipulation affected the pupillary response: greater dilation was observed in trials requiring switching attention between talkers.

Infant pitch perception: Missing fundamental melody discrimination

Although recent results show that 3-month-olds can discriminate complex tones by their missing fundamental, it is arguable whether they are discriminating on the basis of a perceived pitch. A defining characteristic of pitch is that it carries melodic information. This study investigated whether 3-month-olds, 7-month-olds, and adults can detect a change in a melody composed of missing fundamental complexes. Participants heard a seven-note melody and learned to respond to a change that violated the melodic contour. To ensure that participants were responding on the basis of pitch, the notes in the melody had missing fundamentals and varied in spectral content on each presentation. In experiment I, all melodies had the same absolute pitch, while in experiment II, the melodies were randomly transposed into one of three different keys on each presentation. Almost all participants learned to ignore the spectral changes and respond to the changed note of the melody in both experiments, strengthening the argument that complex tones elicit a sense of musical pitch in infants. These results provide evidence that complex pitch perception is functional by 3 months of age.

Superoptimal Perceptual Integration Suggests a Place-Based Representation of Pitch at High Frequencies

Pitch, the perceptual correlate of sound repetition rate or frequency, plays an important role in speech perception, music perception, and listening in complex acoustic environments. Despite the perceptual importance of pitch, the neural mechanisms that underlie it remain poorly understood. Although cortical regions responsive to pitch have been identified, little is known about how pitch information is extracted from the inner ear itself. The two primary theories of peripheral pitch coding involve stimulus-driven spike timing, or phase locking, in the auditory nerve (time code), and the spatial distribution of responses along the length of the cochlear partition (place code). To rule out the use of timing information, we tested pitch discrimination of very high-frequency tones (>8 kHz), well beyond the putative limit of phase locking. We found that high-frequency pure-tone discrimination was poor, but when the tones were combined into a harmonic complex, a dramatic improvement in discrimination ability was observed that exceeded performance predicted by the optimal integration of peripheral information from each of the component frequencies. The results are consistent with the existence of pitch-sensitive neurons that rely only on place-based information from multiple harmonically related components. The results also provide evidence against the common assumption that poor high-frequency pure-tone pitch perception is the result of peripheral neural-coding constraints. The finding that place-based spectral coding is sufficient to elicit complex pitch at high frequencies has important implications for the design of future neural prostheses to restore hearing to deaf individuals. SIGNIFICANCE STATEMENT The question of how pitch is represented in the ear has been debated for over a century. Two competing theories involve timing information from neural spikes in the auditory nerve (time code) and the spatial distribution of neural activity along the length of the cochlear partition (place code). By using very high-frequency tones unlikely to be coded via time information, we discovered that information from the individual harmonics is combined so efficiently that performance exceeds theoretical predictions based on the optimal integration of information from each harmonic. The findings have important implications for the design of auditory prostheses because they suggest that enhanced spatial resolution alone may be sufficient to restore pitch via such implants.

Sustained Cortical and Subcortical Measures of Auditory and Visual Plasticity following Short-Term Perceptual Learning

Short-term training can lead to improvements in behavioral discrimination of auditory and visual stimuli, as well as enhanced EEG responses to those stimuli. In the auditory domain, fluency with tonal languages and musical training has been associated with long-term cortical and subcortical plasticity, but less is known about the effects of shorter-term training. This study combined electroencephalography (EEG) and behavioral measures to investigate short-term learning and neural plasticity in both auditory and visual domains. Forty adult participants were divided into four groups. Three groups trained on one of three tasks, involving discrimination of auditory fundamental frequency (F0), auditory amplitude modulation rate (AM), or visual orientation (VIS). The fourth (control) group received no training. Pre- and post-training tests, as well as retention tests 30 days after training, involved behavioral discrimination thresholds, steady-state visually evoked potentials (SSVEP) to the flicker frequencies of visual stimuli, and auditory envelope-following responses simultaneously evoked and measured in response to rapid stimulus F0 (EFR), thought to reflect subcortical generators, and slow amplitude modulation (ASSR), thought to reflect cortical generators. Enhancement of the ASSR was observed in both auditory-trained groups, not specific to the AM-trained group, whereas enhancement of the SSVEP was found only in the visually-trained group. No evidence was found for changes in the EFR. The results suggest that some aspects of neural plasticity can develop rapidly and may generalize across tasks but not across modalities. Behaviorally, the pattern of learning was complex, with significant cross-task and cross-modal learning effects.

Behavioral and physiological measures of auditory processing in individuals with autism spectrum disorder

Sensory processing abnormalities are a hallmark of autism spectrum disorder (ASD). In the auditory domain, hyper- and hypo-sensitivity to sound, reduced orientation to speech, and difficulty listening in noise are commonly reported. However, the etiology of these auditory processing abnormalities is poorly understood. In this preliminary study, multi-talker speech perception thresholds, otoacoustic emissions, electrophysiological responses, as well as standardized measures of cognition, language, and adaptive function were measured in each individual subject with ASD and their age-matched controls. Speech perception was assessed by estimating target-to-masker ratios at 50% correct for speech targets (0° azimuth) presented with two spatially separated (±45° azimuth) simultaneous speech maskers. The electrophysiological battery used to characterize the transmission and representation of sound included: (1) a click-evoked supra-threshold auditory brainstem response, (2) an envelope following response recorded to a 400-ms-long 4 kHz pure tone carrier amplitude modulated at 100 Hz at two modulations depths (0 and -6 dB) and (3) a binaural potential evoked by an interaural phase difference embedded in an amplitude-modulated carrier tone. Together, these measures provide a behavioral and physiological assay of auditory function and show the potential to further our understanding of the mechanisms underlying auditory abnormalities in individuals with ASD.Sensory processing abnormalities are a hallmark of autism spectrum disorder (ASD). In the auditory domain, hyper- and hypo-sensitivity to sound, reduced orientation to speech, and difficulty listening in noise are commonly reported. However, the etiology of these auditory processing abnormalities is poorly understood. In this preliminary study, multi-talker speech perception thresholds, otoacoustic emissions, electrophysiological responses, as well as standardized measures of cognition, language, and adaptive function were measured in each individual subject with ASD and their age-matched controls. Speech perception was assessed by estimating target-to-masker ratios at 50% correct for speech targets (0° azimuth) presented with two spatially separated (±45° azimuth) simultaneous speech maskers. The electrophysiological battery used to characterize the transmission and representation of sound included: (1) a click-evoked supra-threshold auditory brainstem response, (2) an envelope following response recorde...

Pitch perception: Spectral and temporal integration at very high frequencies

Recent work has shown that complex pitch can be extracted from spectrally resolved harmonics, even when all the harmonics are higher than 6 kHz, and so unlikely to be represented by a temporal phase-locked neural code. This study measured spectral and temporal integration of frequency and fundamental-frequency (F0) discrimination at such high frequencies and compared the patterns of results to those obtained at lower frequencies. Difference between the present and earlier studies included the use of level roving on individual components to reduce loudness cues, and the careful control of audibility at very high frequencies. The low and high spectral regions consisted of harmonics 6–10 of two nominal F0s (280 and 1400 Hz). F0 difference limens (DLs) of the complexes and DLs of each harmonic individually were measured at two durations (30 and 210 ms) to measure spectral and temporal integration effects. All tones were presented in background noise to mask distortion products. Preliminary data show similar t...

The influence of short term perceptual learning of pitch discrimination and modulation discrimination on subcortical envelope-following and cortical steady-state EEG responses

Many perceptual skills improve with training, and research suggests that long- and short-term experiences modify auditory neural structures and function. Long-term cortical and subcortical plasticity has been associated with musical training and fluency in tonal languages, and short-term training effects have been regularly observed in cortical responses. Less is known about short-term subcortical plasticity, or the simultaneous relationships between subcortical and cortical responses under training conditions. The current study examines short-term learning and neural plasticity, using behavioral measures in combination with simultaneous subcortical and cortical steady-state EEG responses. Untrained, naive subjects were randomly assigned to groups that were trained on fundamental-frequency (F0) discrimination, amplitude-modulation rate discrimination, or visual orientation discrimination. All auditory stimuli consisted of unresolved harmonic complexes with a nominal F0 of 137 Hz that were sinusoidally amp...

Infants' ability to perceive the pitch of unresolved harmonics

An important phenomenon for models of pitch perception is that adult listeners’ can extract pitch from complexes containing only unresolved harmonics. Although 3-month-olds discriminate resolved harmonics on the basis of missing fundamental (MF) pitch, their ability to discriminate unresolved harmonics is unknown. This study investigated the ability of adults, 7- and 3-month-olds, to perceive the pitch of unresolved harmonics using an observer-based method. Stimuli were MF complexes that were bandpass filtered with a -12 dB/octave slope, combined in random phase, and presented at 70 dB SPL for 650 ms with a 50 ms rise/fall and with a pink noise to mask distortion products. The experiment consisted of two conditions: (1) “low” unresolved harmonics between 2500 and 4500 Hz based on MFs of 160 Hz (H17–H26) and 200 Hz (H13–H22) and (2) “high” unresolved harmonics between 4000 and 6000 Hz based on MFs of 190 Hz (H22–H31) and 200 Hz (H20–H29). To demonstrate MF pitch discrimination, participants were required to ignore spectral changes in complexes with the same fundamental and to respond only when the fundamental changed. Interestingly, variable performance in the “high” condition was observed with adult participants. However, nearly all infants tested categorized complexes by MF pitch in both conditions, suggesting discrimination of unresolved harmonics at 3 months.