Tim Green

Enhancing temporal cues to voice pitch in continuous interleaved sampling cochlear implants

The limited spectral resolution of cochlear implant systems means that voice pitch perception depends on weak temporal envelope cues. Enhancement of such cues was investigated in implant users and in acoustic simulations. Subjects labeled the pitch movement of processed synthetic diphthongal glides. In standard processing, noise carriers (simulations) or pulse trains (implant users) were modulated by 400 Hz low-pass envelopes. In modified processing, carriers were modulated by two components: (1) Slow-rate (<32 Hz) envelope modulations, conveying dynamic spectral shape changes crucial for speech; (2) a simplified waveform (e.g., a sawtooth) matching the periodicity of the input diphthong. In both normal listeners and implant users performance was better with modified processing, though temporal envelope cues were less effective with higher F0. Factors contributing to the advantage for modified processing may include increased modulation depth and use of a modulation waveform featuring a rapid onset in each period, resulting in a clearer representation of F0 in the neural firing pattern. Eliminating slow-rate spectral dynamics, so that within-channel amplitude changes solely reflected F0, showed that dynamic spectral variation obscured temporal pitch cues. Though significant, advantages for modified processing were small, suggesting that the potential for developing strategies delivering enhanced pitch perception is limited.

Spectral and temporal cues to pitch in noise-excited vocoder simulations of continuous-interleaved-sampling cochlear implants

Four-band and single-band noise-excited vocoders were used in acoustic simulations to investigate spectral and temporal cues to melodic pitch in the output of a cochlear implant speech processor. Noise carriers were modulated by amplitude envelopes extracted by half-wave rectification and low-pass filtering at 32 or 400 Hz. The four-band, but not the single-band processors, may preserve spectral correlates of fundamental frequency (F0). Envelope smoothing at 400 Hz preserves temporal correlates of F0, which are eliminated with 32-Hz smoothing. Inputs to the processors were sawtooth frequency glides, in which spectral variation is completely determined by F0, or synthetic diphthongal vowel glides, whose spectral shape is dominated by varying formant resonances. Normal listeners labeled the direction of pitch movement of the processed stimuli. For processed sawtooth waves, purely temporal cues led to decreasing performance with increasing F0. With purely spectral cues, performance was above chance despite the limited spectral resolution of the processors. For processed diphthongs, performance with purely spectral cues was at chance, showing that spectral envelope changes due to formant movement obscured spectral cues to F0. Performance with temporal cues was poorer for diphthongs than for sawtooths, with very limited discrimination at higher F0. These data suggest that, for speech signals through a typical cochlear implant processor, spectral cues to pitch are likely to have limited utility, while temporal envelope cues may be useful only at low F0.

Enhancement of temporal periodicity cues in cochlear implants: Effects on prosodic perception and vowel identification

Standard continuous interleaved sampling processing, and a modified processing strategy designed to enhance temporal cues to voice pitch, were compared on tests of intonation perception, and vowel perception, both in implant users and in acoustic simulations. In standard processing, 400 Hz low-pass envelopes modulated either pulse trains (implant users) or noise carriers (simulations). In the modified strategy, slow-rate envelope modulations, which convey dynamic spectral variation crucial for speech understanding, were extracted by low-pass filtering (32 Hz). In addition, during voiced speech, higher-rate temporal modulation in each channel was provided by 100% amplitude-modulation by a sawtooth-like wave form whose periodicity followed the fundamental frequency (F0) of the input. Channel levels were determined by the product of the lower- and higher-rate modulation components. Both in acoustic simulations and in implant users, the ability to use intonation information to identify sentences as question or statement was significantly better with modified processing. However, while there was no difference in vowel recognition in the acoustic simulation, implant users performed worse with modified processing both in vowel recognition and in formant frequency discrimination. It appears that, while enhancing pitch perception, modified processing harmed the transmission of spectral information.

Talker intelligibility differences in cochlear implant listeners

People vary in the intelligibility of their speech. This study investigated whether across-talker intelligibility differences observed in normally-hearing listeners are also found in cochlear implant (CI) users. Speech perception for male, female, and child pairs of talkers differing in intelligibility was assessed with actual and simulated CI processing and in normal hearing. While overall speech recognition was, as expected, poorer for CI users, differences in intelligibility across talkers were consistent across all listener groups. This suggests that the primary determinants of intelligibility differences are preserved in the CI-processed signal, though no single critical acoustic property could be identified.

Comparing live to recorded speech in training the perception of spectrally shifted noise-vocoded speech.

Two experimental groups were trained for 2 h with live or recorded speech that was noise-vocoded and spectrally shifted and was from the same text and talker. These two groups showed equivalent improvements in performance for vocoded and shifted sentences, and the group trained with recorded speech showed consistently greater improvements than untrained controls. Another group trained with unshifted noise-vocoded speech improved no more than untrained controls. Computer-based training thus appears at least as effective as labor-intensive live-voice training for improving the perception of spectrally shifted noise-vocoded speech, and by implication, for training of users of cochlear implants.

Variations in carrier pulse rate and the perception of amplitude modulation in cochlear implant users.

Objectives: A major focus of recent attempts to enhance cochlear implant (CI) systems has been to increase the rate at which pulses are delivered to the electrode array. One basis for these attempts has been the expectation that faster stimulation rates would lead to an enhanced representation of temporal modulation information. However, there is recent physiological and behavioral evidence to suggest that the reverse may be the case. Here, the effects of stimulation rate on the perception of amplitude modulation were assessed using both modulation detection and modulation frequency discrimination tasks for a range of pulse rates extending considerably higher than the highest rate tested in previous studies and for different speech-relevant modulation frequencies. Design: Detection of sinusoidal amplitude modulation was assessed in five CI users using monopolar pulse trains presented to a single electrode at rates of 482, 723, 1447, 2894, and 5787 pulses per second (pps). Adaptive procedures were used to find the minimal detectable modulation depth at modulation frequencies of 10 and 100 Hz and at carrier levels of 25%, 50%, and 75% of the electrode’s dynamic range. Discrimination of modulation frequency was examined for the same range of pulse rates for the highest carrier level. Similar adaptive procedures determined the minimum increase in modulation frequency that could be detected relative to reference modulation frequencies of 10, 100, and 200 Hz. In both tasks, level roving was implemented to minimize possible loudness cues. Results: Consistent with previous evidence, modulation detection thresholds were better for higher carrier levels and lower modulation frequencies. When modulation depth at threshold was expressed in terms of the ratio of the depth of the modulation and the carrier level in dB (i.e., 20 log m), performance was significantly better at lower pulse rates. However, when modulation depth was expressed relative to dynamic range, the effect of pulse rate was no longer significant, reflecting the fact that dynamic range increases with pulse rate. Modulation frequency discrimination clearly worsened with increasing modulation frequency, but there was no significant effect of pulse rate. Conclusions: In contrast to some recent evidence, no clearly harmful effect of higher pulse rates on modulation perception was found. However, even with very fast stimulation rates, tested over a wide range of modulation frequencies and with two different tasks, there is no evidence of benefit from faster stimulation rates in the perception of amplitude modulation.

Frequency selectivity of contralateral residual acoustic hearing in bimodal cochlear implant users, and limitations on the ability to match the pitch of electric and acoustic stimuli

Abstract Objective: To assess the reliability of across-ear, acoustic-electric pitch/timbre comparisons for determining effective characteristic frequencies of cochlear implant electrodes. Study sample: Nine CI users with contralateral residual acoustic hearing. Design: Absolute acoustic thresholds in the unimplanted ear were measured and frequency selectivity was assessed via psychophysical tuning curves. An adjustment method was used to match the percepts elicited by pulse trains on individual electrodes with various acoustic signals (pure tones, narrow-band noises, and bandpass filtered pulse trains). The starting frequency of the acoustic signal was roved and matches were obtained at different loudness levels. Results: Acoustic frequency selectivity varied widely. Two subjects showed clear evidence of frequency selectivity extending above 500 Hz. Only these subjects produced consistent pitch matches over repeated measurements. For other subjects, the acoustic frequency eventually selected tended to correlate with the initially presented frequency. There was limited evidence of level effects and these were inconsistent across subjects and electrodes. Conclusions: Across-modality pitch/timbre matching appears unlikely to provide a generally applicable method for determining the effective characteristic frequencies of cochlear implant electrodes. Frequency selectivity above 500 Hz may be necessary for consistent pitch/timbre matches.

Advantages from bilateral hearing in speech perception in noise with simulated cochlear implants and residual acoustic hearing

Acoustic simulations were used to study the contributions of spatial hearing that may arise from combining a cochlear implant with either a second implant or contralateral residual low-frequency acoustic hearing. Speech reception thresholds (SRTs) were measured in twenty-talker babble. Spatial separation of speech and noise was simulated using a spherical head model. While low-frequency acoustic information contralateral to the implant simulation produced substantially better SRTs there was no effect of spatial cues on SRT, even when interaural differences were artificially enhanced. Simulated bilateral implants showed a significant head shadow effect, but no binaural unmasking based on interaural time differences, and weak, inconsistent overall spatial release from masking. There was also a small but significant non-spatial summation effect. It appears that typical cochlear implant speech processing strategies may substantially reduce the utility of spatial cues, even in the absence of degraded neural processing arising from auditory deprivation.

The role of auditory memory traces in attention to frequency

Three cued signal detection experiments demonstrated a role for auditory memory traces in frequency selectivity. The extent to which the cue predicted the signal frequency affected the size of the advantage for signals at the cue frequency over those at distant frequencies when the cue-signal gap was 10 sec but not when it was 1 sec. Detection of occasional signals presented at uncued frequencies was enhanced when they matched the frequency of cues from recent trials. With “relative” cues, which were usually followed by signals at the musical fifth above the cue frequency, performance on occasional signals at the cue frequency was enhanced relative to other unexpected frequencies. These results suggest that, regardless of the listener’s expectations and intentions, the detectability of a signal is enhanced if its frequency matches an existing memory trace. One form of voluntary attention to frequency may involve maintaining traces that would otherwise slowly decay.

Use of a single channel dedicated to conveying enhanced temporal periodicity cues in cochlear implants: Effects on prosodic perception and vowel identification

The continuous interleaved sampling (CIS) strategy for cochlear implants has well-established limitations for the perception of pitch changes in speech. This study investigated a modification of CIS in which one channel was dedicated to the transmission of a temporal encoding of fundamental frequency (F0). Normal hearing subjects listening to noise-excited vocoders, and implantees were tested on labelling the pitch movement of diphthongal glides, on using intonation information to identify sentences as question or statement, and on vowel recognition. There were no significant differences between modified processing and CIS in vowel recognition. However, while there was limited evidence of improved pitch perception relative to CIS with simplified F0 modulation applied to the most basal channel, in general it appears that for most implant users, restricting F0-related modulation to one channel does not provide significantly enhanced pitch information.