Jessica J. M. Monaghan

Development of social vocalizations in mice.

Adult mice are highly vocal animals, with both males and females vocalizing in same sex and cross sex social encounters. Mouse pups are also highly vocal, producing isolation vocalizations when they are cold or removed from the nest. This study examined patterns in the development of pup isolation vocalizations, and compared these to adult vocalizations. In three litters of CBA/CaJ mice, we recorded isolation vocalizations at ages postnatal day 5 (p5), p7, p9, p11, and p13. Adult vocalizations were obtained in a variety of social situations. Altogether, 28,384 discrete vocal signals were recorded using high-frequency-sensitive equipment and analyzed for syllable type, spectral and temporal features, and the temporal sequencing within bouts. We found that pups produced all but one of the 11 syllable types recorded from adults. The proportions of syllable types changed developmentally, but even the youngest pups produced complex syllables with frequency-time variations. When all syllable types were pooled together for analysis, changes in the peak frequency or the duration of syllables were small, although significant, from p5 through p13. However, individual syllable types showed different, large patterns of change over development, requiring analysis of each syllable type separately. Most adult syllables were substantially lower in frequency and shorter in duration. As pups aged, the complexity of vocal bouts increased, with a greater tendency to switch between syllable types. Vocal bouts from older animals, p13 and adult, had significantly more sequential structure than those from younger mice. Overall, these results demonstrate substantial changes in social vocalizations with age. Future studies are required to identify whether these changes result from developmental processes affecting the vocal tract or control of vocalization, or from vocal learning. To provide a tool for further research, we developed a MATLAB program that generates bouts of vocalizations that correspond to mice of different ages.

A statistical, formant-pattern model for segregating vowel type and vocal-tract length in developmental formant data

This paper investigates the theoretical basis for estimating vocal-tract length (VTL) from the formant frequencies of vowel sounds. A statistical inference model was developed to characterize the relationship between vowel type and VTL, on the one hand, and formant frequency and vocal cavity size, on the other. The model was applied to two well known developmental studies of formant frequency. The results show that VTL is the major source of variability after vowel type and that the contribution due to other factors like developmental changes in oral-pharyngeal ratio is small relative to the residual measurement noise. The results suggest that speakers adjust the shape of the vocal tract as they grow to maintain a specific pattern of formant frequencies for individual vowels. This formant-pattern hypothesis motivates development of a statistical-inference model for estimating VTL from formant-frequency data. The technique is illustrated using a third developmental study of formant frequencies. The VTLs of the speakers are estimated and used to provide a more accurate description of the complicated relationship between VTL and glottal pulse rate as children mature into adults.

Factors affecting the use of envelope interaural time differences in reverberation

At high frequencies, interaural time differences (ITDs) are conveyed by the sound envelope. Sensitivity to envelope ITDs depends crucially on the envelope shape. Reverberation degrades the envelope shape, reducing the modulation depth of the envelope and the slope of its flanks. Reverberation also reduces the envelope interaural coherence (i.e., the similarity of the envelopes at two ears). The current study investigates the extent to which these changes affect sensitivity to envelope ITDs. The first experiment measured ITD discrimination thresholds at low and high frequencies in a simulated room. The stimulus was either a low-frequency narrowband noise or the same noise transposed to a higher frequency. The results suggest that the effect of reverberation on ITD thresholds was multiplicative. Given that the threshold without reverberation was larger for the transposed than for the low-frequency stimulus, this meant that, in absolute terms, the thresholds for the transposed stimulus showed a much greater increase due to reverberation than those for the low-frequency stimulus. Three further experiments indicated that the effect of reverberation on the envelope ITD thresholds was due to the combined effect of the reduction in the envelope modulation depth and slopes, as well as the decrease in the envelope interaural coherence.

Speech enhancement based on neural networks improves speech intelligibility in noise for cochlear implant users

&NA; Speech understanding in noisy environments is still one of the major challenges for cochlear implant (CI) users in everyday life. We evaluated a speech enhancement algorithm based on neural networks (NNSE) for improving speech intelligibility in noise for CI users. The algorithm decomposes the noisy speech signal into time‐frequency units, extracts a set of auditory‐inspired features and feeds them to the neural network to produce an estimation of which frequency channels contain more perceptually important information (higher signal‐to‐noise ratio, SNR). This estimate is used to attenuate noise‐dominated and retain speech‐dominated CI channels for electrical stimulation, as in traditional n‐of‐m CI coding strategies. The proposed algorithm was evaluated by measuring the speech‐in‐noise performance of 14 CI users using three types of background noise. Two NNSE algorithms were compared: a speaker‐dependent algorithm, that was trained on the target speaker used for testing, and a speaker‐independent algorithm, that was trained on different speakers. Significant improvements in the intelligibility of speech in stationary and fluctuating noises were found relative to the unprocessed condition for the speaker‐dependent algorithm in all noise types and for the speaker‐independent algorithm in 2 out of 3 noise types. The NNSE algorithms used noise‐specific neural networks that generalized to novel segments of the same noise type and worked over a range of SNRs. The proposed algorithm has the potential to improve the intelligibility of speech in noise for CI users while meeting the requirements of low computational complexity and processing delay for application in CI devices. HighlightsAn algorithm for improving speech understanding in noise for cochlear implant users is evaluated.Significant improvements were found for stationary and non‐stationary noise types.It generalizes to a novel speaker and works over a range of signal‐to‐noise ratios.The small algorithmic delay makes it suitable for real‐time application.

Low‐dimensional, auditory feature vectors that improve vocal‐tract‐length normalization in automatic speech recognition

Human speech recognition is robust to large changes in vocal tract length (VTL) but automatic speech recognition is not. In an effort to improve VTL normalization, an auditory model was used to derive formant‐like features from syllables. The robustness supported by these auditory features was compared to the robustness provided by traditional MFCCs (Mel‐Frequency Cepstral Coefficients), using a standard HMM recognizer (Hidden‐Markov‐Model). The speech database consisted of 180 syllables, each scaled with the vocoder STRAIGHT to have a wide range VTLs and glottal pulse rates. Training took place with syllables from a small, central range of scale values. When tested on the full range of scaled syllables, average performance for MFCC‐based recognition was 73.5%, with performance falling close to 0% for syllables with extreme VTL values. The feature vectors constructed with the auditory model led to much better performance; the average for the full range of scaled syllables was 91%, and performance never fe...

A method to enhance the use of interaural time differences for cochlear implants in reverberant environments

The ability of normal-hearing (NH) listeners to exploit interaural time difference (ITD) cues conveyed in the modulated envelopes of high-frequency sounds is poor compared to ITD cues transmitted in the temporal fine structure at low frequencies. Sensitivity to envelope ITDs is further degraded when envelopes become less steep, when modulation depth is reduced, and when envelopes become less similar between the ears, common factors when listening in reverberant environments. The vulnerability of envelope ITDs is particularly problematic for cochlear implant (CI) users, as they rely on information conveyed by slowly varying amplitude envelopes. Here, an approach to improve access to envelope ITDs for CIs is described in which, rather than attempting to reduce reverberation, the perceptual saliency of cues relating to the source is increased by selectively sharpening peaks in the amplitude envelope judged to contain reliable ITDs. Performance of the algorithm with room reverberation was assessed through simulating listening with bilateral CIs in headphone experiments with NH listeners. Relative to simulated standard CI processing, stimuli processed with the algorithm generated lower ITD discrimination thresholds and increased extents of laterality. Depending on parameterization, intelligibility was unchanged or somewhat reduced. The algorithm has the potential to improve spatial listening with CIs.

Sensitivity to Envelope Interaural Time Differences at High Modulation Rates

Sensitivity to interaural time differences (ITDs) conveyed in the temporal fine structure of low-frequency tones and the modulated envelopes of high-frequency sounds are considered comparable, particularly for envelopes shaped to transmit similar fidelity of temporal information normally present for low-frequency sounds. Nevertheless, discrimination performance for envelope modulation rates above a few hundred Hertz is reported to be poor—to the point of discrimination thresholds being unattainable—compared with the much higher (>1,000 Hz) limit for low-frequency ITD sensitivity, suggesting the presence of a low-pass filter in the envelope domain. Further, performance for identical modulation rates appears to decline with increasing carrier frequency, supporting the view that the low-pass characteristics observed for envelope ITD processing is carrier-frequency dependent. Here, we assessed listeners’ sensitivity to ITDs conveyed in pure tones and in the modulated envelopes of high-frequency tones. ITD discrimination for the modulated high-frequency tones was measured as a function of both modulation rate and carrier frequency. Some well-trained listeners appear able to discriminate ITDs extremely well, even at modulation rates well beyond 500 Hz, for 4-kHz carriers. For one listener, thresholds were even obtained for a modulation rate of 800 Hz. The highest modulation rate for which thresholds could be obtained declined with increasing carrier frequency for all listeners. At 10 kHz, the highest modulation rate at which thresholds could be obtained was 600 Hz. The upper limit of sensitivity to ITDs conveyed in the envelope of high-frequency modulated sounds appears to be higher than previously considered.

Speech enhancement based on neural networks applied to cochlear implant coding strategies

Traditionally, algorithms that attempt to significantly improve speech intelligibility in noise for cochlear implant (CI) users have met with limited success, particularly in the presence of a fluctuating masker. In the present study, a speech enhancement algorithm integrating an artificial neural network (NN) into CI coding strategies is proposed. The algorithm decomposes the noisy input signal into time-frequency units, extracts a set of auditory-inspired features and feeds them to the NN to produce an estimation of which CI channels contain more perceptually important information (higher signal-to-noise ratio, SNR). This estimate is then used accordingly to retain a subset of channels for electrical stimulation, as in traditional n-of-m coding strategies. The proposed algorithm was tested with 10 normal-hearing participants listening to CI noise-vocoder simulations against a conventional Wiener filter based enhancement algorithm. Significant improvements in speech intelligibility in stationary and fluctuating noise were found over both unprocessed and Wiener filter processed conditions.

Auditory inspired machine learning techniques can improve speech intelligibility and quality for hearing-impaired listenersa)

Machine-learning based approaches to speech enhancement have recently shown great promise for improving speech intelligibility for hearing-impaired listeners. Here, the performance of three machine-learning algorithms and one classical algorithm, Wiener filtering, was compared. Two algorithms based on neural networks were examined, one using a previously reported feature set and one using a feature set derived from an auditory model. The third machine-learning approach was a dictionary-based sparse-coding algorithm. Speech intelligibility and quality scores were obtained for participants with mild-to-moderate hearing impairments listening to sentences in speech-shaped noise and multi-talker babble following processing with the algorithms. Intelligibility and quality scores were significantly improved by each of the three machine-learning approaches, but not by the classical approach. The largest improvements for both speech intelligibility and quality were found by implementing a neural network using the feature set based on auditory modeling. Furthermore, neural network based techniques appeared more promising than dictionary-based, sparse coding in terms of performance and ease of implementation.

Speech enhancement for hearing-impaired listeners using deep neural networks with auditory-model based features

Speech understanding in adverse acoustic environments is still a major problem for users of hearing-instruments. Recent studies on supervised speech segregation show good promise to alleviate this problem by separating speech-dominated from noise-dominated spectro-temporal regions with estimated time-frequency masks. The current study compared a previously proposed feature set to a novel auditory-model based feature set using a common deep neural network based speech enhancement framework. The performance of both feature extraction methods was evaluated with objective measurements and a subjective listening test to measure speech perception scores in terms of intelligibility and quality with 17 hearing-impaired listeners. Significant improvements in speech intelligibility and quality ratings were found for both feature extraction systems. However, the auditory-model based feature set showed superior performance compared to the comparison feature set indicating that auditory-model based processing could provide further improvements for supervised speech segregation systems and their potential applications in hearing instruments.