Daniel McCloy

The advantage of knowing the talker.

BACKGROUND Many audiologists have observed a situation where a patient appears to understand something spoken by his or her spouse or a close friend but not the same information spoken by a stranger. However, it is not clear whether this observation reflects choice of communication strategy or a true benefit derived from the talkers voice. PURPOSE The current study measured the benefits of long-term talker familiarity for older individuals with hearing impairment in a variety of listening situations. RESEARCH DESIGN In Experiment 1, we measured speech recognition with familiar and unfamiliar voices when the difficulty level was manipulated by varying levels of a speech-shaped background noise. In Experiment 2, we measured the benefit of a familiar voice when the background noise was other speech (informational masking). STUDY SAMPLE A group of 31 older listeners with high-frequency sensorineural hearing loss participated in the study. Fifteen of the participants served as talkers and 16 as listeners. In each case, the talker-listener pair for the familiar condition represented a close, long-term relationship (spouse or close friend). DATA COLLECTION AND ANALYSIS Speech-recognition scores were compared using controlled stimuli (low-context sentences) recorded by the study talkers. The sentences were presented in quiet and in two levels of speech-spectrum noise (Experiment 1) as well as in multitalker babble (Experiment 2). Repeated-measures analysis of variance was used to compare performance between the familiar and unfamiliar talkers, within and across conditions. RESULTS Listeners performed better when speech was produced by a talker familiar to them, whether that talker was in a quiet or noisy environment. The advantage of the familiar talker was greater in a more adverse listening situation (i.e., in the highest level of background noise) but was similar for speech-spectrum noise and multitalker babble. CONCLUSIONS The present data support a frequent clinical observation: listeners can understand their spouse better than a stranger. This effect was present for all our participants and occurred under strictly controlled conditions in which the only possible cue was the voice itself, rather than under normal communicative conditions where listener accommodation strategies on the part of the talker may confound the measurable benefit. The magnitude of the effect was larger than shown for short-term familiarity in previous work. This suggests that older listeners with hearing loss who inherently operate under deficient auditory conditions can benefit from experience with the voice characteristics of a long-term communication partner over many years of a relationship.

Talker Versus Dialect Effects on Speech Intelligibility: A Symmetrical Study

This study investigates the relative effects of talker-specific variation and dialect-based variation on speech intelligibility. Listeners from two dialects of American English performed speech-in-noise tasks with sentences spoken by talkers of each dialect. An initial statistical model showed no significant effects for either talker or listener dialect group, and no interaction. However, a mixed-effects regression model including several acoustic measures of the talker’s speech revealed a subtle effect of talker dialect once the various acoustic dimensions were accounted for. Results are discussed in relation to other recent studies of cross-dialect intelligibility.

ASR for Under-Resourced Languages From Probabilistic Transcription

In many under-resourced languages it is possible to find text, and it is possible to find speech, but transcribed speech suitable for training automatic speech recognition (ASR) is unavailable. In the absence of native transcripts, this paper proposes the use of a probabilistic transcript: A probability mass function over possible phonetic transcripts of the waveform. Three sources of probabilistic transcripts are demonstrated. First, self-training is a well-established semisupervised learning technique, in which a cross-lingual ASR first labels unlabeled speech, and is then adapted using the same labels. Second, mismatched crowdsourcing is a recent technique in which nonspeakers of the language are asked to write what they hear, and their nonsense transcripts are decoded using noisy channel models of second-language speech perception. Third, EEG distribution coding is a new technique in which nonspeakers of the language listen to it, and their electrocortical response signals are interpreted to indicate probabilities. ASR was trained in four languages without native transcripts. Adaptation using mismatched crowdsourcing significantly outperformed self-training, and both significantly outperformed a cross-lingual baseline. Both EEG distribution coding and text-derived phone language models were shown to improve the quality of probabilistic transcripts derived from mismatched crowdsourcing.

Temporal alignment of pupillary response with stimulus events via deconvolution

Analysis of pupil dilation has been used as an index of attentional effort in the auditory domain. Previous work has modeled the pupillary response to attentional effort as a linear time-invariant system with a characteristic impulse response, and used deconvolution to estimate the attentional effort that gives rise to changes in pupil size. Here it is argued that one parameter of the impulse response (the latency of response maximum, t(max)) has been mis-estimated in the literature; a different estimate is presented, and it is shown how deconvolution with this value of t(max) yields more intuitively plausible and informative results.

Pupillometry shows the effort of auditory attention switchinga)

Successful speech communication often requires selective attention to a target stream amidst competing sounds, as well as the ability to switch attention among multiple interlocutors. However, auditory attention switching negatively affects both target detection accuracy and reaction time, suggesting that attention switches carry a cognitive cost. Pupillometry is one method of assessing mental effort or cognitive load. Two experiments were conducted to determine whether the effort associated with attention switches is detectable in the pupillary response. In both experiments, pupil dilation, target detection sensitivity, and reaction time were measured; the task required listeners to either maintain or switch attention between two concurrent speech streams. Secondary manipulations explored whether switch-related effort would increase when auditory streaming was harder. In experiment 1, spatially distinct stimuli were degraded by simulating reverberation (compromising across-time streaming cues), and target-masker talker gender match was also varied. In experiment 2, diotic streams separable by talker voice quality and pitch were degraded by noise vocoding, and the time alloted for mid-trial attention switching was varied. All trial manipulations had some effect on target detection sensitivity and/or reaction time; however, only the attention-switching manipulation affected the pupillary response: greater dilation was observed in trials requiring switching attention between talkers.

Modeling intrinsic intelligibility variation: vowel-space size and structure

This paper describes a model of talker intelligibility in read sentences, based on a variety of vowel-space and prosodic predictors. Data are analysed using mixed-effects regression, including explicit modeling of random variation due to talker, listener, and sentence. Intelligibility is found to correlate with the overall area of the vowel space, the mean area of individual vowel phonemes, and the degree of crowding or encroachment between adjacent phonemes in F2xF1 space. The relationship between talker intelligibility and the prosodic predictors tested remains unclear. The model was tested using a new set of talkers and listeners from a different dialect region, using the same set of sentences. The model did not fully generalize to the second group of talkers and listeners; however, it is argued that differences between the two groups reflect properties of the talker samples rather than genuine dialectal differences.

Investigating the relationship between phoneme categorization and reading ability

Dyslexia is associated with abnormal performance on many auditory psychophysics tasks, particularly those involving the categorization of speech sounds. However, it is debated whether those apparent auditory deficits arise from (a) reduced sensitivity to particular acoustic cues, (b) the difficulty of experimental tasks, or (c) unmodeled lapses of attention. Here we investigate the relationship between phoneme categorization and reading ability, with special attention to the nature of the cue encoding the phoneme contrast (static versus dynamic), differences in task paradigm difficulty, and methodological details of psychometric model fitting. We find a robust relationship between reading ability and categorization performance, show that task difficulty cannot fully explain that relationship, and provide evidence that the deficit is not restricted to dynamic cue contrasts, contrary to prior reports. Finally, we demonstrate that improved modeling of behavioral responses suggests that the gap between dyslexics and typical readers may be smaller than previously reported.

Gender, the individual, and intelligibility

In the clinic and in the laboratory, opinions differ on the relative intelligibility of the speech of women and men. However, the effect of gender alone has rarely been studied explicitly. Here we present a study of 30 talkers (15 male) and 32 listeners assessing intelligibility of a 180-sentence subset of the IEEE sentences presented in steady-state speech-shaped noise. Four signal-to-noise ratios (−4, −2, 0, + 2 dB SNR) were tested with 45 sentences each. Results showed substantial overlap between intelligibility scores for each gender. Although standard statistical approaches show a slight advantage for female talkers at all SNRs, post-hoc analyses indicated that the gender effect is an artifact driven by a few particularly unintelligible males. These results do not address intrinsic gender-related differences in speech intensity, or in the ability to overcome background noise by speaking clearly, but suggest that gender-related differences are negligible when those factors are controlled. More generally, even with a large sample of talkers, the high degree of talker-intrinsic variability in intelligibility can lead to conclusions that do not generalize to the population of interest, an issue that could affect comparisons rooted in gender, dialect, or other social factors. In the clinic and in the laboratory, opinions differ on the relative intelligibility of the speech of women and men. However, the effect of gender alone has rarely been studied explicitly. Here we present a study of 30 talkers (15 male) and 32 listeners assessing intelligibility of a 180-sentence subset of the IEEE sentences presented in steady-state speech-shaped noise. Four signal-to-noise ratios (−4, −2, 0, + 2 dB SNR) were tested with 45 sentences each. Results showed substantial overlap between intelligibility scores for each gender. Although standard statistical approaches show a slight advantage for female talkers at all SNRs, post-hoc analyses indicated that the gender effect is an artifact driven by a few particularly unintelligible males. These results do not address intrinsic gender-related differences in speech intensity, or in the ability to overcome background noise by speaking clearly, but suggest that gender-related differences are negligible when those factors are controlled. More general...

Assessing the fit between phonological feature theories and neural data

Phonological features have been called the most important advance in linguistic theory of the 20th century, yet there is still no consensus regarding their epistemic basis. Many feature systems are based on articulatory properties of consonants and acoustic properties of vowels, others are solely articulatory, and perceptually-based feature systems are rare, despite the fact that phonemes (and by extension, features) allegedly reflect how speech sounds are represented in the brain. Here we examine the fit between several phonological feature systems and electroencephalographic (EEG) recordings made during speech perception. We label EEG data with phonological feature values from different systems, train classifiers to categorize the EEG signals, and assess the accuracy of classifying held-out unlabeled EEG data. By varying only the phonological feature system from which we draw the training labels, we can assess the extent to which neural data recapitulate the patterns inherent in the different feature sy...

Modeling native phonology and non-native speech perception using electroencephalography signals

Studies of language learning in adulthood show that learners’ native language phonologies shape their non-native perception and production abilities. Nonetheless, listeners are able to learn to perceive new speech sound contrasts given training. The present study attempts to model how foreign consonants are perceived prior to training or second language study. Brain responses were recorded during passive listening using electroencephalography (EEG), using twelve monolingual English-native listeners and isolated consonant-vowel (CV) syllables read by speakers of several languages (Dutch, English, Hungarian, Hindi, Swahili). To model native-language phonology, EEG responses to native (English) syllables were used to train classifiers based on phonological feature labels for the consonants in each syllable. The trained classifiers were then applied to the EEG responses to foreign syllables, and the classifier outputs used to generate confusion probabilities between each pair of foreign and native consonants ...