Jean C. Krause

Acoustic properties of naturally produced clear speech at normal speaking rates

Sentences spoken “clearly” are significantly more intelligible than those spoken “conversationally” for hearing-impaired listeners in a variety of backgrounds [Picheny et al., J. Speech Hear. Res. 28, 96–103 (1985); Uchanski et al., ibid. 39, 494–509 (1996); Payton et al., J. Acoust. Soc. Am. 95, 1581–1592 (1994)]. While producing clear speech, however, talkers often reduce their speaking rate significantly [Picheny et al., J. Speech Hear. Res. 29, 434–446 (1986); Uchanski et al., ibid. 39, 494–509 (1996)]. Yet speaking slowly is not solely responsible for the intelligibility benefit of clear speech (over conversational speech), since a recent study [Krause and Braida, J. Acoust. Soc. Am. 112, 2165–2172 (2002)] showed that talkers can produce clear speech at normal rates with training. This finding suggests that clear speech has inherent acoustic properties, independent of rate, that contribute to improved intelligibility. Identifying these acoustic properties could lead to improved signal processing schemes for hearing aids. To gain insight into these acoustical properties, conversational and clear speech produced at normal speaking rates were analyzed at three levels of detail (global, phonological, and phonetic). Although results suggest that talkers may have employed different strategies to achieve clear speech at normal rates, two global-level properties were identified that appear likely to be linked to the improvements in intelligibility provided by clear/normal speech: increased energy in the 1000–3000-Hz range of long-term spectra and increased modulation depth of low frequency modulations of the intensity envelope. Other phonological and phonetic differences associated with clear/normal speech include changes in (1) frequency of stop burst releases, (2) VOT of word-initial voiceless stop consonants, and (3) short-term vowel spectra.

Development of speechreading supplements based on automatic speech recognition

In manual-cued speech (MCS) a speaker produces hand gestures to resolve ambiguities among speech elements that are often confused by speechreaders. The shape of the hand distinguishes among consonants; the position of the hand relative to the face distinguishes among vowels. Experienced receivers of MCS achieve nearly perfect reception of everyday connected speech. MCS has been taught to very young deaf children and greatly facilitates language learning, communication, and general education. This manuscript describes a system that can produce a form of cued speech automatically in real time and reports on its evaluation by trained receivers of MCS. Cues are derived by a hidden markov models (HMM)-based speaker-dependent phonetic speech recognizer that uses context-dependent phone models and are presented visually by superimposing animated handshapes on the face of the talker. The benefit provided by these cues strongly depends on articulation of hand movements and on precise synchronization of the actions of the hands and the face. Using the system reported here, experienced cue receivers can recognize roughly two-thirds of the keywords in cued low-context sentences correctly, compared to roughly one-third by speechreading alone (SA). The practical significance of these improvements is to support fairly normal rates of reception of conversational speech, a task that is often difficult via SA.

Evaluating the role of spectral and envelope characteristics in the intelligibility advantage of clear speech

In adverse listening conditions, talkers can increase their intelligibility by speaking clearly [Picheny, M.A., et al. (1985). J. Speech Hear. Res. 28, 96-103; Payton, K. L., et al. (1994). J. Acoust. Soc. Am. 95, 1581-1592]. This modified speaking style, known as clear speech, is typically spoken more slowly than conversational speech [Picheny, M. A., et al. (1986). J. Speech Hear. Res. 29, 434-446; Uchanski, R. M., et al. (1996). J. Speech Hear. Res. 39, 494-509]. However, talkers can produce clear speech at normal rates (clear/normal speech) with training [Krause, J. C., and Braida, L. D. (2002). J. Acoust. Soc. Am. 112, 2165-2172] suggesting that clear speech has some inherent acoustic properties, independent of rate, that contribute to its improved intelligibility. Identifying these acoustic properties could lead to improved signal processing schemes for hearing aids. Two global-level properties of clear/normal speech that appear likely to be associated with improved intelligibility are increased energy in the 1000-3000-Hz range of long-term spectra and increased modulation depth of low-frequency modulations of the intensity envelope [Krause, J. C., and Braida, L. D. (2004). J. Acoust. Soc. Am. 115, 362-378]. In an attempt to isolate the contributions of these two properties to intelligibility, signal processing transformations were developed to manipulate each of these aspects of conversational speech independently. Results of intelligibility testing with hearing-impaired listeners and normal-hearing listeners in noise suggest that (1) increasing energy between 1000 and 3000 Hz does not fully account for the intelligibility benefit of clear/normal speech, and (2) simple filtering of the intensity envelope is generally detrimental to intelligibility. While other manipulations of the intensity envelope are required to determine conclusively the role of this factor in intelligibility, it is also likely that additional properties important for highly intelligible speech at normal speech at normal rates remain to be identified.

An adaptive clinical test of temporal resolution: Within-channel and across-channel gap detection

Abstract Objective: Several methods exist to measure temporal resolution in a clinical setting. The Adaptive Tests of Temporal Resolution (ATTR©) are unique in that they incorporate an adaptive psychophysical procedure to present stimuli via computer sound card. The purpose of this study was to determine if ATTR gap detection thresholds (GDTs) were stable across presentation levels (80 dB SPL, listener-selected level) and sound cards (high-end, inexpensive). Design: GDTs were measured using three conditions of the ATTR: broad-band noise within-channel (BBN-WC), narrowband noise within-channel (NBN-WC), and narrowband noise across-channel (NBN-AC). Analysis of the acoustical properties of ATTR stimuli was made by measuring the electrical signals produced by each sound card. Study Sample: Participants were 30 young adults with normal hearing. Results: The ATTR GDTs did not differ between presentation levels for all three stimulus conditions. Also, neither ATTR stimuli nor ATTR GDTs differed between sound cards for all conditions. Conclusions: The ATTR may be used in a clinical setting with a relatively inexpensive sound card and listener-selected levels. Normative performance values for each ATTR condition are provided. Sumario Objetivo: Existen varios métodos en la clínica para medir la resolución temporal. La prueba adaptativa para la resolución temporal (ATTR@) es u´nica en el sentido en que incorpora un procedimiento psico-físico para presentar estímulos por medio de una tarjeta de sonidos de una computadora. El propósito de este estudio fue determinar si los umbrales de detección de brechas (GDT) de la ATTR eran estables en diferentes niveles de presentación (80 dB SPL, nivel seleccionado por el oyente) y con diferentes tarjetas de sonido (hig-end, barata). Diseño: Los GDT fueron medidos en tres condiciones de ATTR: banda ancha, ruido en el canal (BBN-WC), ruido de banda angosta en el canal (NBN-WC) y ruido de banda angosta a través del canal (NBN-AC). El análisis de las propiedades acústicas de los estímulos de ATTR fue realizado midiendo las señales eléctricas producidas por cada tarjeta de sonido. Muestra: Los participantes fueron 30 adultos jóvenes con audición normal. Resultados: Los GDT ATTR no difirieron en los niveles de presentación en las tres condiciones de estímulo. Además, ni los estímulos de ATTR ni los GDT ATTR difirieron en con las diferentes tarjetas de sonido en todas las condiciones. Conclusiones: Los ATTR pueden ser utilizados en condiciones clínicas con una tarjeta de sonido relativamente barata y a niveles seleccionados por el oyente. Se proporcionan los valores normativos de desempeño para cada condición de ATTR.

The effects of speaking rate on the intelligibility of speech for various speaking modes

In adverse listening conditions, large and robust increases in intelligibility can be achieved by speaking clearly. The most striking differences between clear and conversational speech are associated with differences in speaking rate. To understand these differences, the intelligibility of speech in a variety of speaking modes was investigated at three different speaking rates. Talkers with significant speaking experience were asked to produce clear and conversational speech at slow, conversational, and quick rates. Previous studies show that the speaking rate for clear speech is roughly one‐half that of conversational speech. Therefore, during training, the talkers were given feedback on their intelligibility in order to elicit the clearest possible speech at conversational and quick speaking rates. Talkers also recorded sentences in several other speaking modes such as soft, loud, and conversational with pauses inserted, as required for input to some automatic speech recognition systems. All speech mat...

Toward Extending the Educational Interpreter Performance Assessment to Cued Speech

The Educational Interpreter Performance Assessment (EIPA) is as an important research tool for examining the quality of interpreters who use American Sign Language or a sign system in classroom settings, but it is not currently applicable to educational interpreters who use Cued Speech (CS). In order to determine the feasibility of extending the EIPA to include CS, a pilot EIPA test was developed and administered to 24 educational CS interpreters. Fifteen of the interpreters’ performances were evaluated two to three times in order to assess reliability. Results show that the instrument has good construct validity and test–retest reliability. Although more interrater reliability data are needed, intrarater reliability was quite high (0.9), suggesting that the pilot test can be rated as reliably as signing versions of the EIPA. Notably, only 48% of interpreters who formally participated in pilot testing performed at a level that could be considered minimally acceptable. In light of similar performance levels previously reported for interpreters who sign (e.g., Schick, Williams, & Kupermintz, 2006), these results suggest that interpreting services for deaf and hard-of hearing students, regardless of the communication option used, are often inadequate and could seriously hinder access to the classroom environment.

A method for transcribing the manual components of Cued Speech

Designed to allow visual communication of speech signals, Cued Speech consists of discrete hand signals that are produced in synchrony with the visual mouth movements of speech. The purpose of this paper is to describe a method for transcribing these hand signals. Procedures are presented for identifying 1) the steady-state portion of the cue to be analyzed, 2) the cues handshape, and 3) the cues placement. Reliability is evaluated, using materials from 12 cuers that were transcribed on two separate occasions (either by the original rater or a second rater). Results show very good intra-rater and inter-rater reliability on average, which remained good across a variety of individual cuers, even when the cuers hand gestures were heavily coarticulated. Given its high reliability, this transcription method may be of benefit to applications that require systematic and quantitative analysis of Cued Speech production in various populations. In addition, some of the transcription principles from this method may be helpful in improving accuracy of automatic Cued Speech recognition systems.

Cued Speech Transliteration: Effects of Accuracy and Lag Time on Message Intelligibility

This paper is the second in a series concerned with the level of access afforded to students who use educational interpreters. The first paper (Krause & Tessler, 2016) focused on factors affecting accuracy of messages produced by Cued Speech (CS) transliterators (expression). In this study, factors affecting intelligibility (reception by deaf consumers) of those messages is explored. Results for eight highly skilled receivers of CS showed that (a) overall intelligibility (72%) was higher than average accuracy (61%), (b) accuracy had a large positive effect on intelligibility, accounting for 26% of the variance, and (c) the likelihood that an utterance reached 70% intelligibility as a function of accuracy was sigmoidal, decreasing sharply for accuracy values below 65%. While no direct effect of lag time on intelligibility could be detected, intelligibility was most likely to exceed 70% for utterances produced with lag times between 0.6 and 1.8 s. Differences between CS transliterators suggested sources of transliterator variability (e.g. speechreadability, facial expression, non-manual markers, cueing rate) that are also likely to affect intelligibility and thus warrant further investigation. Such research, along with investigations of other communication options (e.g. American Sign Language, signed English systems, etc.) are important for ensuring accessibility to all students who use educational interpreters.

Cued Speech Transliteration: Effects of Speaking Rate and Lag Time on Production Accuracy.

Many deaf and hard-of-hearing children rely on interpreters to access classroom communication. Although the exact level of access provided by interpreters in these settings is unknown, it is likely to depend heavily on interpreter accuracy (portion of message correctly produced by the interpreter) and the factors that govern interpreter accuracy. In this study, the accuracy of 12 Cued Speech (CS) transliterators with varying degrees of experience was examined at three different speaking rates (slow, normal, fast). Accuracy was measured with a high-resolution, objective metric in order to facilitate quantitative analyses of the effect of each factor on accuracy. Results showed that speaking rate had a large negative effect on accuracy, caused primarily by an increase in omitted cues, whereas the effect of lag time on accuracy, also negative, was quite small and explained just 3% of the variance. Increased experience level was generally associated with increased accuracy; however, high levels of experience did not guarantee high levels of accuracy. Finally, the overall accuracy of the 12 transliterators, 54% on average across all three factors, was low enough to raise serious concerns about the quality of CS transliteration services that (at least some) children receive in educational settings.

Using Landmark Detection to measure effective Clear Speech

While the relationship of clear speaking style to intelligibility under challenging conditions has been established, it is a common observation that some speakers seem to be more intelligible than others for most listeners. In previous work, we have reported that automatic measures based on the technique of Landmark Detection appear to track differences between Clear and Conversational speaking style. One question that remains is whether Landmark measures can be used to predict which speakers are most likely to produce highly intelligible speech. In this study, we took advantage of a set of previously acquired databases to examine correlations between our Landmark-based measures and the Clear Speech productions of highly intelligible speech. Across these databases, we had data on intelligibility for a range of speakers. Results showed that speakers with high overall intelligibility in Clear Speech showed significantly different patterns on Landmark-based automatic measures, compared to speakers with more moderate performance on intelligibility measures. Applications of these results to problems in speech technology, linguistic education and clinical practice will be discussed.