Kerry P. Green

Speaking Rate and Segments: A Look at the Relation between Speech Production and Speech Perception for the Voicing Contrast

When listeners process segmentally relevant properties of the speech signal they do so in a rate-dependent manner. This is seen as a shift in the perceptual category boundary; as rate declines and overall syllable duration increases, the category boundary moves toward a longer value of the acoustic property in question. Focusing on the /b/-/p/ distinction specified by voice onset time (VOT), we investigated whether the acoustic modifications that occur with an alteration in speaking rate accord with this pattern of perceptual boundary shift. Two main findings emerged. First, as speaking rate became slower and overall syllable duration became longer, the VOT value of the consonant, especially that of the voiceless /p/, also became longer. Second, and most important, the VOT value that optimally separated the /b/ and /p/ VOT distributions also changed with rate, increasing with increasing syllable duration. However, the magnitude of the boundary shift obtained for these production data was greater than that typically found in perceptual experiments. This suggests the existence of constraints on the extent to which the perceptual system can accommodate for alterations due to rate of speech.

On the role of visual rate information in phonetic perception.

It is well established that listeners process segmentally relevant properties of the speech signal in relation to the rate at which the speech was produced. We investigated whether the critical rate information for this effect is limited to the auditory modality or, alternatively, whether visual rate information provided by the talker’s face also plays a role. Audio-visual syllables were created by pairing tokens from a moderate-rate, auditory /bi/-/pi/ series with visual tokens of /bi/ or /pi/ produced at a faster or slower rate of speech; these visual tokens provided information about speaking rate, but could not themselves be identified correctly as /bi/ or /pi/. Each audiovisual pairing produced the phenomenal experience of a single, unified syllable, /bi/ or /pi/, spoken at a single rate of speech. The change in visual rate information across the syllables influenced the judged rate of the audio-visual syllables and, more importantly, affected their identification as /bi/ or /pi/. These results indicate that visual information about speaking rate is relevant to the perception of voicing and, more generally, suggest that the mechanisms underlying rate-dependent speech processing have a bimodal (or amodal) component.

A distinction between the effects of sentential speaking rate and semantic congruity on word identification

It is well known that numerous aspects of sentential context can influence the manner in which a word within the sentence is identified. We investigated two such contextual effects, that of the speaking rate of the sentence in which the target word occurs and that of the semantic congruence between the sentence and the target word. We observed that although the two effects are similar on the surface, in that each is realized as a change in the identification of acoustically ambiguous (but not unambiguous) items along a speech series, they are strikingly different in their susceptibility to changes in task demands. Specifically, changes in the task that readily eliminate the semantic congruity effect do not serve to eliminate the rate effect, suggesting that the two effects arise at different stages of analysis. The implications of this finding for models of speech processing are discussed.

Factors affecting the integration of auditory and visual information in speech: The effect of vowel environment

In the McGurk effect, observers typically report the illusory syllable /da/ when they hear the auditory syllable /ba/ presented in synchrony with a video display of a talker saying /ga/. While the effect itself has been well established, there is still little research on the conditions under which the effect occurs. In the experiment reported here, the number of illusory /d/ responses to the auditory /b/‐visual /g/ combination is examined in three vowel environments: /a/, /i/, and /u/. The results of this study indicate that the magnitude of the illusion is not the same across different vowel environments. It appears to be strongest for the /i/ vowel, moderate for /a/, and almost nonexistent for /u/. The results thus show that vowel environment is an important factor in determining the magnitude of the McGurk effect, which needs to be considered in accounts of auditory‐visual integration during speech perception. [Work supported by NIH.]

On the nature of rate-dependent processing during phonetic perception

We previously reported (Miller& Liberman, 1979) that when listeners use transition duration to identify syllables as /ba/or /wa/, they do so in relation to the overall duration of the syllables: As the syllables become longer and are perceived as having been produced at a slower rate of speech, the transition duration that differentiates /ba/ from /wa/ also becomes longer. In the present experiments, we investigated whether the adjustment for rate was based on the actual physical rate of the syllables (i.e., syllable duration) or on their subjective rate. We used an anchoring procedure to alter the subjective rate of the syllables while keeping physical rate constant, and then assessed the effect of this manipulation on the identification of the syllables as /ba/ or /wa/.We found that changes in subjective rate induced by the anchoring procedure had no reliable effect on syllable identification. However, in a control study, changes in physical rate that produced equivalent changes in subjective rate did reliably alter identification of the syllables. These findings indicate that, during phonetic perception, listeners accommodate for changes in the physical rate of speech, not for changes in its subjective rate.

The influence of an inverted face on the McGurk effect

The McGurk effect is a situation in which the perception of the speech signal is influenced by watching a talker’s face articulate a conflicting speech utterance. For example, observers typically report the syllable /da/ when the auditory syllable /ba/ is presented in synchrony with a videotape of the talker saying the syllable /ga/. It is well‐known that inverted faces are much more difficult to recognize than faces presented in a normal, upright orientation. The current study investigated whether inverting the talker’s face would also influence the McGurk effect. Two groups of subjects were presented with a videotape consisting of conflicting auditory and visual syllables designed to produce the typical McGurk effect. The first group viewed the test tape on a video monitor set in its normal upright position. This group showed a typical, robust McGurk effect. The second group viewed the videotape on the monitor set in an inverted position. This group showed a significantly weaker McGurk effect. Thus inve...

The perception of speaking rate using visual information from a talker’s face

It has been well documented that listeners are able to estimate speaking rate when listening to a talker, but almost no work has been done on perception of rate information provided by looking at a talker’s face. In the present study, the method of magnitude estimation was used to collect estimates of the rate at which a talker was speaking. The estimates were collected under four experimental conditions: auditory only, visual only, combined auditory-visual, and inverted visual only. The results showed no difference in the slope of the functions relating perceived rate to physical rate for the auditory only, visual only, and combined auditory-visual presentations. There was, however, a significant difference between the normal visual-only and the inverted-visual presentations. These results indicate that there is visual rate information available on a talker’s face and, more importantly, suggest that there is a correspondence between the auditory and visual modalities for the perception of speaking rate, but only when the visual information is presented in its normal orientation.

An examination of word–initial‐stop closure interval in English, Spanish, and Spanish–English bilinguals

This study investigated the effect of the linguistic experience on the duration of the preceding closure interval (CI) on word–initial‐stop consonants. Native speakers of English (NE), Spanish (NS), and Spanish–English bilinguals produced sentences containing words beginning with either a voiced or a voiceless stop consonant. As is typical for word–initial stops in English, no difference in the CI occurred between voiced and voiceless consonants for the NE speakers. The NS speakers produced the voiceless stops with CIs similar to the NE voiced stops (consistent with the fact that both are classified phonetically as short‐lag stops). The voiced stops, however, had significantly shorter CIs. Like the NE speakers, the bilinguals in the English mode produced voiced and voiceless stops with equal CIs. In the Spanish mode, the bilinguals maintained a distinction in CI between voiced and voiceless stops, although the CIs were significantly different from their NS counterparts. The results suggest that Spanish sp...

On the distinction between prosodic and semantic factors in word identification

Two factors contributing to the identification of a word are the prosody of the sentence in which the word occurs and the semantic relation between the word and its context sentence. To investigate further these two factors, we created a series of target words that ranged from “bath” to “path” by systematically varying word‐initial voice‐onset‐time, placed these words in a number of different sentence frames, and asked listeners to identify each word as “bath” or “path.” Both the speaking rate of the sentence and the semantic congruity between the target word and the sentence affected the way in which acoustically ambiguous words were identified. However, the effects were quite different in nature. Whereas a change in speaking rate influenced word identification even when the listener was not required by task demands to attend to the context sentence, a change in semantic congruity altered identification only when attention to the context sentence was required. This suggests that in the course of identify...

Factors affecting the integration of auditory and visual information in speech: The level effect

The “McGurk effect” is a phenomenon in which an illusory syllable is perceived when discrepant auditory and visual information are combined. For example, an illusory /da/ is perceived when an auditory syllable /ba/ is paired with a video display of a talker saying /ga/. In a series of studies reported here, it was found that the level of the auditory signal substantially affects the illusion. Moreover, this effect of auditory level works in a direction that is counterintuitive. One might have postulated that as the level of the auditory signal is increased, making the auditory signal more pronounced, the illusion would decrease. These results indicate just the opposite. As the level of the auditory signal goes up, rather than it outweighing the visual signal and thereby decreasing the illusion, the effect is to increase the number of illusory responses. Paired were the same auditory syllable /ba/ at three levels, soft (45 dB SPL), moderate (58 dB SPL), and loud (66 dB SPL), with a video display of /ga/. Significant increases in illusory /da/ responses occurred as the auditory signal increased from soft to moderate and again from moderate to loud. The work thus shows that auditory level is an important determinant in the perception of the illusion. The paradox of the situation is that as the auditory signal increases in strength, the resulting percept becomes less veridical rather than more so. [Work supported by NIH.]