Kuniko Nielsen

Segmental differences in the visual contribution to speech intelligibility

It is well known that the presence of visual cues increases the intelligibility of a speech signal (Sumby and Pollack, 1954). Although much is known about segmental differences in visual‐only perception, little is known about the contribution of visual cues to auditory–visual perception for individual segments. The purpose of this study was to examine (1) whether segments differ in their visual contribution to speech intelligibility, and (2) whether the contribution of visual cues is always to increase speech intelligibility. One talker produced triples of real words containing 15 different English consonants. Forced‐choice word‐identification experiments were carried out with these recordings under auditory–visual (AV) and auditory‐only (A) conditions with varying S/N ratios, and identification accuracy for the 15 consonants was compared for A versus AV conditions. As expected, there were significant differences in the visual contribution for the different consonants, with visual cues greatly improving s...

Imitability of contextual vowel nasalization and interactions with lexical neighborhood density

This study investigates the imitability of contextual vowel nasalization in English. Unlike other phonetic features reported to be imitable [e.g., vowel formants (Babel, 2012), VOT (Nielsen, 2011)], vowel nasality is non-contrastive in English. Nasality is, however, systematically variable: words from dense lexical neighborhoods (high-ND words) are produced with greater nasality than words from sparse neighborhoods [Scarborough (2004), (2012)]. Two experiments were conducted to test (1) whether (experimentally manipulated) nasality can be imitated in a shadowing task, and (2) whether direction of manipulation (more or less nasality, enhancing or countering natural neighborhood-conditioned patterns) affects shadowing behavior. Subjects shadowed 16 high-ND words (which are naturally more nasal) containing a vowel-nasal sequence and modified by spectral mixing to exhibit either greater-than-natural (experiment 1) or less-than-natural (experiment 2) nasality. Both the increase and the decrease in nasality were imitated (though not overall degree of nasality, as our imitation model was more nasal in both conditions than any of our subjects). This change persisted into a post-shadowing task for just the less-nasal condition. These results indicate that speakers are sensitive to non-contrastive phonetic detail in nasality, affecting their subsequent production. Further, the naturalness of nasality (reflecting neighborhood-conditioned variation) may affect the pattern of imitation.

Phonetic imitation of coarticulatory vowel nasalization

This study investigates the spontaneous phonetic imitation of coarticulatory vowel nasalization. Speakers produced monosyllabic words with a vowel-nasal sequence either from dense or sparse phonological neighborhoods in shadowing and word-naming tasks. During shadowing, they were exposed to target words that were modified to have either an artificially increased or decreased degree of coarticulatory vowel nasality. Increased nasality, which is communicatively more facilitative in that it provides robust predictive information about the upcoming nasal segment, was imitated more strongly during shadowing than decreased nasality. An effect of neighborhood density was also observed only in the increased nasality condition, where high neighborhood density words were imitated more robustly in early shadowing repetition. An effect of exposure to decreased nasality was observed during post-shadowing word-naming only. The observed imitation of coarticulatory nasality provides evidence that speakers and listeners are sensitive to the details of coarticulatory realization, and that imitation need not be mediated by abstract phonological representations. Neither a communicative account nor a representational account could single-handedly predict these observed patterns of imitation. As such, it is argued that these findings support both communicative and representational accounts of phonetic imitation.

Phonetic imitation by individuals with Autism Spectrum Disorders: Investigating the role of procedural and declarative memory

This study investigates the role of procedural and declarative memory in phonetic imitation, by examining the word- and phoneme-specificity of imitation produced by individuals with autism spectrum disorders (ASD). Previous research has shown that individuals with ASD process language differently from the Neurotypical population [e.g., Ullman (2004), Walenski et al. (2006)], with Autistic individuals relying more on declarative memory. Previous work with the general population has shown a robust effect of phonetic convergence [e.g., Pardo (2006)], as well as generalization and weak word-specificity effects [Nielsen (2011)]. To test whether individuals with ASD exhibit increased specificity, we used Nielsens (2011) experimental paradigm, which has been shown to elicit generalized phonetic imitation in the general population. A linear mixed effects regression analysis revealed that increased VOT on the modeled phoneme /p/ was imitated by both ASD and control groups [p < 0.05]. However, different patterns e...

Asymmetries in the perception of sub-categorical variation

Both VOT and vowel nasality show low-level variation in English that may be perceptually motivated (and which, in any case, has perceptual consequences). This study examines the perceptual salience of such variation, investigating in particular a possible perceptual asymmetry between increased vs. decreased degrees of these phonologically relevant features. On each trial of an AXB perceptual task, listeners heard three repetitions of the same word: a token with unchanged phonetic feature (X) and tokens with phonetic features that were artificially increased and decreased to the same degree (A and B). Listeners then had to determine which of the two flanking items (A or B) sounded more similar to the middle item (X). Test words included 18 monosyllabic words with initial /p/ (in simple onsets) and 19 monosyllabic words with nasal codas and were heard by 40 listeners. Based on preliminary work, participants are expected to judge decreased-feature stimuli of both types (VOT and nasality) as more similar to t...

Generalization in VOT imitation: Feature adaptation or acoustic-phonetic covariation?

After hearing instances of a word initial voiceless stop with lengthened VOT (e.g., long-[ph]), speakers lengthen their VOTs in unheard words beginning with the same stop ([ph]) and, critically, in words beginning with a different stop ([kh]; Nielsen, 2011). This pattern of generalized phonetic imitation has previously been attributed to talker adaptation at the level of distinctive features or gestural organization. Implicit in these accounts is the assumption that talkers implement the same feature or gesture in a uniform way: generalization from long-[ph] to long-[kh] is an effective and rational adaptation strategy only if VOT values for different stops covary across talkers. Building on previous findings (e.g., Theodore, 2009; Chodroff & Wilson, 2015), we establish that covariation is robust in the laboratory speech of Nielsen (2011). In baseline productions, the mean VOTs of [ph] and [kh] were almost perfectly correlated across participants (r = 0.97, p <.001). Following familiarization the correlat...

Effects of following onsets on voice onset time in English

Voice Onset Time (VOT) in English voiceless stops has been shown to be sensitive to place of articulation (Fischer-Jorgensen 1954), to contextual factors such as the height, tenseness, and duration of the following vowel and the voicing of coda consonants (Klatt 1975, Port & Rotunno 1979), to prosodic factors like stress and pitch (Lisker & Abramson 1967), and also to F0 (McCrea & Morris 2005) and speaking rate (Kessinger & Blumstein 1997, Allen 2003). We report two additional factors involving following consonants. We analyzed 120 /p/- and /k/-initial words produced by 148 Canadian English speakers (n = 17742). VOTs of the initial stops were measured semi-automatically and all other segment durations were measured using forced alignment. The results of a mixed-effects regression support earlier findings that VOT is longer in /k/, directly related to following vowel duration, inversely related to speech rate, longer before tense vowels, and shorter before voiceless codas. Additionally, we find that VOT is shorter when the next syllable starts with a phonetically voiceless plosive (i.e., excluding flapped /t/), and that the most relevant measure of vowel duration includes the duration of postvocalic liquids, even those that are typically analyzed as onsets.

Listeners’ sensitivity to nasal coarticulation and its interactions with lexical neighborhood density

Previous research has shown that words from dense phonological neighborhoods, thus subject to greater lexical competition, are hyperarticulated (Wright, 2004; Munson and Solomon, 2004) and produced with a greater degree of coarticulation (Scarborough, 2013). The current study investigates listeners’ sensitivity to lexically conditioned degree of nasal coarticulation. If the basis of attested Neighborhood Density (ND) effects is listener-oriented (as suggested by Wright), listeners’ sensitivity to different degrees of coarticulation may be influenced by ND patterns as well. Forty-three native speakers of American English participated in a forced-choice discrimination task. The stimuli consisted of 32 monosyllabic words with nasal codas, and the degree of vowel nasality was natural or artificially increased or decreased. A Generalized Linear Mixed Effects regression was performed on Correct Response with Direction of Manipulation and ND as fixed factors. The model revealed a significant interaction of Direc...

Phonetic realization of Japanese vowel devoicing

Despite the widely used term “vowel devoicing”, previous studies have reported that the phonetic realization of Japanese devoicing ranges from devoiced vowel to complete vowel deletion, and is often described as deleted or reduced as opposed to “devoiced” (Beckman, 1982; Keating and Huffman, 1984; Tsuchida, 1997; Kondo, 1997; Maekawa and Kikuchi, 2005, Ogasawara and Warner, 2009). The current study presents an acoustic investigation of phonetic implementation of Japanese vowel devoicing to examine the relative likelihood of vowel devoicing, reduction, and deletion. Twenty-four native speakers of Tokyo Japanese produced 80 words containing high vowels in single devoicing environments. The data revealed that the majority of vowel devoicing tokens (>95%) were phonetically realized as vowel deletions, with no trace of devoiced vowels (= energy excited at frequencies of vowel formants with an aspiration source). Given this result, vowel “deletion” may better characterize the phonetic implementation of Japanese...

Investigating developmental changes in phonological representation using the imitation paradigm

This study investigates the developmental changes in phonological representation, by examining the word- and phoneme-level specificity of phonetic imitation by children. Prevailing linguistic theories assume three levels of phonological representations: word, phoneme, and feature. Previous research suggests that phonological representations develop throughout childhood, and that phonological awareness develops from larger to smaller units (e.g., Edwards et al., 2004; Treiman & Zukowski, 1996). It has been shown that adult speakers implicitly imitate the phonetic properties of recently heard speech (e.g. Goldinger, 1998), and recently, Nielsen (2011) showed the sub-phonemic generalizability and word- and phoneme-level specificity of imitation, indicating that three levels of phonological representations simultaneously contribute to the observed patterns of phonetic imitation. In order to test whether young children manifest similar patterns of imitation and specificity, an experiment with a modified imitat...