Jeffrey J. Berry

The Electromagnetic Articulography Mandarin Accented English (EMA-MAE) corpus of acoustic and 3D articulatory kinematic data

There is a significant need for more comprehensive electromagnetic articulography (EMA) datasets that can provide matched acoustics and articulatory kinematic data with good spatial and temporal resolution. The Marquette University Electromagnetic Articulography Mandarin Accented English (EMA-MAE) corpus provides kinematic and acoustic data from 40 gender and dialect balanced speakers representing 20 Midwestern standard American English L1 speakers and 20 Mandarin Accented English (MAE) L2 speakers, half Beijing region dialect and half are Shanghai region dialect. Three dimensional EMA data were collected at a 400 Hz sampling rate using the NDI Wave system, with articulatory sensors on the midsagittal lips, lower incisors, tongue blade and dorsum, plus lateral lip corner and tongue body. Sensors provide three-dimensional position data as well as two-dimensional orientation data representing the orientation of the sensor plane. Data have been corrected for head movement relative to a fixed reference sensor and also adjusted using a biteplate calibration system to place the data in an articulatory working space relative to each subjects individual midsagittal and maxillary occlusal planes. Speech materials include isolated words chosen to focus on specific contrasts between the English and Mandarin languages, as well as sentences and paragraphs for continuous speech, totaling approximately 45 minutes of data per subject. A beta version of the EMA-MAE corpus is now available, and the full corpus is in preparation for public release to help advance research in areas such as pronunciation modeling, acoustic-articulatory inversion, L1-L2 comparisons, pronunciation error detection, and accent modification training.

Covariation among vowel height effects on acoustic measures

Covariation among vowel height effects on vowel intrinsic fundamental frequency (IF(0)), voice onset time (VOT), and voiceless interval duration (VID) is analyzed to assess the plausibility of a common physiological mechanism underlying variation in these measures. Phrases spoken by 20 young adults, containing words composed of initial voiceless stops or /s/ and high or low vowels, were produced in habitual and voluntarily increased F(0) conditions. High vowels were associated with increased IF(0) and longer VIDs. VOT and VID exhibited significant covariation with IF(0) only for males at habitual F(0). The lack of covariation for females and at increased F(0) is discussed.

Speaking rate effects on locus equation slope

A locus equation describes a 1st order regression fit to a scatter of vowel steady-state frequency values predicting vowel onset frequency values. Locus equation coefficients are often interpreted as indices of coarticulation. Speaking rate variations with a constant consonant-vowel form are thought to induce changes in the degree of coarticulation. In the current work, the hypothesis that locus slope is a transparent index of coarticulation is examined through the analysis of acoustic samples of large-scale, nearly continuous variations in speaking rate. Following the methodological conventions for locus equation derivation, data pooled across ten vowels yield locus equation slopes that are mostly consistent with the hypothesis that locus equations vary systematically with coarticulation. Comparable analyses between different four-vowel pools reveal variations in the locus slope range and changes in locus slope sensitivity to rate change. Analyses across rate but within vowels are substantially less consistent with the locus hypothesis. Taken together, these findings suggest that the practice of vowel pooling exerts a non-negligible influence on locus outcomes. Results are discussed within the context of articulatory accounts of locus equations and the effects of speaking rate change.

Parallel Reference Speaker Weighting for Kinematic-Independent Acoustic-to-Articulatory Inversion

Acoustic-to-articulatory inversion, the estimation of articulatory kinematics from an acoustic waveform, is a challenging but important problem. Accurate estimation of articulatory movements has the potential for significant impact on our understanding of speech production, on our capacity to assess and treat pathologies in a clinical setting, and on speech technologies such as computer aided pronunciation assessment and audio-video synthesis. However, because of the complex and speaker-specific relationship between articulation and acoustics, existing approaches for inversion do not generalize well across speakers. As acquiring speaker-specific kinematic data for training is not feasible in many practical applications, this remains an important and open problem. This paper proposes a novel approach to acoustic-to-articulatory inversion, Parallel Reference Speaker Weighting (PRSW), which requires no kinematic data for the target speaker and a small amount of acoustic adaptation data. PRSW hypothesizes that acoustic and kinematic similarities are correlated and uses speaker-adapted articulatory models derived from acoustically derived weights. The system was assessed using a 20-speaker data set of synchronous acoustic and Electromagnetic Articulography (EMA) kinematic data. Results demonstrate that by restricting the reference group to a subset consisting of speakers with strong individual speaker-dependent inversion performance, the PRSW method is able to attain kinematic-independent acoustic-to-articulatory inversion performance nearly matching that of the speaker-dependent model, with an average correlation of 0.62 versus 0.63. This indicates that given a sufficiently complete and appropriately selected reference speaker set for adaptation, it is possible to create effective articulatory models without kinematic training data.

Tracking articulator movements using orientation measurements

This paper introduces a new method to track articulator movements, specifically jaw position and angle, using 5 degree of freedom (5 DOF) orientation data. The approach uses a quaternion rotation method to accomplish this jaw tracking during speech using a single senor on the mandibular incisor. Data were collected using the NDI Wave Speech Research System for one pilot subject with various speech tasks. The degree of jaw rotation from the proposed approach is compared with traditional geometric calculation. Results show that the quaternion based method is able to describe jaw angle trajectory and gives more accurate and smooth estimation of jaw kinematics.

Control of short lag VOT (voice‐onset time) for voiced English stops

The American English stops /b/, /d/, and /g/ are produced with negative VOT (−150–0 ms) or short‐lag VOT (0–30 ms). These two variants have been shown to occur with approximately equal frequency across speakers, although individual preferences are common. It has been hypothesized that the characteristic VOTs of short‐lag stops are determined by relatively simpler articulatory control mechanisms than negative VOT variants. Specifically, if the vocal folds are approximated prior to constriction release, and intraoral pressure is vented only at the point of maximum constriction (without supraglottal expansion), the VOT will be determined by the venting process controlled by the time‐varying constriction aperture. Thus, the release kinematics could account for effects of place of articulation and vowel context on short‐lag VOTs. In the current work, the plausibility of this hypothesis is evaluated. The kinematics of constriction release gestures was examined from x‐ray microbeam data. Descriptions of the time...

A Kinematic Analysis of Coarticulation Effects on Schwa

Purpose: This study investigated coarticulatory effects on schwa. The purpose was to establish a viable phonetic environment and analysis for future studies of coarticulation, particularly in motor speech disorders. Method: Seven female adults read 2 phrases containing 4 target words in a CVC structure embedded in the carrier phrases, “Put a CVC here” and “Get CVC a puppy.” A 3D electromagnetic articulography system was used to track lingual movements during schwa production. Two indices of coarticulation were employed: (a) tongue position difference between the temporal midpoint (grand mean of each speaker) and the onset/offset of schwa immediately following or preceding a transconsonantal vowel and (b) tongue position differences between the grand mean of each speaker and each utterance repetition, both measured at the temporal midpoint of the schwa. Results: Both measures indicated that schwa is significantly influenced by transconsonantal corner vowels. The magnitude of coarticulation effects on schwa varied among the 4 corner vowels in the order of /i/>/ɑ/ = /æ/>/u/. Conclusion: Findings support the use of schwa in future studies when examining coarticulatory effects in people with and without motor speech disorders. Some methodological issues such as selection of measurement points and speech stimuli are also discussed.

Jaw Rotation in Dysarthria Measured With a Single Electromagnetic Articulography Sensor

Purpose This study evaluated a novel method for characterizing jaw rotation using orientation data from a single electromagnetic articulography sensor. This method was optimized for clinical application, and a preliminary examination of clinical feasibility and value was undertaken. Method The computational adequacy of the single-sensor orientation method was evaluated through comparisons of jaw-rotation histories calculated from dual-sensor positional data for 16 typical talkers. The clinical feasibility and potential value of single-sensor jaw rotation were assessed through comparisons of 7 talkers with dysarthria and 19 typical talkers in connected speech. Results The single-sensor orientation method allowed faster and safer participant preparation, required lower data-acquisition costs, and generated less high-frequency artifact than the dual-sensor positional approach. All talkers with dysarthria, regardless of severity, demonstrated jaw-rotation histories with more numerous changes in movement direction and reduced smoothness compared with typical talkers. Conclusions Results suggest that the single-sensor orientation method for calculating jaw rotation during speech is clinically feasible. Given the preliminary nature of this study and the small participant pool, the clinical value of such measures remains an open question. Further work must address the potential confound of reduced speaking rate on movement smoothness.

Sensorimotor adaptation of speech using real-time articulatory resynthesis

Sensorimotor adaptation is an important focus in the study of motor learning for non-disordered speech, but has yet to be studied substantially for speech rehabilitation. Speech adaptation is typically elicited experimentally using LPC resynthesis to modify the sounds that a speaker hears himself producing. This method requires that the participant be able to produce a robust speech-acoustic signal and is therefore not well-suited for talkers with dysarthria. We have developed a novel technique using electromagnetic articulography (EMA) to drive an articulatory synthesizer. The acoustic output of the articulatory synthesizer can be perturbed experimentally to study auditory feedback effects on sensorimotor learning. This work aims to compare sensorimotor adaptation effects using our articulatory resynthesis method with effects from an established, acoustic-only method. Results suggest that the articulatory resynthesis method can elicit speech adaptation, but that the articulatory effects of the two methods differ.

Acoustic effects of speaking rate changes in articulatory models

Speaking rate effects on articulatory phasing, velocity, and magnitude have been reported in the speech‐kinematic literature. Yet little is known about the acoustic consequences of manipulating these dimensions. Using an articulatory modeling method developed by Milenkovic, simulations of the acoustic effects of changes in articulatory phasing, velocity, and magnitude have been explored. Articulatory models have been optimized for speakers from the University of Wisconsin X‐Ray Microbeam Speech Production Database. Preliminary work has revealed that models of rate change which use only a single dimension (e.g., phasing) may reveal less plausible acoustic solutions than models involving multiple dimensions. Using sample productions of ‘‘a boy’’ to determine articulatory parameter time histories, the acoustic consequences of rate change simulations have been described by the F2 value at onset (F2on), the maximum F2 value (F2tar), and the shape of the F2 trajectory throughout the diphthong. Speaking rate mod...