Pierre Badin

Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images.

In this study, previous articulatory midsagittal models of tongue and lips are extended to full three-dimensional models. The geometry of these vocal organs is measured on one subject uttering a corpus of sustained articulations in French. The 3D data are obtained from magnetic resonance imaging of the tongue, and from front and profile video images of the subjects face marked with small beads. The degrees of freedom of the articulators, i.e., the uncorrelated linear components needed to represent the 3D coordinates of these articulators, are extracted by linear component analysis from these data. In addition to a common jaw height parameter, the tongue is controlled by four parameters while the lips and face are also driven by four parameters. These parameters are for the most part extracted from the midsagittal contours, and are clearlyinterpretable in phonetic/biomechanical terms. This implies that most 3D features such as tongue groove or lateral channels can be controlled by articulatory parameters defined for the midsagittal model. Similarly, the 3D geometry of the lips is determined by parameters such as lip protrusion or aperture, that can be measured from a profile view of the face.

Can you 'read' tongue movements? Evaluation of the contribution of tongue display to speech understanding

Lip reading relies on visible articulators to ease speech understanding. However, lips and face alone provide very incomplete phonetic information: the tongue, that is generally not entirely seen, carries an important part of the articulatory information not accessible through lip reading. The question is thus whether the direct and full vision of the tongue allows tongue reading. We have therefore generated a set of audiovisual VCV stimuli with an audiovisual talking head that can display all speech articulators, including tongue, in an augmented speech mode. The talking head is a virtual clone of a human speaker and the articulatory movements have also been captured on this speaker using ElectroMagnetic Articulography (EMA). These stimuli have been played to subjects in audiovisual perception tests in various presentation conditions (audio signal alone, audiovisual signal with profile cutaway display with or without tongue, complete face), at various Signal-to-Noise Ratios. The results indicate: (1) the possibility of implicit learning of tongue reading, (2) better consonant identification with the cutaway presentation with the tongue than without the tongue, (3) no significant difference between the cutaway presentation with the tongue and the more ecological rendering of the complete face, (4) a predominance of lip reading over tongue reading, but (5) a certain natural human capability for tongue reading when the audio signal is strongly degraded or absent. We conclude that these tongue reading capabilities could be used for applications in the domains of speech therapy for speech retarded children, of perception and production rehabilitation of hearing impaired children, and of pronunciation training for second language learners.

Linear degrees of freedom in speech production: Analysis of cineradio- and labio-film data and articulatory-acoustic modeling

The following contribution addresses several issues concerning speech degrees of freedom in French oral vowels, stop, and fricative consonants based on an analysis of tongue and lip shapes extracted from cineradio- and labio-films. The midsagittal tongue shapes have been submitted to a linear decomposition where some of the loading factors were selected such as jaw and larynx position while four other components were derived from principal component analysis (PCA). For the lips, in addition to the more traditional protrusion and opening components, a supplementary component was extracted to explain the upward movement of both the upper and lower lips in [v] production. A linear articulatory model was developed; the six tongue degrees of freedom were used as the articulatory control parameters of the midsagittal tongue contours and explained 96% of the tongue data variance. These control parameters were also used to specify the frontal lip width dimension derived from the labio-film front views. Finally, this model was complemented by a conversion model going from the midsagittal to the area function, based on a fitting of the midsagittal distances and the formant frequencies for both vowels and consonants.

The vocal tract of newborn humans and Neanderthals: Acoustic capabilities and consequences for the debate on the origin of language. A reply to Lieberman (2007a)

In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elseviers archiving and manuscript policies are encouraged to visit:

A three-dimensional articulatory model of the velum and nasopharyngeal wall based on MRI and CT data

An original three-dimensional (3D) linear articulatory model of the velum and nasopharyngeal wall has been developed from magnetic resonance imaging (MRI) and computed tomography images of a French subject sustaining a set of 46 articulations, covering his articulatory repertoire. The velum and nasopharyngeal wall are represented by generic surface triangular meshes fitted to the 3D contours extracted from MRI for each articulation. Two degrees of freedom were uncovered by principal component analysis: first, VL accounts for 83% of the velum variance, corresponding to an oblique vertical movement seemingly related to the levator veli palatini muscle; second, VS explains another 6% of the velum variance, controlling a mostly horizontal movement possibly related to the sphincter action of the superior pharyngeal constrictor. The nasopharyngeal wall is also controlled by VL for 47% of its variance. Electromagnetic articulographic data recorded on the velum fitted these parameters exactly, and may serve to recover dynamic velum 3D shapes. The main oral and nasopharyngeal area functions controlled by the articulatory model, complemented by the area functions derived from the complex geometry of each nasal passage extracted from coronal MRIs, were fed to an acoustic model and gave promising results about the influence of velum movements on the spectral characteristics of nasals.

Vocalic nomograms: Acoustic and articulatory considerations upon formant convergences

Fant [G. Fant, Acoustic Theory of Speech Production (Mouton, The Hague, 1960)] has been the first to use the four‐tube model of the vocal tract and to introduce vocalic nomograms to speech production studies. These nomograms proved to be an efficient tool for the study of articulatory–acoustic relationships. This paper deals with particular regions of these nomograms, referred to as focal points. These are points where formant convergences occur and where affiliations between formants and cavities are exchanged. A study of appropriate nonsense words has confirmed the existence of these points in natural speech. Generating nomograms with an articulatory model of the vocal tract allowed the location of these convergence regions along the vocal tract, and demonstrated that vowels [i,a,u,y] are located in the vicinity of focal points. The examination of the human speakers’ nomograms obtained by Ladefoged and Bladon from real speaker productions [P. Ladefoged and R. A. W. Bladon, Speech Commun. 1, 185–198 (198...

Development of the transmission line matrix method in acoustics applications to higher modes in the vocal tract and other complex ducts

Most traditional theories of speech production are currently based on plane waves. However, it is well known that, for acoustic waveguides, higher acoustical modes start to propagate and can become predominant above cut-on frequencies. This paper thus presents the transmission line matrix method, a numerical method initially designed for electromagnetic waves, and its adaptation to acoustic waveguides. The method, and in particular the representation of boundary conditions, is validated by comparison with known analytical theories. It is then used to show the dramatic effect of higher order modes upon the radiation characteristics of uniform ducts, as well as the importance of source location. Finally, first applications to bent and bifurcating rectangular ducts are presented, and the transfer function of a vowel [a] is shown to display frequency patterns typical of those measured on human subjects and that cannot be explained by one-dimensional propagation only.

Vocal tract acoustics using the transmission line matrix (TLM) method

Most traditional theories of speech production are currently based on plane waves and on one-dimensional analysis. It is however well-known that when the frequency of sound reaches a cut-on frequency, higher acoustical modes start to propagate and can become predominant. It is therefore important to evaluate the effects of these higher modes, especially in order to improve acoustical models of the vocal tract. The paper describes a new numerical method to study the propagation and the radiation of speech sounds, and to compute acoustic characteristics of the vocal tract. This method, named transmission line matrix or modelling (TLM), has been used for simulating electromagnetic wave propagation and is used for the first time in acoustics. The TLM method provides time domain solutions in 2D and 3D spaces. The main advantage of this method is the simplicity of formulation and programming for a large range of applications. The authors first describe the principle on which the TLM method is based. The method as well as the boundary conditions used are validated using classical tests. A systematic study of higher order mode propagation and radiation is then presented. They focus on the influence of some critical parameters such as vocal tract width and location of the sound source. In particular, they show how, using TLM simulation, it is possible to derive modal reflection and transmission characteristics of the vocal tract. A typical example of simulation is presented and discussed.

Grounding stop place systems in the perceptuo-motor substance of speech: On the universality of the labial-coronal-velar stop series

Abstract Vowels are by far the best understood units in human sound systems, and are well characterized at the articulatory, acoustic, and perceptual levels. This has permitted explanations of vowel systems as structured by perception, and has led to effective substance-based theories. By contrast, stops are far less thoroughly understood. In this paper we use an articulatory–acoustic model of the vocal tract to examine stop consonant place in terms of both articulation and formant values. This allows us to locate each place of articulation in the F1–F2–F3 space, and to demonstrate in “articulatory nomograms” how formants evolve while closure is displaced from the front to the back of the vocal tract. Then, in the framework of the “Perception for Action Control Theory” that we have developed in recent years, we show that the near universal labial–coronal–velar stop series (i.e., /b d ɡ/ or /p t k/) is a perceptually optimal structure for stops just as /i a u/ is for vowels, provided that it is embedded in a suitable perceptuo-motor framework.

Recovery of vocal tract geometry fromformants for vowels and fricative consonants using a midsagittal-to-area function conversion model

This study deals with the ill-posed problem of inversion of the articulatory-to-acoustic relationship, i.e. the recovery of the vocal tract geometry from formant frequencies. A small database of articulatory-acoustic data has been established for one subject. A midsagittal-to-area function conversion model, which works both for vowel and fricative consonants, has been developed from these data. This model has finally been used as a major constraint for an optimization algorithm based on a gradient descente technique, in order to regularize the ill-posed inversion problem. Other spatial and temporal smooting constraints have been also used. Single vocal tract configurations, as well as entire [VC] sequences, could be recovered using adequate initial conditions