Christiane Neuschaefer-Rube

Towards a neurocomputational model of speech production and perception

The limitation in performance of current speech synthesis and speech recognition systems may result from the fact that these systems are not designed with respect to the human neural processes of speech production and perception. A neurocomputational model of speech production and perception is introduced which is organized with respect to human neural processes of speech production and perception. The production-perception model comprises an artificial computer-implemented vocal tract as a front-end module, which is capable of generating articulatory speech movements and acoustic speech signals. The structure of the production-perception model comprises motor and sensory processing pathways. Speech knowledge is collected during training stages which imitate early stages of speech acquisition. This knowledge is stored in artificial self-organizing maps. The current neurocomputational model is capable of producing and perceiving vowels, VC-, and CV-syllables (V=vowels and C=voiced plosives). Basic features of natural speech production and perception are predicted from this model in a straight forward way: Production of speech items is feedforward and feedback controlled and phoneme realizations vary within perceptually defined regions. Perception is less categorical in the case of vowels in comparison to consonants. Due to its human-like production-perception processing the model should be discussed as a basic module for more technical relevant approaches for high-quality speech synthesis and for high performance speech recognition.

Model-Based Reproduction of Articulatory Trajectories for Consonant–Vowel Sequences

We present a novel quantitative model for the generation of articulatory trajectories based on the concept of sequential target approximation. The model was applied for the detailed reproduction of movements in repeated consonant-vowel syllables measured by electromagnetic articulography (EMA). The trajectories for the constrictor (lower lip, tongue tip, or tongue dorsum) and the jaw were reproduced. Thereby, we tested the following hypotheses about invariant properties of articulatory commands: (1) The target of the primary articulator for a consonant is invariant with respect to phonetic context, stress, and speaking rate. (2) Vowel targets are invariant with respect to speaking rate and stress. (3) The onsets of articulatory commands for the jaw and the constrictor are synchronized. Our results in terms of high-quality matches between observed and model-generated trajectories support these hypotheses. The findings of this study can be applied to the development of control models for articulatory speech synthesis.

Sex matters: Neural correlates of voice gender perception

The basis for different neural activations in response to male and female voices as well as the question, whether men and women perceive male and female voices differently, has not been thoroughly investigated. Therefore, the aim of the present study was to examine the behavioral and neural correlates of gender-related voice perception in healthy male and female volunteers. fMRI data were collected while 39 participants (19 female) were asked to indicate the gender of 240 voice stimuli. These stimuli included recordings of 3-syllable nouns as well as the same recordings pitch-shifted in 2, 4 and 6 semitone steps in the direction of the other gender. Data analysis revealed a) equal voice discrimination sensitivity in men and women but better performance in the categorization of opposite-sex stimuli at least in men, b) increased responses to increasing gender ambiguity in the mid cingulate cortex and bilateral inferior frontal gyri, and c) stronger activation in a fronto-temporal neural network in response to voices of the opposite sex. Our results indicate a gender specific processing for male and female voices on a behavioral and neuronal level. We suggest that our results reflect higher sensitivity probably due to the evolutionary relevance of voice perception in mate selection.

More than Just Two Sexes: The Neural Correlates of Voice Gender Perception in Gender Dysphoria

Gender dysphoria (also known as “transsexualism”) is characterized as a discrepancy between anatomical sex and gender identity. Research points towards neurobiological influences. Due to the sexually dimorphic characteristics of the human voice, voice gender perception provides a biologically relevant function, e.g. in the context of mating selection. There is evidence for a better recognition of voices of the opposite sex and a differentiation of the sexes in its underlying functional cerebral correlates, namely the prefrontal and middle temporal areas. This fMRI study investigated the neural correlates of voice gender perception in 32 male-to-female gender dysphoric individuals (MtFs) compared to 20 non-gender dysphoric men and 19 non-gender dysphoric women. Participants indicated the sex of 240 voice stimuli modified in semitone steps in the direction to the other gender. Compared to men and women, MtFs showed differences in a neural network including the medial prefrontal gyrus, the insula, and the precuneus when responding to male vs. female voices. With increased voice morphing men recruited more prefrontal areas compared to women and MtFs, while MtFs revealed a pattern more similar to women. On a behavioral and neuronal level, our results support the feeling of MtFs reporting they cannot identify with their assigned sex.

MRT-Sequenzen als Datenbasis eines visuellen Artikulationsmodells

Articulatory models can be used in phoniatrics for the visualisation of speech disorders, and can thus be used in teaching, the counselling of patients and their relatives, and in speech therapy. The articulatory model developed here was based on static MRI data of sustained sounds. MRI sequences are now being used to further refine the model with respect to speech movements. Medio-sagittal MRI sections were recorded for 12 consonants in the symmetrical context of the three point vowels [i:], [a:] and [u:] for this corpus. The recording-rate was eight images/s. The data show a strong influence of the vocalic context on the articulatory target-positions of all consonants. A method for the reduction of the MRI data for subsequent qualitative and quantitative analyses is presented.

A method for measurement of the vocal tract impedance at the mouth

In this contribution a method is presented for the measurement of vocal tract resonances. The technique uses a non-invasive acoustic excitation of the vocal tract and a fast and robust detection. The method is an alternative to the linear predictive coding (LPC) analysis for patients with voice and speech disorders. Sweep signals are emitted and recorded simultaneously from the small end of a tube placed in front of the mouth opening. The use of a pressure sensor and a velocity sensor provides a direct measurement of the vocal tract impedance at the mouth (VTMI). For selected sustained German vowels, and some consonants, a comparison of results from LPC analysis and VTMI measurements is given. The results indicate a good agreement in the frequency range from 500 to 5,000 Hz. The feasibility of the VTMI method for diagnostic and therapeutic applications is subject to current research.

Towards an articulation-based developmental robotics approach for word processing in face-to-face communication

While we are capable of modeling the shape, e.g. face, arms, etc. of humanoid robots in a nearly natural or humanlike way, it is much more difficult to generate human-like facial or body movements and human-like behavior like e.g. speaking and co-speech gesturing. In this paper it will be argued for a developmental robotics approach for learning to speak. On the basis of current literature a blueprint of a brain model will be outlined for this kind of robots and preliminary scenarios for knowledge acquisition will be described. Furthermore it will be illustrated that natural speech acquisition mainly results from learning during face-to-face communication and it will be argued that learning to speak should be based on human-robot face-to-face communication. Here the human acts like a caretaker or teacher and the robot acts like a speech-acquiring toddler. This is a fruitful basic scenario not only for learning to speak, but also for learning to communicate in general, including to produce co-verbal manual gestures and to produce co-verbal facial expressions.

Sektorbezogene Grauwertanalyse videosonographisch aufgezeichneter Zungenbewegungen beim Schlucken

ZusammenfassungErkrankungen von Mundhöhle und Mundboden sowie Erkrankungen des Nervensystems können von Zungenbewegungsstörungen beim Schlucken begleitet werden. Derartige Störungen lassen sich videosonographisch diagnostizieren, wobei der Untersucher umfangreiches Bildmaterial zu beurteilen hat. Zielsetzung des vorliegenden Beitrags war es, diese Beurteilung durch Einsatz eines computergestützten, reproduzierbaren und quantitativen Auswertungsverfahrens zu erleichtern. Hierzu wurden Videosequenzen von 56 gesunden Erwachsenen und von 19 Patienten mit Schluckstörungen unterschiedlicher Genese analysiert. Für die numerische Analyse der Schluckbewegungen wurde, abstrahierend von den dargestellten Strukturen (Bolus, Luft, Zungen- und Mundbodenmuskulatur, Zungenoberflächenkontur, Hyoid u.a.), eine bildpunktbezogene Auswertung durchgeführt, indem die Graustufenintensitätsänderungen der Bildpunkte innerhalb zuvor definierter radiärer Bildausschnitte (Sektoren) im Zeitverlauf quantifiziert und in Kurvenform abgebildet wurden. Bei den gesunden Erwachsenen ließ sich im Regelfall ein charakteristisches Muster zweier Bewegungsmaxima erkennen. Diese Maxima waren in fast allen Sektorkurven vorhanden und entsprachen der Bolustransportbewegung der Zunge und ihrer Bewegung zur Ruhelage. Bei Patienten mit Erkrankungen der Zunge oder mit Bewegungsstörungen infolge von Erkrankungen des zentralen Nervensystems zeigten sich dagegen lokale oder generalisierte Bewegungsreduktionen, Bewegungsverlangsamungen, Mehrfachschluckbewegungen oder ungeordnete Zusatzbewegungen.SummaryDiseases of the oral cavity, floor of the mouth, and nervous system can be accompanied by disturbances in tongue movement during swallowing. These disturbances can be diagnosed by videosonography whereby the examiner has to evaluate extensive video documentation of lingual motion. It was the aim of this study to facilitate this evaluation by the application of a reproducible computer-assisted quantitative analysis procedure. Video sequences of 56 healthy adults and 19 patients with dysphagias of different etiologies were analysed. A numerical estimation of swallowing movements was carried out in abstraction from the structures imaged (bolus, air, muscles of the tongue, floor of the mouth, hyoid, etc.). Intensity changes of the pixels within previously defined radial image sectors were quantified in relationship to time and depicted as sector curves. The healthy adults demonstrated a characteristic pattern of two motion maxima that appeared within almost all sector curves. These maxima represented bolus transport movements and the reset movement of the tongue. Patients with diseases of the tongue or neuromuscular changes caused by disturbances of the central nervous system showed pathological deviations on videosonography. These appeared as local or general reductions in movement, slow speed motions, repetitive swallowing or unsorted additional movements of the tongue during swallowing.

Peripheral hearing disorders in childhood. Results of an evidenced based consensus conference

1. Definition der Schallleitungsschwerhorigkeit (SLS) Unter Schallleitungsschwerhorigkeiten werden Storungen der Schallubertragung im auseren Gehorgang und/oder im Mittelohr verstanden. Sie konnen passager auftreten, sind dann Ausdruck einer Tubenbeluftungsstorung bzw. einer Mittelohrentzundung und stellen die groste Gruppe der kindlichen konduktiven Schwerhorigkeiten dar. Die permanenten Schallleitungsschwerhorigkeiten sind im Kindesalter demgegenuber vergleichsweise selten und Folge angeborener oder erworbener Defekte der schallubertragenden Strukturen im Gehorgang bzw. Mittelohr. 2. Definition der Schallempfindungsschwerhorigkeit (SES) Unter Schallempfindungsschwerhorigkeiten werden Horstorungen verstanden, die durch Innenohr- oder Hornervenschadigungen bedingt sind. Schallempfindungsschwerhorigkeiten sind mit Ausnahme der im Kindesalter auserst seltenen Horsturze permanente Horstorungen, die zu einem geringen Prozentsatz progredient sind.

The contribution of phonation type to the perception of vocal emotions in German: an articulatory synthesis study.

Vocal emotions are signaled by specific patterns of prosodic parameters, most notably pitch, phone duration, intensity, and phonation type. Phonation type was so far the least accessible parameter in emotion research, because it was difficult to extract from speech signals and difficult to manipulate in natural or synthetic speech. The present study built on recent advances in articulatory speech synthesis to exclusively control phonation type in re-synthesized German sentences spoken with seven different emotions. The goal was to find out to what extent the sole change of phonation type affects the perception of these emotions. Therefore, portrayed emotional utterances were re-synthesized with their original phonation type, as well as with each purely breathy, modal, and pressed phonation, and then rated by listeners with respect to the perceived emotions. Highly significant effects of phonation type on the recognition rates of the original emotions were found, except for disgust. While fear, anger, and the neutral emotion require specific phonation types for correct perception, sadness, happiness, boredom, and disgust primarily rely on other prosodic parameters. These results can help to improve the expression of emotions in synthesized speech and facilitate the robust automatic recognition of vocal emotions.