Clément Moulin-Frier

Adverse conditions improve distinguishability of auditory, motor and perceptuo-motor theories of speech perception: an exploratory Bayesian modeling study

In this paper, we put forward a computational framework for the comparison between motor, auditory, and perceptuo-motor theories of speech communication. We first recall the basic arguments of these three sets of theories, either applied to speech perception or to speech production. Then we expose a unifying Bayesian model able to express each theory in a probabilistic way. Focusing on speech perception, we demonstrate that under two hypotheses, regarding communication noise and inter-speaker variability, providing perfect conditions for speech communication, motor, and auditory theories are indistinguishable. We then degrade successively each hypothesis to study the distinguishability of the different theories in “adverse” conditions. We first present simulations on a simplified implementation of the model with mono-dimensional sensory and motor variables, and secondly we consider a simulation of the human vocal tract providing more realistic auditory and articulatory variables. Simulation results allow us to emphasise the respective roles of motor and auditory knowledge in various conditions of speech perception in adverse conditions, and to suggest some guidelines for future studies aiming at assessing the role of motor knowledge in speech perception.

Recognizing speech in a novel accent: the motor theory of speech perception reframed

The motor theory of speech perception holds that we perceive the speech of another in terms of a motor representation of that speech. However, when we have learned to recognize a foreign accent, it seems plausible that recognition of a word rarely involves reconstruction of the speech gestures of the speaker rather than the listener. To better assess the motor theory and this observation, we proceed in three stages. Part 1 places the motor theory of speech perception in a larger framework based on our earlier models of the adaptive formation of mirror neurons for grasping, and for viewing extensions of that mirror system as part of a larger system for neuro-linguistic processing, augmented by the present consideration of recognizing speech in a novel accent. Part 2 then offers a novel computational model of how a listener comes to understand the speech of someone speaking the listener’s native language with a foreign accent. The core tenet of the model is that the listener uses hypotheses about the word the speaker is currently uttering to update probabilities linking the sound produced by the speaker to phonemes in the native language repertoire of the listener. This, on average, improves the recognition of later words. This model is neutral regarding the nature of the representations it uses (motor vs. auditory). It serve as a reference point for the discussion in Part 3, which proposes a dual-stream neuro-linguistic architecture to revisits claims for and against the motor theory of speech perception and the relevance of mirror neurons, and extracts some implications for the reframing of the motor theory.

On the cognitive nature of speech sound systems

During the last 50 years, the question of the cognitive nature of phonological units has followed the rhythm of the persistent debate between auditory and motor theories of speech communication. Though recent advances in cognitive neuroscience and cognitive psychology have largely renewed this debate, a consensus is still out of reach, and the true nature of speech units in the human brain remains elusive.A dimension of importance in this debate is a systemic one: speech units are not isolated, they are part of a phonological system, and they obey structural principles regarding well-investigated properties as distinctiveness, compositionality, contextual dependencies or systemic regularities. The phonological system itself is also part of a complex network of interaction with low-level biomechanical and sensory-motor systems, with higher-level brain structures regulating cognition, emotion and motivation, and finally with the social structures in which all these systems are embedded.Connecting assumptions or theories about the nature of speech units with a structuralist view about the relationship between phonetic properties and phonological systems has given rise to a number of major breakthroughs in speech science, for instance Lindblom’s bridges between the Variable Adaptive Theory (or its Hyper-Hypo variant) of speech communication (Lindblom, 1990) and the Dispersion Theory of vowel systems (Lindblom, 1986); or Stevens’ Quantal Theory (Stevens, 1972, 1989) addressing both the invariance issue and the search for the origins of distinctiveness and phonetic features; or the tandem between the Motor Theory of Speech Perception (Liberman & Mattingly, 1985) and Articulatory Phonology (Browman & Goldstein, 1992) in the Haskins Labs. This Special Issue is centered around a target paper by Moulin-Frier et al. that aims at relating the question of the auditory vs. motor vs. perceptuo-motor nature of speech units with simulations of vowel, plosive and syllable systems of human languages emerging from agent interactions, in a computational Bayesian framework. In this context, the papers in the special issue explore further the systemic perspective, studying how various dimensions of physical, cognitive, motivational and interactional systems can inform our understanding of the origins of speech forms.