Noël Nguyen

The dynamical approach to speech perception: From fine phonetic detail to abstract phonological categories

phonological units, in a way that leads variations in how a given word is produced to be factored out at an early stage of processing (e.g., Fitzpatrick and Wheeldon, 2000; Lahiri and Reetz, 2002; Stevens, 2002). It is often hypothesized that inter-individual variations are removed prior to building up this abstract representation by means of a speaker normalization procedure (see Johnson, 2005b, for a recent review). A clear demarcation is established in that framework between the surface phonetic form of a word and the underlying phonological representation associated with that word. The abstractionist approach contends that the phonological representation for each word is both unique and permanently stored in memory. Readers are referred to Cutler et al. (2007), Eulitz and Lahiri (2004), Lahiri and Reetz (2002), Pallier et al. (2001), and Stevens (2002), for recent important papers in the abstractionist framework. In the terminology employed by Pitt and Johnson (2003), abstractionist models such as Lahiri and Reetz’s (2002) FUL model offer a representationbased solution to the speech variability problem. According to the FUL (Featurally Underspecified Lexicon) model, phonological representations in the lexicon are underspecified for certain features such as [coronal], and it is assumed that this makes listeners insensitive to surface variations shown by words for these features. For example, assimilation of word-final coronals to the place of articulation of the following consonant is considered to remain consistent with the underspecified phonological representation of the carrier word and, because of this so-called no-mismatch relationship between the word’s surface and underlying forms, assimilation is expected to have no disruptive effect on the identification of the word. As opposed to representationbased models, processing-based models rely on the assumption that recognizing the assimilated variant of a word entails recovering the word’s unassimilated shape through a phonological inference process. In spite of the fact that they too are representation-oriented, exemplar models stand, in their most prototypical form at least, in sharp opposition to abstractionist models like FUL along many dimensions. A major difference with the abstractionist stance relates to the fact that each exemplar is viewed as corresponding to a language chunk that is stored in memory with all the details specific to the particular circumstances in which it has been produced or encountered. This includes sensory-motor, semantic and pragmatic characteristics, but also indexical information about the speaker’s identity and the situation of occurrence, to mention but a few properties. Exemplars are therefore deeply anchored into their context of occurrence in the largest possible Roles of fine phonetic detail and abstract phonological categories in speech

Communicative Behavior and Physiology in Social Interactions

We introduce here a new experimental set-up that provides temporally aligned behavioral (including linguistic) together with physiological activity time-series recorded during social interactions. It brings the experimental approach closer to ecological social interactions while preserving important experimental features for transposing the set-up to other physiological and neurophysiological measures. The set-up is implemented as a full-size pilot study that contrasts two experimental factors: the nature of the agent the research participant interacts with (a human vs. a virtual agent) and the nature of the interaction (live vs. video). The analysis presented here focuses on investigating co-occurrences between electrodermal activity on the one hand and behaviors, speech and gaze in particular, on the other hand. We also investigated into the linguistic productions in order to evaluate the degree of communicative interaction as a function of the experimental conditions. We are reporting our first results on these different aspects, and argue that they pave the way for promising studies exploiting this set-up.

Effet de l'accélération artificielle du signal de parole sur la perception des dysarthries cérébelleuses: à propos de deux cas

Objectif: Dans les dysarthries cérébelleuses, la perte de la précision des mouvements articulatoires de la parole induit une dysrégulation temporelle avec un allongement irrégulier des durées contribuant globalement au ralentissement du débit de la parole. Ainsi, nous supposons que les distorsions du signal de parole sont largement influencées par le ralentissement et que la correction de ce ralentissement, par accélération du signal de parole, peut améliorer l’intelligibilité. Patientes et méthode: Deux patientes avec une dysarthrie ataxique ont été étudiées. L’intelligibilité de la parole a été évaluée subjectivement avec une échelle visuelle analogique et objectivement par la méthode des stimuli constants associée à une tâche d’identification d’un phonème cible en temps réel. L’accélération temporelle des stimuli est effectuée par l’algorithme SOLA. Un total de 144 stimuli a été créé à 3 vitesses différentes: 48 phrases à «vitesse naturelle» (sans accélération, VN), 48 phrases à «vitesse intermédiaire» (50% de l’accélération maximale, VI) et 48 phrases à «vitesse maximale» (vitesse moyenne des locuteurs témoins, VM). Les 144 phrases, dans les 3 présentations expérimentales, ont été réparties en 3 listes, chaque auditeur entendant une seule fois chaque phrase. Chaque liste est écoutée par 8 auditeurs. Les phrases sont présentées dans un ordre randomisé. Pour estimer l’effet de l’accélération, nous avons utilisé un modèle logit mixte pour le score de l’épreuve subjectif et les temps de réaction, un test de χ2 pour le nombre des erreurs. Résultats: Pour les 2 locutrices, la probabilité d’une meilleure intelligibilité à l’épreuve subjective est significativement plus grande pour la condition VI ou VM que pour la condition VN. Le passage de la condition VN à la condition VI augmente le taux de satisfaction des auditeurs quant à l’intelligibilité de 78 à 87% pour Mme B et de 46 à 63% pour Mme Z. Les erreurs sont moins fréquentes avec la compression pour Mme Z. L’effet de l’accélération n’est jamais significatif sur les temps de réaction (F2, 1024 = 2,14, p = 0,12). Conclusions: Cette étude est la première à analyser l’effet d’une accélération temporelle uniforme sur de la parole pathologique. Elle montre que chez 2 patientes présentant une dysarthrie ataxique, le niveau perçu de l’intelligibilité est amélioré.

Analyzing vocal tract movements during speech accommodation

When two people engage in verbal interaction, they tend to accommodate on a variety of linguistic levels. Although recent attention has focused on to the acoustic characteristics of convergence in speech, the underlying articulatory mechanisms remain to be explored. Using 3D electromagnetic articulography (EMA), we simultaneously recorded articulatory movements in two speakers engaged in an interactive verbal game, the domino task. In this task, the two speakers take turn in chaining bi-syllabic words according to a rhyming rule. By using a robust speaker identification strategy, we identified for which specific words speakers converged or diverged. Then, we explored the different vocal tract features characterizing speech accommodation. Our results suggest that tongue movements tend to slow down during convergence whereas maximal jaw opening during convergence and divergence differs depending on syllable position.

ISIAA 2017: 1st international workshop on investigating social interactions with artificial agents (workshop summary)

The workshop “Investigating Social Interactions With Artificial Agents” organized within the “International Conference on Multimodal Interactions 2017” attempts to bring together researchers from different fields sharing a similar interest in human interactions with other agents. If interdisciplinarity is necessary to address the question of the “Turing Test”, namely “can an artificial conversational artificial agent be perceived as human”, it is also a very promising new way to investigate social interactions in the first place. Biology is represented by social cognitive neuroscience, aiming to describe the physiology of human social behaviors. Linguistics, from humanities, attempts to characterize a specifically human behavior and language. Social Signal Processing is a recent approach to analyze automatically, using advanced Information Technologies, the behaviors pertaining to natural human interactions. Finally, from Artificial Intelligence, the development of artificial agents, conversational and/or embodied, onscreen or physically attempts to recreate non-human socially interactive agents for a multitude of applications.

Studying the link between inter-speaker coordination and speech imitation through human-machine interactions

According to accounts of inter-speaker coordination based on internal predictive models, speakers tend to imitate each other each time they need to coordinate their behavior. According to accounts based on the notion of dynamical coupling, imitation should be observed only if it helps stabilizing the specific coordinative pattern produced by the interlocutors or if it is a direct consequence of inter-speaker coordination. To compare these accounts, we implemented an artificial agent designed to repeat a speech utterance while coordinating its behavior with that of a human speaker performing the same task. We asked 10 Italian speakers to repeat the utterance /topkop/ simultaneously with the agent during short time intervals. In some interactions, the agent was parameterized to cooperate with the speakers (by producing its syllables simultaneously with those of the human) while in others it was parameterized to compete with them (by producing its syllables in-between those of the human). A positive correlation between the stability of inter-speaker coordination and the degree of f0 imitation was observed only in cooperative interactions. However, in line with accounts based on prediction, speakers imitate the f0 of the agent regardless of whether this is parameterized to cooperate or to compete with them.