Laurent Prévot

Communicative Behavior and Physiology in Social Interactions

We introduce here a new experimental set-up that provides temporally aligned behavioral (including linguistic) together with physiological activity time-series recorded during social interactions. It brings the experimental approach closer to ecological social interactions while preserving important experimental features for transposing the set-up to other physiological and neurophysiological measures. The set-up is implemented as a full-size pilot study that contrasts two experimental factors: the nature of the agent the research participant interacts with (a human vs. a virtual agent) and the nature of the interaction (live vs. video). The analysis presented here focuses on investigating co-occurrences between electrodermal activity on the one hand and behaviors, speech and gaze in particular, on the other hand. We also investigated into the linguistic productions in order to evaluate the degree of communicative interaction as a function of the experimental conditions. We are reporting our first results on these different aspects, and argue that they pave the way for promising studies exploiting this set-up.

Processing Units in Conversation: A Comparative Study of French and Mandarin Data*:

Human spoken language production is directed towards communication delivering comprehensible information to recipients. Speech segmentation into small units efficiently enhances a sensible and interpretable discourse structure. Such processing units in real-life communication may be applied to semantic, syntactic, or prosodic structures. Previous studies have proposed various theories of speech segmentation, mainly based on qualitative analyses. The present study utilizes corpus-based quantitative data to examine how conversational speech in French and Mandarin is structured in terms of three different processing units, and how these units interact with one another. Unit completion location was identified by semantic structure (discourse unit), prosodic pattern (prosodic unit), and sequences of parts of speech (chunk). Quantitative analyses for both languages were carried out by applying comparable processing procedures. This article presents our efforts to establish a dataset for two typologically divers...

Investigating the dimensions of conversational agents' social competence using objective neurophysiological measurements

Assessing the social competence of anthropomorphic artificial agents developed to produce engaging social interactions with humans has become of primary importance to effectively compare various appearances and/or behaviours. Here we attempt to objectify the social competence of artificial agents, across different dimensions, using human brain neurophysiology. Whole brain activity is recorded with functional Magnetic Resonance Imaging (fMRI) while participants, naïve to the real purpose of the experiment, discuss either with a human confederate or with an artificial agent, presently the robotic conversational head Furhat controlled with a Wizard of Oz procedure. This allows a direct comparison of local brain responses, not only at the cortical level but also in subcortical structures associated with motivational drive and impossible to investigate with non-invasive neurophysiology techniques such as surface recordings. The present data (n = 21 participants) demonstrates the feasibility of this approach, and results confirm an increased activity in subcortical structures, in particular the amygdala involved in emotional processing and the hypothalamus, known to secrete, among others, the neurohormone oxytocin involved in social bonding, as well as the temporoparietal junction bilaterally involved in the attribution of mental states, when interacting with a human compared to an artificial agent. The reverse contrast revealed more dorsal cortical areas. Altogether, these results support the use of fMRI to objectify the social competence of artificial agents along distinct dimensions.

Studying the link between inter-speaker coordination and speech imitation through human-machine interactions

According to accounts of inter-speaker coordination based on internal predictive models, speakers tend to imitate each other each time they need to coordinate their behavior. According to accounts based on the notion of dynamical coupling, imitation should be observed only if it helps stabilizing the specific coordinative pattern produced by the interlocutors or if it is a direct consequence of inter-speaker coordination. To compare these accounts, we implemented an artificial agent designed to repeat a speech utterance while coordinating its behavior with that of a human speaker performing the same task. We asked 10 Italian speakers to repeat the utterance /topkop/ simultaneously with the agent during short time intervals. In some interactions, the agent was parameterized to cooperate with the speakers (by producing its syllables simultaneously with those of the human) while in others it was parameterized to compete with them (by producing its syllables in-between those of the human). A positive correlation between the stability of inter-speaker coordination and the degree of f0 imitation was observed only in cooperative interactions. However, in line with accounts based on prediction, speakers imitate the f0 of the agent regardless of whether this is parameterized to cooperate or to compete with them.

Activation of writing-specific brain regions when reading Chinese as a second language. Effects of training modality and transfer to novel characters

&NA; We examined the implication of training modality on the cortical representation of Chinese words in adult second language learners of Chinese. In particular, we tested the implication of the neural substrates of writing in a reading task. The brain network sustaining finger writing was defined neuroanatomically based on an independent functional localizer. We examined the brain activations elicited by Chinese words learned via writing vs. pronunciation, and by novel untrained words, within regions of interest (ROIs) defined according to the position of the activation peaks in the localizer, and at the whole brain level. We revealed activations in the reading task that overlapped with several parts of the finger writing network. In addition, our results provide evidence that the neural substrates of writing are differentially involved in reading depending on the stored knowledge for words, as revealed by the fine‐grained response of several regions including the left superior parietal lobule and left precentral gyrus / superior frontal sulcus to the experimental manipulations. Training modality and the linguistic properties of the characters also impacted the response of the left mid‐fusiform gyrus, confirming its involvement as the brain region where linguistic, visual and sensorimotor information converge during orthographic processing. At the behavioral level, global handwriting quality during the training sessions was positively correlated to the final translation performance. Our results demonstrate substantial overlap in the neural substrates of reading and writing, and indicate that some regions sustaining handwriting are differentially involved in reading depending on the type of knowledge associated with words. HighlightsTraining modality effects on brain representation of Chinese words in French learners.Overlap between neural substrates of reading and motor processing involved in writing.The writing network is differentially involved in reading dependent upon stored knowledge.Kinematic handwriting quality positively correlates to the stored knowledge of words.Stored knowledge of words impacts the response of the left mid‐fusiform gyrus.