Dominique Boutet

A Child’s Multimodal Negations from 1 to 4: The Interplay Between Modalities

Previous research on first language acquisition has highlighted the relation between actions, gestures and speech to express negation. As discussed by Watson (1925) and Clark (1978), children’s first negative constructions seem to take over from early gestures of rejection and avoidance. This study explores the status and evolution of actions, gestures and words expressing negation. Our detailed multimodal analyses are conducted on the longitudinal data of an English monolingual girl interacting with her mother recorded monthly at home for one hour between 10 months and 4 years old. We categorised all the child’s actions, gestures and words used in isolation or in combination according to their form and function. We distinguished five periods between 10 months and 4 years old. (1) Mostly actions are used at first; (2) more conventional gestures are used alone; (3) gestures are combined with one or two words; (4) the child’s speech becomes complex and she uses fewer gestures; (5) she reintroduces gestures but uses them as co-speech gestures along with longer, more complex negative spoken utterances. Her multimodal negative communication follows a specific pathway that goes from embodied negation to symbolic negation. After four, the use of gestures becomes fully integrated into multimodal complex productions with a rich diversification of functions and forms.

Characterisation of gestural units in light of human-avatar interaction

We present a method for characterizing coverbal gestural units intended for human-avatar interaction. We recorded 12 gesture types, using a motion-capture system. We used the markers positions thus obtained to determine the gestural units after stroke segmentation. We complement our linguistic analysis of gestures with an elaboration of our biomechanical hypotheses, our method of segmentation, our characterization hypotheses and the results obtained.

Photocaligraphy: Writing Sign Language

The de facto language of deaf people is sign language, a gesture based communication process. Being quite different from oral languages (grammar, modality, syntax), it needs a writing system of its own. Despite a few attempts, no clear writing system for sign language has emerged. The work we present in this chapter constitutes a contribution to its formation through a graphic design approach. Our hypothesis is as follows: in its execution, the gestural signs contain readable graphic traces. In order to visualise them, we use a photographic system based on long exposure, creating graphic objects we name photocalligraphies. We experimented with deaf people and created two corpora made up of isolated signs. With the first one we study the legibility of such a representation of a sign: how well it is recognised, how well its meaning is conveyed. With the second we deepen the study of something we observed during the realisation of the first corpus: during the photographic capture of the signs, the sign language speaker makes alterations to the prototypic sign, signing it differently in order to make its graphic rendering more readable. We then discuss potential structures for those alterations that we call graphic inscribing strategies.