Judith Guez

A virtual reality agent-based platform for improvisation between real and virtual actors using gestures

This paper describes the process of designing a Virtual Reality Agent-based platform to explore an interactive gestural dialogue between real and virtual actors and propose an analysis of the results. It is part of the CIGALE interdisciplinary project involving researchers in the fields of linguistics, digital arts and computer science, as well as actors and professors. Several experiments of improvisation conducted with actors through computer-mediated interactions help in characterizing specific dynamics of gestures, recurrent behaviors and gestural patterns. We extract specific descriptors of movement and behaviors that will drive the improvisation. The virtual actor is able to improvise and generate its gestures using a finite state machine that triggers three main behaviors according to the actors body posture and arm movements: the virtual actor can mime in real-time the real actor, it can propose expressive or linguistic gestures from four MoCap databases, or it can generate a new gesture using a genetic algorithm. A first installation experienced by the participants during an international art festival, shows they are able to improvise fluently with the virtual actor in an expressive dialogue.

InterACTE: Improvising with a Virtual Actor

This paper describes an interactive installation that invites participants to improvise with an autonomous virtual actor. Based on real-time analysis of the participants gestures, the virtual actor is capable of either propose expressive and linguistic gestures (selected from a set of four Motion Capture Databases), either imitate the participant, or either generate its own gestures using a Genetic Algorithm. A software agent is used to trigger the previously mentioned behaviors based on the analysis results and on a set of predefined rules. The installation is presented in two acts: i. In the first act, the participant interacts with the projected shadow of the virtual actor; ii. In the second act, the participant is being immersed in a 3D world using a virtual reality HMD---the virtual actor is being presented to her/him in the form of an avatar made of particles.