Marc Cavazza

Motion control of virtual humans

The article surveys virtual humans and techniques to control the face and body. It also covers higher level interfaces for direct speech input and issues of real-time control.

High-level interpretation in virtual environments

As virtual environments applications become more complex, there is a need to interpret user interaction in terms of high-level concepts. In this paper, the relations between conceptual representations of actions and their physical simulation in virtual worlds is investigated. A model inspired fromnatural language processing (NLP) research in the linguistic interpretation of dynamic scenes has been developed. The experiments are based on real-time animation software, which has been enhanced with a symbolic information processing layer, originally developed for NLP-based animation. We report the implementation of a high-level interpretation module that is able to recognize complex actions fromlow-level physical events in the virtual world and discuss its performance as well as directions for further developments.

Natural language control of interactive 3D animation and computer games

In this paper we describe a fully implemented system for speech and natural language control of 3D animation and computer games. The experimental framework has features that have been emulated from the popular DOOM™ computer game. It implements an integrated parser based on a linguistic formalism tailored to the processing of the specific natural language instructions required to control a player character. This parser outputs structured message formats to the animation layer, which further interprets these messages to generate behaviours for the scene objects. We have found that interactive control significantly impacts on the behavioural interpretation of natural language semantics. Besides bringing stringent requirements in terms of response times for the natural language processing step, it determines the level of autonomy that the animated character should possess, which in turn influences the generation of behavioral scripts from natural language instructions.

A constrained inverse kinematics technique for real-time motion capture animation

In this paper we present a constrained inverse kinematics algorithm for real-time motion capture in virtual environments, that has its origins in the simulation of multi-body systems. We apply this algorithm to an articulated human skeletal model using an electromagnetic motion tracking system with a small number of sensors to create avatar postures. The method offers efficient inverse kinematics computation and it is also generalised for the configurations of an articulated skeletal model. We investigate the possibility of capturing fast gestures by analysing the convergence patterns of the algorithm with the motion tracking sampling frequency for a range of actions.

Multimedia semantic features and image content description

Content based indexing and retrieval of images and video requires a proper semantic description for image content. The paper discusses the mapping of high level, application specific features to the visual primitives that are accessible through image processing techniques. A major difficulty is that there are currently no established methodologies for describing the contents of an image in terms of semantic features, and we suggest that semiotic approaches could be adapted to the task of image description in limited areas of expertise. After reviewing current trends in the mapping of high level to low level features, we present preliminary results that suggest a possible strategy for mapping semiotic descriptions to image processing primitives.

Real-Time Inverse Kinematics through Constrained Dynamics

Motion capture is an essential technique for interactive systems, such as distributed virtual reality, video games, and entertainment. Inverse kinematics algorithms are often used to minimize the number of trackers required for motion capture systems. The solving of inverse kinematics problems can be computationally expensive. We introduce a real-time algorithm for inverse kinematics computation, originally from the field of molecular simulation, called SHAKE. We also describe the implementation of the algorithm in our motion capture system for avatar motion generation through constraint dynamics. We demonstrate that the algorithm has advantages over conventional methods with properties of fast convergence, energy stability, and constraint system consistency when adding additional constraints.

The Virtual Interactive Presenter: A Conversational Interface for Interactive Television

With the advent of multi-channel digital TV, accessing Electronic Programme Guides (EPGs) will become increasingly complex. This paper describes ongoing work in the development of a conversational interface for Interactive TV. We claim that a conversational interface can support incremental refinement of user selections thus assisting user choice without requiring knowledge of editorial categories. We describe the system architecture, which integrates various components dedicated to language processing, EPG access and animation of the conversational character. Finally, we discuss the control strategy for the whole system from the perspective of human-computer dialogue.

Real-Time Requirements for the Implementation of Speech-Controlled Artificial Actors

Avatars and Artificial Actors in Virtual Environments can be controlled by speech, as an alternative to motion capture techniques. In this paper, we discuss some specific requirements for the successful implementation of speech-based control of guided actors. We describe our sublanguage approach to speech-based control and its associated parsing techniques, based on lexicalised grammars. After an introduction to the REALISM animation software, we report work in progress in the real-time processing of spoken commands based on the integration of speech processing in the REALISM control loop. We conclude by discussing the possible impact of voice-controlled artificial actors in interactive systems.