Pierre De Loor

Semantic modeling of Virtual Environments using MASCARET

Many Virtual Reality (VR) applications, such as Virtual Learning Environments or Interactive Virtual Tours, are based on a rich semantic description of the environment and tasks that users have to perform. These applications are built upon Virtual Environments (VEs) in which artificial agents act autonomously while interacting in realtime with users. Semantic modelling of a VR environment makes it possible the knowledge-driven access from the description of VEs that simplifies the development of VR applications. It eases the development of these types of applications. Semantic modelling should provide a consistent representation of the following aspects: 1) The simulated world, its structure and the behavior of its entities, 2) Interactions and tasks, that users and agents can perform in the environment, 3) Knowledge items, that autonomous agents can use for decision-making or for communication with users. This paper presents MASCARET, a model-based approach, for the design of semantic VR environments. This approach is based on the Unified Modeling Language (UML). In this approach, UML is used to provide a knowledge-driven access to the semantic contents of the VE and not for code generation, as in classical software development process. Interests of a UML-based approach are that its metamodel covers different views of the semantic modelling: ontology, structure, behaviors, interactions, activities. It is also an extensible language that can be specialized to provide formal operational semantics. We also present how MASCARET can be used to develop content-rich interactive applications that can be deployed over various VR platforms. Finally, we discuss the benefits of such a metamodel-based approach and show how the multi-layer semantic model can be used in different VR applications, in which adaptive behaviors of artificial agents acting within complex environments have to be simulated.

MASCARET: A Pedagogical Multi-Agent System for Virtual Environments for Training

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Mascaret: Pedagogical multi-agents system for virtual environment for training. Cédric Buche, Ronan Querrec, Pierre De Loor, Pierre Chevaillier

Enaction-Based Artificial Intelligence: Toward Co-evolution with Humans in the Loop

This article deals with the links between the enaction paradigm and artificial intelligence. Enaction is considered a metaphor for artificial intelligence, as a number of the notions which it deals with are deemed incompatible with the phenomenal field of the virtual. After explaining this stance, we shall review previous works regarding this issue in terms of artificial life and robotics. We shall focus on the lack of recognition of co-evolution at the heart of these approaches. We propose to explicitly integrate the evolution of the environment into our approach in order to refine the ontogenesis of the artificial system, and to compare it with the enaction paradigm. The growing complexity of the ontogenetic mechanisms to be activated can therefore be compensated by an interactive guidance system emanating from the environment. This proposition does not however, resolve that of the relevance of the meaning created by the machine (sense-making). Such reflections lead us to integrate human interaction into this environment in order to construct relevant meaning in terms of participative artificial intelligence. This raises a number of questions with regards to setting up an enactive interaction. The article concludes by exploring a number of issues, thereby enabling us to associate current approaches with the principles of morphogenesis, guidance, the phenomenology of interactions and the use of minimal enactive interfaces in setting up experiments which will deal with the problem of artificial intelligence in a variety of enaction-based ways.Version Francaise de: Enaction Based Arti cial Intelligence: What about Human in the Loop, paru dans Minds and Machines 19 (2009) 319-343.

Real-time gesture recognition based on motion quality analysis

This paper presents a robust and anticipative realtime gesture recognition and its motion quality analysis module. By utilizing a motion capture device, the system recognizes gestures performed by a human, where the recognition process is based on skeleton analysis and motion features computation. Gestures are collected from a single person. Skeleton joints are used to compute features which are stored in a reference database, and Principal Component Analysis (PCA) is computed to select the most important features, useful in discriminating gestures. During real-time recognition, using distance measures, real-time selected features are compared to the reference database to find the most similar gesture. Our evaluation results show that: i) recognition delay is similar to human recognition delay, ii) our module can recognize several gestures performed by different people and is morphology-independent, and iii) recognition rate is high: all gestures are recognized during gesture stroke. Results also show performance limits.

Ensuring semantic spatial constraints in virtual environments using UML/OCL

Spatial objects and relationships between them, compose a spatial model that is the backbone of virtual environments (VEs). However, due to the natural complexity of both spatial objects and spatial information, the modeling of such spatial relationships is still a difficult task. This paper presents a novel approach for representing semantic spatial relations in VEs using the Unified Modeling Language (UML) and the Object Constraint Language (OCL). Our approach first uses the UML class model as a conceptual model for VEs. We then propose a spatial extension of OCL named VRX-OCL as a high-level and flexible language to cover multidimensional, manifold, and reference frame-dependent spatial constraints. We mainly focus on two important classes of spatial relations, namely, topological and projective relations that allow nonmetric representation of space. The applicability of our approach is demonstrated in the Virtual Physics Laboratory, a VE for learning physics. Based on the constraints satisfaction, the system is able to visualize abstract spatial information and thus provides educational assistance to the learners.

Effects of Coupling in Human-Virtual Agent Body Interaction

This paper presents a study of the dynamic coupling between a user and a virtual character during body interaction. Coupling is directly linked with other dimensions, such as co-presence, engagement, and believability, and was measured in an experiment that allowed users to describe their subjective feelings about those dimensions of interest. The experiment was based on a theatrical game involving the imitation of slow upper-body movements and the proposal of new movements by the user and virtual agent. The agent’s behaviour varied in autonomy: the agent could limit itself to imitating the user’s movements only, initiate new movements, or combine both behaviours. After the game, each participant completed a questionnaire regarding their engagement in the interaction, their subjective feeling about the co-presence of the agent, etc. Based on four main dimensions of interest, we tested several hypotheses against our experimental results, which are discussed here.

The INGREDIBLE Database: A First Step Toward Dynamic Coupling in Human-Virtual Agent Body Interaction

This paper presents the ANR Ingredible Project and the collection of a database of dyadic bodily expressive interactions.While there continues to be exciting developments in research related to virtual characters, improvements are still needed to create plausibly human-like behaviors. In this paper, we present a synthetic parameterized memory model which includes sensory, working, and long-term memories and mechanisms for acquiring and retrieving memories. With the aid of this model, autonomous virtual humans are able to perform more reasonable interactions with objects and agents in their environment. The memory model also facilitates emergent behaviors, enhances behavioral animation, and assists in creating heterogeneous populations. To demonstrate the effectiveness of the memory model, we also provide an example in a 3D game environment and have conducted a user study in which we found general guidance in determining parameter values for the memory model, resulting in NPCs with more human-like game playing performances.

INGREDIBLE: A platform for full body interaction between human and virtual agent that improves co-presence

This paper presents a platform dedicated to a full body interaction between a virtual agent and human or between two virtual agents. It is based on the notion of coupling and the metaphor of the alive communication that come from studies in psychology. The platform, based on a modular architecture, is composed of modules that communicate through messages. Four modules have been implemented for human tracking, motion analysis, decision computation and rendering. The paper describes all of them. Part of the decision module is generic, that is it could be used for different interactions based on sensorimotor, while part of it is strictly dependent on the type of scenario one wants to obtain. An application example for a fitness exergame scenario is also presented in this work.

Anticipatory behavior in virtual universe, application to a virtual juggler

To be believable, virtual entities must be equipped with the ability to anticipate, that is, to predict the behavior of other entities and the subsequent consequences on the environment. For that purpose, we propose an original approach where the entity possesses an autonomous world of simulation within simulation, in which it can simulate itself (with its own model of behavior) and simulate the environment (with the representation of the behaviors of the other entities). This principle is illustrated by the development of an artificial juggler in 3D. In this application, the juggler predicts the motion of the balls in the air and uses its predictions to coordinate its own behavior to continue to juggle.Copyright

Aliveness metaphor for an evolutive gesture interaction based on coupling between a human and a virtual agent

This paper presents a model that provides adaptive and evolutive interaction between a human and a virtual agent. After introducing the theoretical justifications, the aliveness metaphor and the notion of coupling are presented. Then, we propose a formalization of the model that relies on the temporal evolution of the coupling between participants and the existence of phases during the interaction. An example on a fitness exergame is provided and some illustrations show the behavior of the model during an interaction. A video complements this example.