Pierre Chevaillier

Semantic modeling of Virtual Environments using MASCARET

Many Virtual Reality (VR) applications, such as Virtual Learning Environments or Interactive Virtual Tours, are based on a rich semantic description of the environment and tasks that users have to perform. These applications are built upon Virtual Environments (VEs) in which artificial agents act autonomously while interacting in realtime with users. Semantic modelling of a VR environment makes it possible the knowledge-driven access from the description of VEs that simplifies the development of VR applications. It eases the development of these types of applications. Semantic modelling should provide a consistent representation of the following aspects: 1) The simulated world, its structure and the behavior of its entities, 2) Interactions and tasks, that users and agents can perform in the environment, 3) Knowledge items, that autonomous agents can use for decision-making or for communication with users. This paper presents MASCARET, a model-based approach, for the design of semantic VR environments. This approach is based on the Unified Modeling Language (UML). In this approach, UML is used to provide a knowledge-driven access to the semantic contents of the VE and not for code generation, as in classical software development process. Interests of a UML-based approach are that its metamodel covers different views of the semantic modelling: ontology, structure, behaviors, interactions, activities. It is also an extensible language that can be specialized to provide formal operational semantics. We also present how MASCARET can be used to develop content-rich interactive applications that can be deployed over various VR platforms. Finally, we discuss the benefits of such a metamodel-based approach and show how the multi-layer semantic model can be used in different VR applications, in which adaptive behaviors of artificial agents acting within complex environments have to be simulated.

Virtual Storytelling for Training: An Application to Fire Fighting in Industrial Environment

The goal of this project is to build a virtual reality platform to educate fire fighters officers. Virtual reality allows to immerse users (teacher and learners) in a universe where the physical environment and human actors behavior are simulated. We propose an architecture where everything is an agent: reactive agents (natural phenomena), cognitive agents (firemen) and avatars (users). The two last types of agents coordinate their actions: they play a role in an organization to execute pre-established missions in team.

MASCARET: A Pedagogical Multi-Agent System for Virtual Environments for Training

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Mascaret: Pedagogical multi-agents system for virtual environment for training. Cédric Buche, Ronan Querrec, Pierre De Loor, Pierre Chevaillier

Fuzzy cognitive maps for the simulation of individual adaptive behaviors

This paper focuses on the simulation of behavior for autonomous entities in virtual environments. The behavior of these entities must determine their responses not only to external stimuli, but also with regard to internal states. We propose to describe such behavior using fuzzy cognitive maps (FCMs), whereby these internal states might be explicitly represented. This paper presents the use of FCMs as a tool to specify and control the behavior of individual agents. First, we describe how FCMs can be used to model behavior. We then present a learning algorithm allowing the adaptation of FCMs through observation. Copyright

Ensuring semantic spatial constraints in virtual environments using UML/OCL

Spatial objects and relationships between them, compose a spatial model that is the backbone of virtual environments (VEs). However, due to the natural complexity of both spatial objects and spatial information, the modeling of such spatial relationships is still a difficult task. This paper presents a novel approach for representing semantic spatial relations in VEs using the Unified Modeling Language (UML) and the Object Constraint Language (OCL). Our approach first uses the UML class model as a conceptual model for VEs. We then propose a spatial extension of OCL named VRX-OCL as a high-level and flexible language to cover multidimensional, manifold, and reference frame-dependent spatial constraints. We mainly focus on two important classes of spatial relations, namely, topological and projective relations that allow nonmetric representation of space. The applicability of our approach is demonstrated in the Virtual Physics Laboratory, a VE for learning physics. Based on the constraints satisfaction, the system is able to visualize abstract spatial information and thus provides educational assistance to the learners.

Collaborative virtual training with physical and communicative autonomous agents

Virtual agents are a real asset in collaborative virtual environment for training (CVET) as they can replace missing team members. Collaboration between such agents and users, however, is generally limited. We present here a whole integrated model of CVET focusing on the abstraction of the real or virtual nature of the actor to define a homogenous collaboration model. First, we define a new collaborative model of interaction. This model notably allows to abstract the real or virtual nature of a teammate. Moreover, we propose a new role exchange approach so that actors can swap their roles during training. The model also permits the use of physically based objects and characters animation to increase the realism of the world. Second, we design a new communicative agent model, which aims at improving collaboration with other actors using dialog to coordinate their actions and to share their knowledge. Finally, we evaluated the proposed model to estimate the resulting benefits for the users and we show that this is integrated in existing CVET applications. Copyright

A Continuous Model for the Management of Turn-Taking in User-Agent Spoken Interactions Based on the Variations of Prosodic Signals

Many recent works on agent architectures have focused on polite and optimal turn-transitions. However, real turn-taking is more complex due to several contextual variables, linked to each agent’s own goals (cooperative or non cooperative for example). For mixed-initiative interactions, we need to go beyond the polite agent context to make more complex patterns of turn-taking emerge. We present here an architecture based on a dynamical and continuous model of turn-taking, able to control the turn-taking behaviors of the agent depending on its willingness to speak or not. We show how we implemented our model based on human data and how complex patterns of turn-taking emerge from agent-agent simulations. Finally, we present the results of a perceptual experiment where we questioned participants about the intentions of two agents interacting.

The C2BDI Agent Architecture for Teamwork Coordination Using Spoken Dialogues between Virtual Agents and Users

In Collaborative Virtual Environments (VEs) for Training, users have to learn how to perform a collaborative task and also how to coordinate with teammates’ activities. Efficient coordination requires teammates to exchange information about their beliefs, goals and plans. The collaborative-conversational BDI agent (C2BDI) endows virtual agents with first, deliberative capabilities about the interdependency of their activities, and second, with task-oriented conversational capabilities that support multiparty spoken dialogues helping them to coordinate their activities with teammates [2]. This proposed solution has been used in two virtual reality applications: a real training scenario [1] and an application dedicated to scientific experiments [2]. The main motivations of this last was to control the characteristics of the collective activity and to be more extensible.

Task-Oriented Conversational Behavior of Agents for Collaboration in Human-Agent Teamwork

Coordination is an essential ingredient for human-agent teamwork. It requires team members to share knowledge to establish common grounding and mutual awareness among them. This paper proposes a behavioral architecture C 2 BDI that enhances the knowledge sharing using natural language communication between team members. Collaborative conversation protocols and resource allocation mechanism have been defined that provide proactive behavior to agents for coordination. This architecture has been applied to a real scenario in a collaborative virtual environment for learning. The solution enables users to coordinate with other team members.

A computational model for the emergence of turn-taking behaviors in user-agent interactions

We propose a computational model that endows conversational agents with the capability to coordinate their speaking turns (turn-taking management) in the context of mixed-initiative two-party dialogs. In human conversations, participants are continuously adjusting their verbal and non-verbal productions for ensuring the effective coordination of speaking turns. In our model, the decision making is a continuous process based on the intrinsic current goal of the agent with respect to turn-taking, namely its motivation to keep-or to leave-its current role (speaker or listener), and on its perception of the intentions of its partner. Concurrently, the agent is also producing signals indicating its willingness to maintain or leave its current role. Our model is based on two models from cognitive psychology: the drift-diffusion model and the theory of behavioral dynamics. After presenting simulations showing how our model makes the coordination emerge from the interactions, we propose a SAIBA-Compliant architecture, named BeAware, created to support the implementation of our model. Finally, using our model, we investigate how an agent’s turn-taking strategy may impact the user’s experience and the effectiveness of the coordination.