Jeff Rickel

Animated agents for procedural training in virtual reality: Perception, cognition, and motor control

This paper describes Steve , an animated agent that helps students learn to perform physical , procedural tasks . The student and Steve cohabit a three - dimensional , simulated mock - up of the students work environment . Steve can demonstrate how to perform tasks and can also monitor students while they practice tasks , providing assistance when needed . This paper describes Steves architecture in detail , including perception , cognition , and motor control . The perception module monitors the state of the virtual world , maintains a coherent representation of it , and provides this information to the cognition and motor control modules . The cognition module interprets its perceptual input , chooses appropriate goals , constructs and executes plans to achieve those goals , and sends out motor commands . The motor control module implements these motor commands , controlling Steves voice , locomotion , gaze , and gestures , allowing Steve to manipulate objects in the virtual world .

Toward the holodeck: integrating graphics, sound, character and story

We describe an initial prototype of a holodeck- like environment that we have created for the Mission Rehearsal Exercise Project. The goal of the project is to create an experience learning system where the participants are immersed in an environment where they can encounter the sights, sounds, and circumstances of real-world scenarios. Virtual humans act as characters and coaches in an interactive story with pedagogical goals.

Embodied agents for multi-party dialogue in immersive virtual worlds

Immersive virtual worlds are increasingly being used for education, training, and entertainment, and virtual humans that can interact with human users in these worlds play many important roles. However, current computational models of dialogue do not address the issues that arise with face-to-face communication situated in three-dimensional worlds, such as the proximity and attentional focus of others, the ability to maintain multi-party conversations, and the interplay between speech and nonverbal signals. This paper presents a new model that integrates and extends prior work on spoken dialogue and embodied conversational agents, and describes an initial implementation that has been applied to training in virtual reality.

Toward a new generation of virtual humans for interactive experiences

Virtual humans - autonomous agents that support face-to-face interaction in a variety of roles - can enrich interactive virtual worlds. Toward that end, the Mission Rehearsal Exercise project involves an ambitious integration of core technologies centered on a common representation of task knowledge.

Toward virtual humans

This article describes the virtual humans developed as part of the Mission Rehearsal Exercise project, a virtual reality-based training system. This project is an ambitious exercise in integration, both in the sense of integrating technology with entertainment industry content, but also in that we have joined a number of component technologies that have not been integrated before. This integration has not only raised new research issues, but it has also suggested some new approaches to difficult problems. We describe the key capabilities of the virtual humans, including task representation and reasoning, natural language dialogue, and emotion reasoning, and show how these capabilities are integrated to provide more human-level intelligence than would otherwise be possible.

Lifelike pedagogical agents and affective computing: an exploratory synthesis

Lifelike pedagogical agents have been the subject of increasing attention in the agents and knowledge-based learning environment communities [2, 17, 19—21]. In parallel developments, recent years have witnessed great strides in work on cognitive models of emotion and affective reasoning [4,18, 22]. As a result, the time is now ripe for exploring how affective reasoning can be incorporated into pedagogical agents to improve students’ learning experiences.

Integrating Pedagogical Agents into Virtual Environments

In order for a virtual environment to be effective as a training tool, it is not enough to concentrate on the fidelity of the renderings and the accuracy of the simulated behaviors. The environment should help trainees develop an understanding of the task and should provide guidance and assistance as needed. This paper describes a system for developing virtual environments in which pedagogical capabilities are incorporated into autonomous agents that interact with trainees and simulations of objects in the environment. These pedagogical agents can monitor trainee progress and provide guidance and assistance. This paper describes the architectural features of the environment and of the agents that accomplish the instructional objectives within the virtual environment. It also discusses how agent-based instruction is combined with other methods of delivering instruction.

Integrating pedagogical capabilities in a virtual environment agent

Virtual environments are a promising milieu for education and training, because they allow students to practice their skills in 3D simulations of work settings. Autonomous agents can improve the eeectiveness of such e n vironments by assisting and collaborating with students as appropriate. This paper describes an autonomous pedagogical agent called Steve that can support the training of procedural skills such as operating or repairing complex equipment. Steves architecture permits him to sense and manipulate dynamic virtual worlds. The architecture also enables Steve to assume alternative realizations, either as a full, articulated, human gure or as abstract pointers and disembodied hands. Steve employs a combination of intelligent capabilities in his interactions with students and the environment: plan revision and execution, explanation, and student monitoring. These capabilities are employed in multiple ways in order to support alternative pedagogical styles. Steves knowledge representation is designed so that agent capabilities can be authored without detailed knowledge of agent architectures and languages.

Automated modeling of complex systems to answer prediction questions

Abstract A question about the behavior of a complex, physical system can be answered by simulating the system—the challenge is building a model of the system that is appropriate for answering the question. If the model omits relevant aspects of the system, the predicted behavior may be wrong. If, on the other hand, the model includes many aspects that are irrelevant to the question, it may be difficult to simulate and explain. The leading approach to automated modeling, “compositional modeling”, constructs a simplest adequate model for a question from building blocks (“model fragments”) that are designed by knowledge engineers. This paper presents a new compositional modeling algorithm that constructs models from simpler building blocks—the individual influences among system variables—and addresses important modeling issues that previous programs left to the knowledge engineer. In the most rigorous test of a modeling algorithm to date, we implemented our algorithm, applied it to a large knowledge base for plant physiology, and asked a domain expert to evaluate the models it produced.

A methodology for developing a web-based factory simulator for manufacturing education

Historically, manufacturing engineering education has focused on teaching mathematical models using simplifying assumptions that can mask the realities of complex manufacturing systems. Recent pedagogical approaches to manufacturing education have focused on developing a more holistic view of the manufacturing enterprise. In this paper, we describe the contents and development methodology of a Virtual Factory Teaching System (VFTS) whose aim is to provide a workspace that illustrates the concepts of factory management and design for complex manufacturing systems. The VFTS is unique in its integration of four domains: web-based simulations, engineering education, the Internet, and virtual factories. Evolutionary development of the VFTS is accomplished by separating the simulation model from the graphical interface and user interaction.