Jason Daly

VESS: coordinating graphics, audio, and user interaction in virtual reality applications

VESS (Virtual Environment Software Sandbox) is a suite of libraries designed to aid in the creation of applications for virtual reality research. It combines the power of a scene graph library, a flexible input device library, and support for other senses (audio, haptics, etc.). It then adds higher-level libraries that integrate graphics and user input, creating a single, coherent platform that is flexible and easy to use. VESS handles the technical challenges facing virtual reality applications, leaving the developer free to focus on the details of the application itself.

Application of augmented reality to visualizing anatomical airways

Visualizing information in three dimensions provides an increased understanding of the data presented. Furthermore, the ability to manipulate or interact with data visualized in three dimensions is superior. Within the medical community, augmented reality is being used for interactive, three-dimensional (3D) visualization. This type of visualization, which enhances the real world with computer generated information, requires a display device, a computer to generate the 3D data, and a system to track the user. In addition to these requirements, however, the hardware must be properly integrated to insure correct visualization. To this end, we present components of an integrated augmented reality system consisting of a novel head-mounted projective display, a Linux-based PC, and a commercially available optical tracking system. We demonstrate the system with the visualization of anatomical airways superimposed on a human patient simulator.

Software infrastructure for multi-modal virtual environments

Virtual environment systems, especially those supporting multi-modal interactions require a robust and flexible software infrastructure that supports a wide range of devices, interaction techniques, and target applications. In addition to interactivity needs, a key factor of robustness of the software is the minimization of latency and more importantly, reduction of jitter (the variability of latency). This paper presents a flexible software infrastructure that has demonstrated robustness in initial prototyping. The infrastructure, based on the VESS Libraries from the University of Central Florida, simplifies the task of creating multi-modal virtual environments. Our extensions to VESS include numerous features to support new input and output devices for new sensory modalities and interaction techniques, as well as some control over latency and jitter.

Technological and usability-based aspects of distributed after action review in a game-based training setting

After action review (AAR) in the distributed setting provides for some unique problems. Some of these problems include remote facilitation of an after action review, keeping a lightweight infrastructure that can handle large amounts of throughput and allowing for different AAR sessions to be run simultaneously. This paper proposes a method for developing a facilitative infrastructure in the AAR setting while providing a solution that allows for syncing of multiple AAR software to one review session.

Haptic enhancements for collaborative scenarios in virtual environment

The more senses a virtual reality system can simulate, the greater the realism and sense of immersion experienced by the user. Nearly every system provides the vital visual sense, and most provide some degree of audio reproduction as well. However, when the task involves a significant amount of manual manipulation of virtual objects, or when the scenario calls for both non-verbal and non-visual communication between its participants, the sense of touch becomes vital.

User interface and information management of scenarios

Scenario-based training has shown to be an effective instructional approach. However, it can be difficult for instructors to create scenarios as they are time-consuming to develop. This can result in the same scenarios being re-used, reducing training effectiveness. Furthermore, scenarios are based on training objectives and can vary in complexity. Researchers have investigated methods to simplify this process by assessing various approaches to scenario generation. Within such a tool, however, there needs to be a way to easily understand the different scenario components and their relationships within the scenario. This paper discusses the PYTHAGORAS (Procedural Yielding Techniques and Heuristics for Automated Generation of Objects with Related and Analogous Scenarios) system and specifically its editor used to create scenarios. We designed this system to simplify the process of scenario generation, providing a clearer understanding of the scenario components and relationships.

Viability of mobile devices for training purposes

Mobile devices offer an advantageous platform on which to perform training simulations. However, they create new issues that are unique to mobile device development. First we will explore several reasons for using mobile devices for training simulators, instead of desktop or legacy-based systems. Once we have established the requirements of mobile simulators, we will discern some of the differences that arise between major mobile platforms in each development area. Finally we propose a solution that will address these differences and aid developers in creating cross-platform mobile simulations.

Integrating Instructional Strategies and Haptic Technologies to Enhance the Training Efficacy of an Army 91W (Combat Medic) Medical Skills Training Simulation

This paper describes the design of HapMed, a haptics-based simulation system for training US Army 91W10 combat medic tasks. Particular emphasis is placed on a cognitive engineering design methodology for integrating haptic technologies and instructional design strategies to enhance HapMed training effectiveness. The unique design of HapMed also addresses the Armys critical need for a modular, field deployable training system that bridges the gap between civilian first responder training and combat casualty care.