Josh Clow

QuickSet: multimodal interaction for distributed applications

QuickSet: Multimodal Interaction for Distributed Applications Philip R. Cohen, Michael Johnston, David McGee, Sharon Oviatt, Jay Pittman, Ira Smith, Liang Chen and Josh Glow Center for Human Computer Communication Oregon Graduate Institute of Science and Technology

QuickSet: Multimodal Interaction for Simulation Set-up and Control

This paper presents a novel multimodal system applied to the setup and control of distributed interactive simulations. We have developed the QuickSet prototype, a pen/voice system running on a hand-held PC, communicating through a distributed agent architecture to NRaDs1 LeatherNet system, a distributed interactive training simulator built for the US Marine Corps (USMC). The paper briefly describes the system and illustrates its use in multimodal simulation setup.

Multimodal interaction for 2D and 3D environments [virtual reality]

The allure of immersive technologies is undeniable. Unfortunately, the users ability to interact with these environments lags behind the impressive visuals. In particular, its difficult to navigate in unknown visual landscapes, find entities, access information and select entities using six-degrees-of-freedom (6-DOF) devices. We believe multimodal interaction-specifically speech and gesture-will make a major difference in the usability of such environments.

The efficiency of multimodal interaction for a map-based task

This paper compares the efficiency of using a standard direct-manipulation graphical user interface (GUI) with that of using the QuickSet pen/voice multimodal interface for supporting a military task. In this task, a user places military units and control measures (e.g., various types of lines, obstacles, objectives) on a map. Four military personnel designed and entered their own simulation scenarios via both interfaces. Analyses revealed that the multimodal interface led to a 3 to 4-fold speed improvement in the average entity creation time, including all error handling. Time to repair errors also was significantly faster when interacting multimodally. These results indicate a substantial efficiency advantage for multimodal over GUI-based interaction during map-based tasks.