Curtis W. Nielsen

Validating human-robot interaction schemes in multitasking environments

The ability of robots to autonomously perform tasks is increasing. More autonomy in robots means that the human managing the robot may have available free time. It is desirable to use this free time productively, and a current trend is to use this available free time to manage multiple robots. We present the notion of neglect tolerance as a means for determining how robot autonomy and interface design determine how free time can be used to support multitasking, in general, and multirobot teams, in particular. We use neglect tolerance to 1) identify the maximum number of robots that can be managed; 2) identify feasible configurations of multirobot teams; and 3) predict performance of multirobot teams under certain independence assumptions. We present a measurement methodology, based on a secondary task paradigm, for obtaining neglect tolerance values that allow a human to balance workload with robot performance.

Ecological Interfaces for Improving Mobile Robot Teleoperation

Navigation is an essential element of many remote robot operations including search and rescue, reconnaissance, and space exploration. Previous reports on using remote mobile robots suggest that navigation is difficult due to poor situation awareness. It has been recommended by experts in human-robot interaction that interfaces between humans and robots provide more spatial information and better situational context in order to improve an operators situation awareness. This paper presents an ecological interface paradigm that combines video, map, and robot-pose information into a 3-D mixed-reality display. The ecological paradigm is validated in planar worlds by comparing it against the standard interface paradigm in a series of simulated and real-world user studies. Based on the experiment results, observations in the literature, and working hypotheses, we present a series of principles for presenting information to an operator of a remote robot.

Shared understanding for collaborative control

This paper presents results from three experiments in which human operators were teamed with a mixed-initiative robot control system to accomplish various indoor search and exploration tasks. By assessing human workload and error together with overall performance, these experiments provide an objective means to contrast different modes of robot autonomy and to evaluate both the usability of the interface and the effectiveness of autonomous robot behavior. The first experiment compares the performance achieved when the robot takes initiative to support human driving with the opposite case when the human takes initiative to support autonomous robot driving. The utility of robot autonomy is shown through achievement of better performance when the robot is in the drivers seat. The second experiment introduces a virtual three-dimensional (3-D) map representation that supports collaborative understanding of the task and environment. When used in place of video, the 3-D map reduced operator workload and navigational error. By lowering bandwidth requirements, use of the virtual 3-D interface enables long-range, nonline-of-sight communication. Results from the third experiment extend the findings of experiment 1 by showing that collaborative control can increase performance and reduce error even when the complexity of the environment is increased and workload is distributed amongst multiple operators.

Comparing the usefulness of video and map information in navigation tasks

One of the fundamental aspects of robot teleoperation is the ability to successfully navigate a robot through an environment. We define successful navigation to mean that the robot minimizes collisions and arrives at the destination in a timely manner. Often video and map information is presented to a robot operator to aid in navigation tasks. This paper addresses the usefulness of map and video information in a navigation task by comparing a side-by-side (2D) representation and an integrated (3D) representation in both a simulated and a real world study. The results suggest that sometimes video is more helpful than a map and other times a map is more helpful than video. From a design perspective, an integrated representation seems to help navigation more than placing map and video side-by-side.

Ecological displays for robot interaction: a new perspective

Most interfaces for robot control have focused on providing users with the most current information and giving status messages about what the robot is doing. While this may work for people that are already experienced in robotic, we need an alternative paradigm for enabling new users to control robots effectively. Instead of approaching the problem as an issue of what information could be useful, the focus should be on presenting essential information in an intuitive way. One way to do this is to leverage perceptual cues that people are accustomed to using. By displaying information in such contexts, people are able to understand and use the interface more effectively. This paper presents interfaces which allow users to navigate in 3-D worlds with integrated range and camera information.

Snapshots for semantic maps

A significant area of research in mobile robotics is in the local representation of a remote environment. In order to include a human in a mobile robot task it becomes important to present the remote information efficiently to a human. A relatively new approach to information presentation is semantic maps. Semantic maps provide more detail about an environment than typical maps because they are augmented by icons or symbols that provide meaning for places or objects of interest. In this paper we present snapshot technology as a means to take pictures from the real world and store them in a semantic map. To make the snapshots and semantic map available to an operator, we identify and discuss general attributes for useful displays and present a mixed reality 3D interface that meets the requirements. The interface and snapshot technology are validated through experiments in real and simulated environments.

Towards predicting robot team performance

In this paper we develop a method for predicting the performance of human-robot teams consisting of a single user and multiple robots. To predict the performance of a team, we first measure the neglect tolerance and interface efficiency of the interaction schemes employed by the team. We then describe a method that shows how these measurements can be used to estimate the teams performance. We validate the performance prediction algorithm by comparing predictions to actual results when a user guides three robots in an exploration and goal-finding mission; comparisons are made for various system configurations.

Towards facilitating the use of a pan-tilt camera on a mobile robot

Pan-tilt cameras are useful for mobile robots because they allow an operator to view the environment around the robot without moving the robot. However, pan-tilt cameras can lead to operator confusion and error when the camera is off-center and the operator is navigating the robot. To facilitate the use of a pan-tilt camera, we use a 3D augmented virtuality display to render the video stream at an angle that corresponds to the orientation of the real camera. This provides an intuitive representation of the pan and tilt angles of the camera as they relate to the orientation of the robot. We validate using the 3D display to represent a pan-tilt cameras orientation through a user study in simulated environments.

Hiding the System from the User: Moving from Complex Mental Models to Elegant Metaphors

In previous work, increased complexity of robot behaviors and the accompanying interface design often led to operator confusion and/or a fight for control between the robot and operator. We believe the reason for the conflict was that the design of the interface and interactions presented too much of the underlying robot design model to the operator. Since the design model includes the implementation of sensors, behaviors, and sophisticated algorithms, the result was that the operators cognitive efforts were focused on understanding the design of the robot system as opposed to focusing on the task at hand. This paper illustrates how this very problem emerged at the INL and how the implementation of new metaphors for interaction has allowed us to hide the design model from the user and allow the user to focus more on the task at hand. Supporting the users focus on the task rather than on the design model allows increased use of the system and significant performance improvement in a search task with novice users.

Using mixed-initiative human-robot interaction to bound performance in a search task

Mobile robots are increasingly used in dangerous domains, because they can keep humans out of harmpsilas way. Despite their advantages in hazardous environments, their general acceptance in other less dangerous domains has not been apparent and, even in dangerous environments, robots are often viewed as a ldquolast-possible choice.rdquo In order to increase the utility and acceptance of robots in hazardous domains researchers at the Idaho National Laboratory have both developed and tested novel mixed-initiative solutions that support the human-robot interactions. In a recent ldquodirtybombrdquo experiment, participants exhibited different search strategies making it difficult to determine any performance benefits. This paper presents a method for categorizing the search patterns and shows that the mixed-initiative solution decreased the time to complete the task and decreased the performance spread between participants independent of prior training and of individual strategies used to accomplish the task.