Julie L. Marble

Shared understanding for collaborative control

This paper presents results from three experiments in which human operators were teamed with a mixed-initiative robot control system to accomplish various indoor search and exploration tasks. By assessing human workload and error together with overall performance, these experiments provide an objective means to contrast different modes of robot autonomy and to evaluate both the usability of the interface and the effectiveness of autonomous robot behavior. The first experiment compares the performance achieved when the robot takes initiative to support human driving with the opposite case when the human takes initiative to support autonomous robot driving. The utility of robot autonomy is shown through achievement of better performance when the robot is in the drivers seat. The second experiment introduces a virtual three-dimensional (3-D) map representation that supports collaborative understanding of the task and environment. When used in place of video, the 3-D map reduced operator workload and navigational error. By lowering bandwidth requirements, use of the virtual 3-D interface enables long-range, nonline-of-sight communication. Results from the third experiment extend the findings of experiment 1 by showing that collaborative control can increase performance and reduce error even when the complexity of the environment is increased and workload is distributed amongst multiple operators.

Evaluation of supervisory vs. peer-peer interaction with human-robot teams

We submit that the most interesting and fruitful human-robot interaction (HRI) may be possible when the robot is able to interact with the human as a true team member, rather than a tool. However, the benefits of shared control can all too easily be overshadowed by challenges inherent to blending human and robot initiative. The most important requirements for peer-peer interaction are system trust and ability to predict system behavior. The human must be able to understand the reason for and effects of robot initiative. These requirements can only be met through careful application of human factors principles and usability testing to determine how users interact with the system. This paper discusses the recent human participant usability testing, which took our current implementation to task using a search and rescue scenario within a complex, real-world environment. The purpose of testing was to examine how human operators work with the robotic system at each level of autonomy, and how interaction with the robot should be structured to enable situation awareness and task completion. Analyses revealed that our architecture equally supported situation awareness and target detection by novices and experts, although experienced users were more likely to have more performance expectations of the interface. Results also had implications regarding the ability of participants to effectively utilize the collaborative workspace and, most importantly, their ability to understand and willingness to accept robot initiative.

Lessons learned from usability tests with a collaborative cognitive workspace for human-robot teams

Use of new robotics technologies is challenged by issues of system trust, unknowns regarding how the system will, can, and should be used, and possibilities for human error that may cause harm to the human operator, system, or environment. This paper discusses initial usability tests of a mixed-initiative robotic system. Participants were asked to search a building using a robot equipped with multiple levels of autonomy to identify 3 targets in pre-specified locations. The experiment showed a significant difference between novice and experienced robotic operators especially regarding willingness to use the autonomous capabilities of the robot. Users unfamiliar with teleoperation were more willing to utilize the autonomous capabilities of the robot, while skilled teleoperators preferred and were more efficient when in direct control. Users were almost always able to successfully complete the search task. However, feedback indicates that users, having been given only a cursory explanation of the system, were sometimes confused by robot initiative even though the interface supplied textual explanations. The experiment shows that mixed-initiative interaction may exceed the limitations of either fully autonomous or teleoperated control; however, potential benefits can easily be overshadowed by control challenges inherent to deploying robot-human teams.

I call shotgun!: an evaluation of mixed-initiative control for novice users of a search and rescue robot

The quality of human-robot interaction trails other advances in robotics and may prove to be a limiting factor when deploying remote, mobile robots for critical applications. One reason is that most autonomous robot behaviors are not robust and often degrade in unstructured environments. Another reason is that the design of human-robot interaction (HRI) and interfaces fails to follow basic usability principles or be informed by basic concepts of human-computer interaction. To address both these challenges, we have used a development cycle of iterative usability testing and redesign to hone both our interface and the robot behaviors that support it. The present paper presents results from a wide swathe of over 100 novices who used the resulting system to accomplish a real-world search and detection task. The current interface proved to be highly usable by novices, regardless of age or gender. The study demonstrates the utility of effective robot autonomy and examines the benefits of mixed-initiative control. In particular, the study compares the performance achieved when the robot takes initiative to support human driving versus the case where the human takes initiative to support autonomous robot driving. Results indicate that performance is better when the robot is in the drivers seat. Optimal performance was achieved when the operator focuses on the search and rescue task and provides only intermittent direction to the robot.

Simplified Expert Elicitation Procedure for Risk Assessment of Operating Events

This report describes a simplified, tractable, and usable procedure within the US Nuclear Regulator Commission (NRC) for seeking expert opinion and judgment. The NRC has increased efforts to document the reliability and risk of nuclear power plants (NPPs) through Probabilistic Risk Assessment (PRA) and Human Reliability Analysis (HRA) models. The Significance Determination Process (SDP) and Accident Sequence Precursor (ASP) programs at the NRC utilize expert judgment on the probability of failure, human error, and the operability of equipment in cases where otherwise insufficient operational data exist to make meaningful estimates. In the past, the SDP and ASP programs informally sought the opinion of experts inside and outside the NRC. This document represents a formal, documented procedure to take the place of informal expert elicitation. The procedures outlined in this report follow existing formal expert elicitation methodologies, but are streamlined as appropriate to the degree of accuracy required and the schedule for producing SDP and ASP analyses.

Mutual initiative in human-machine teams

Autonomous systems are widely used today in industry. The human-machine relationship in these systems is primarily that of a human supervisory role. This paper explores the concept of human-robot teams where each member of the team has the ability to assume initiative within a task. Key to this effort is not only the ability of the human to understand and predict robot performance, but the robots ability to identify human needs and select intervention points to assume different levels of initiative. The objective is to incorporate mobile autonomous robots into human teams to augment both the humans cognitive and physical abilities in the performance of potentially hazardous tasks.

Intelligent autonomy for remote characterization of hazardous environments

Remote characterization of high radiation environments is a pressing application area where robots have the potential to provide benefits in terms of time, cost, safety and quality of data. However, the ability to design robots that can be used effectively has proven to be no easy task. In 2001, the Idaho National Engineering and Environmental Laboratory (INEEL) successfully deployed a teleoperated robotic system coupled with a Gamma Locating and Isotopic Identification Device (RGL&IID) to characterize an area that had been closed to human entry for many years. This paper examines the limitations to the control strategy used and discusses how current efforts at the INEEL are developing intelligent controls that can actively mediate between the human and the robotic elements of the system. The resulting, mixed-initiative control architecture allows the user to shift the level of robot initiative throughout the task as needed. This system offers the opportunity for the human and robot to become a team where each can support the capabilities and limitations of the other.

PROOF OF CONCEPT FOR A HUMAN RELIABILITY ANALYSIS METHOD FOR HEURISTIC USABILITY EVALUATION OF SOFTWARE

An ongoing issue within human-computer interaction (HCI) is the need for simplified or “discount” methods. The current economic slowdown has necessitated innovative methods that are results driven and cost effective. The myriad methods of design and usability are currently being cost-justified, and new techniques are actively being explored that meet current budgets and needs. Recent efforts in human reliability analysis (HRA) are highlighted by the ten-year development of the Standardized Plant Analysis Risk HRA (SPAR-H) method. The SPAR-H method has been used primarily for determining humancentered risk at nuclear power plants. The SPAR-H method, however, shares task analysis underpinnings with HCI. Despite this methodological overlap, there is currently no HRA approach deployed in heuristic usability evaluation. This paper presents an extension of the existing SPAR-H method to be used as part of heuristic usability evaluation in HCI.

Human Reliability Analysis in the U.S. Nuclear Power Industry: A Comparison of Atomistic and Holistic Methods

A variety of methods have been developed to generate human error probabilities for use in the US nuclear power industry. When actual operations data are not available, it is necessary for an analyst to estimate these probabilities. Most approaches, including THERP, ASEP, SLIM-MAUD, and SPAR-H, feature an atomistic approach to characterizing and estimating error. The atomistic approach is based on the notion that events and their causes can be decomposed and individually quantified. In contrast, in the holistic approach, such as found in ATHEANA, the analysis centers on the entire event, which is typically quantified as an indivisible whole. The distinction between atomistic and holistic approaches is important in understanding the nature of human reliability analysis quantification and the utility and shortcomings associated with each approach.