Christopher A. Miller

Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control

Objective: To develop a method enabling human-like, flexible supervisory control via delegation to automation. Background: Real-time supervisory relationships with automation are rarely as flexible as human task delegation to other humans. Flexibility in human-adaptable automation can provide important benefits, including improved situation awareness, more accurate automation usage, more balanced mental workload, increased user acceptance, and improved overall performance. Method: We review problems with static and adaptive (as opposed to “adaptable”) automation; contrast these approaches with human-human task delegation, which can mitigate many of the problems; and revise the concept of a “level of automation” as a pattern of task-based roles and authorizations. We argue that delegation requires a shared hierarchical task model between supervisor and subordinates, used to delegate tasks at various levels, and offer instruction on performing them. A prototype implementation called Playbook® is described. Results: On the basis of these analyses, we propose methods for supporting human-machine delegation interactions that parallel human-human delegation in important respects. We develop an architecture for machine-based delegation systems based on the metaphor of a sports teams “playbook.” Finally, we describe a prototype implementation of this architecture, with an accompanying user interface and usage scenario, for mission planning for uninhabited air vehicles. Conclusion: Delegation offers a viable method for flexible, multilevel human-automation interaction to enhance system performance while maintaining user workload at a manageable level. Application: Most applications of adaptive automation (aviation, air traffic control, robotics, process control, etc.) are potential avenues for the adaptable, delegation approach we advocate. We present an extended example for uninhabited air vehicle mission planning.

Trust and etiquette in high-criticality automated systems

1 It might, perhaps, be more accurate to say the etiquette is perceived by human users rather than exhibited by the automation itself, but that subtlety is largely irrelevant to the work we review here. Whereas the other articles in this section discuss human-computer etiquette in traditional social interactions involving the use of computers that explicitly strive to elicit a perception of “personhood” from the human participant, we focus on computers that occupy more traditional roles as complex and largely unpersonified machines involved in high-criticality working relationships with humans—where the consequences of failure can be catastrophic in terms of lives, money, or both. Politeness and social niceties are important in many human-human social interactions, but in critical, highly technical work, there is the common misperception that we can “dispense with protocol” and get down to business, even with those who are not particularly courteous. In fact, excessive adherence to polite norms can seem stilted and sometimes frustrating in such settings. Here, we argue the etiquette exhibited 1

Beyond Levels of Automation: An Architecture for More Flexible Human-Automation Collaboration

Supervisory control has long been recognized as a design concept for enhancing the effectiveness of human-automation interaction, but real time supervisory relationships with automation have rarely approached the flexibility of human-human supervisory relationships. In part, this may be because our models of human-automation relationships— primarily, Sheridan and Verplanks (1978) concept of Levels of Automation—have been unidimensional and too coarse-grained. Parasuraman, Wickens and Sheridan (2000) acknowledged this limitation and extended the levels of automation model to include a second, albeit coarse-grained, dimension based on stages of information processing. We argue that, for the purpose of providing a detailed architecture to support collaborative delegation interactions, this process needs to be taken further. By applying the levels of automation spectra across a detailed task model of tasks in a work domain, we create a suitably complex and flexible model of task performance which can be shared between humans and automation and used as the vocabulary for delegation interactions. This architecture is illustrated in human tasking of multiple Unmanned Air Vehicles.

18. Delegation Interfaces for Human Supervision of Multiple Unmanned Vehicles: Theory, Experiments, and Practical Applications

A fundamental issue driving much of the current research is the design of the interface between humans and ROVs. Autonomous robots are sufficiently different from most computer systems as to require new research and design principles (Adams & Skubic, 2005; Kiesler & Hinds, 2004). Previous work on coordination between humans and automated agents has revealed both benefits and costs of automation for system performance (Parasuraman & Riley, 1997). Automation is clearly essential for the operation of many complex human–machine systems. But in some circumstances automation can also lead to novel problems for operators. Automation can increase workload and training requirements, impair situation awareness and, when particular events co-occur in combination with poorly designed interfaces, lead to accidents (e.g., Degani, 2004; Parasuraman & Riley, 1997).

Multi-Unmanned Aerial Vehicle Systems Control via Flexible Levels of Interaction: An Adaptable Operator-Automation Interface Concept Demonstration

This paper provides a detailed description of a concept demonstration that illustrates an adaptable operator-automation interface concept for applications involving multiple Unmanned Aerial Systems (UAS). Key hardware and software components are described, as well as the changes made on a previous prototype that focused on single UAS applications. The concept demonstration illustrates four control methods, ranging from manual (operator controls the vehicle’s flight with stick and throttle control) to high level “plays” (operator’s verbal command initiates planning for a series of automated tasks), as well as the means to flexibly change or intermingle methods for different vehicles at any time. To support multiUAS control, the effort involved expanding the previous design to include task-centric (rather than vehicle-centric) interface components.

Delegating to Automation Performance, Complacency and Bias Effects under Non-Optimal Conditions

We have advocated adaptable automation approaches—those in which the human retains the role of instructing and tasking—and specifically have used the metaphor of a sports team’s “playbook”. Several prior experiments have shown benefits to flexible play calling, so the present work focuses on performance in “non-optimal play environments” (NOPEs) where the defined plays are a poor fit resulting in a need to either modify them dynamically (provide additional instruction) or to abandon play-level automation and resort to more manual levels of control. We might expect that prolonged play usage under optimal conditions would result in automation complacency effects and even loss of training. In two reported experiments, we find little evidence for complacency effects and, instead, show that having access to plays sometimes provides benefits even during NOPE intervals where they were not (directly) useful.

Adaptable Automation Interface for Multi-Unmanned Aerial Systems Control Preliminary Usability Evaluation

With advances in automation technologies, systems are now being considered wherein a single operator supervises multiple unmanned aerial vehicles. Supervisory control of highly autonomous systems will require a new interface design. The present effort extends a delegation control concept to enable a pilot to flexibly change the role of automation during the course of a mission, seamlessly transitioning between four different control modes ranging from manual (pilot controls the vehicle’s flight with stick and throttle control) to high level “plays” (pilot’s command initiates a series of automated tasks). This novel concept was instantiated into a dynamic laptop simulation to support a usability evaluation in which participants employed the multi-level control architecture during ninety minute sessions. Data include comments recorded with a think-aloud paradigm and questionnaire responses. Results indicated that this adaptable pilot-automation interface for multi-unmanned systems control is promising. The findings include perspectives from both pilot and gamer participants that will help advance design of multi-level control for future aviation systems.

Adaptable Operator-Automation Interface for Future Unmanned Aerial Systems Control: Development of a Highly Flexible Delegation Concept Demonstration

A prototype demonstration was created to illustrate a delegation control approach for future unmanned aerial systems (UAS) applications. With the goal of being able to demonstrate a flexible, natural, multi-level architecture for UAS control, the simulation illustrates four different control modes, ranging from manual (pilot controls the vehicles flight with stick and throttle control) to high level plays (pilots verbal command initiates planning for a series of automated tasks). The present paper will provide a detailed description of the concept demonstration, including key hardware and software components. This demonstration was used to support an operator-centered effort that involves acquiring feedback from operators to guide interface designs for future UAS applications. Feedback obtained to date is summarized in a companion paper titled Future Unmanned Aerial Systems Control: Feedback on Highly Flexible Operator-Automation Delegation Interface Concept.

Human Control of Teams of Unmanned/Robotic Vehicles: Exploring the Limits of the Possible

Robotic vehicles under human control are currently being used for exploration and surveillance in variety of environments that are inaccessible or dangerous for humans. The operation of such vehicles is currently labor intensive, however, with multiple operators typically required for each vehicle employed. This panel reports on progress being made in the control of teams of robotic vehicles by much smaller teams of human operators through the combination of sophisticated automated control algorithms and human supervision. The goal is mixed initiative control—flexible and dynamic shifting of responsibility between humans and automation as required over the course of a mission. The panel will discuss cognitive engineering approaches to the design of a mixed initiative system for multi-vehicle robotic control including development and experimental testing of prototype interface designs, workload modeling, and optimal team design. The strengths, limitations, and potential complementarity of these approaches will be discussed.