Robert J. Shively

Shaping Trust Through Transparent Design: Theoretical and Experimental Guidelines

The current research discusses transparency as a means to enable trust of automated systems. Commercial pilots (N = 13) interacted with an automated aid for emergency landings. The automated aid provided decision support during a complex task where pilots were instructed to land several aircraft simultaneously. Three transparency conditions were used to examine the impact of transparency on pilot’s trust of the tool. The conditions were: baseline (i.e., the existing tool interface), value (where the tool provided a numeric value for the likely success of a particular airport for that aircraft), and logic (where the tool provided the rationale for the recommendation). Trust was highest in the logic condition, which is consistent with prior studies in this area. Implications for design are discussed in terms of promoting understanding of the rationale for automated recommendations.

A Human-Autonomy Teaming Approach for a Flight-Following Task

Human involvement with increasingly autonomous systems must adjust to allow for a more dynamic relationship involving cooperation and teamwork. As part of an ongoing project to develop a framework for human-autonomy teaming (HAT) in aviation, a study was conducted to evaluate proposed tenets of HAT. Participants performed a flight-following task at a ground station both with and without HAT features enabled. Overall, participants preferred the ground station with HAT features enabled over the station without the HAT features. Participants reported that the HAT displays and automation were preferred for keeping up with operationally important issues. Additionally, participants reported that the HAT displays and automation provided enough situation awareness to complete the task, reduced the necessary workload and were efficient. Overall, there was general agreement that HAT features supported teaming with the automation. These results will be used to refine and expand our proposed framework for human-autonomy teaming.

Field study of communication and workload in police helicopters - Implications for AI cockpit design

This paper reports on the work performed by civilian helicopter crews, using audio and video recordings and a variety of workload measures (heart rate and subjective ratings) obtained in a field study of public service helicopter missions. The number and frequency of communications provided a significant source of workload. This is relevant to the design of automated cockpit systems, since many designs presuppose the use of voice I/O systems. Fluency of communications (including pauses, hesitation markers, repetitions, and false starts) furnished an early indication of the effects of fatigue. Three workload measures were correlated to identify high workload segments of flight, and to suggest alternate task allocations between crew members.

Unmanned Aircraft System Response to Air Traffic Control Clearances Measured Response

In the National Airspace System (NAS), Air Traffic Control (ATC) expects aircraft to complete ATC clearances in a timely manner in order to maintain minimum separation between aircraft. The end-to-end response time for an aircraft to complete a clearance, as measured from the end of ATC instructing the pilot of the clearance to the just noticeable difference (JND) on the ATC display of the aircraft satisfying the clearance (i.e., initiation/completion of an altitude climb), can be referred to as measured response (MR). This MR is not quantified in Federal Aviation Administration (FAA) standards, regulations, or policy; however, as manned aircraft have developed along with the Air Traffic Management System, a shared understanding of reasonable and timely response has evolved. By contrast, the introduction of unmanned aircraft systems (UAS) into the NAS has highlighted this issue. This paper seeks to define MR and its components, and describe a methodology, with an example, that can be used to investigate it.

Decision support for workload assessment - Introducing WC FIELDE

Currently there is a great demand for mental workload evaluation in the course of system design and modification. In light of this demand, a microprocessor-based decision support system has been created called WC FIELDE: Workload Consultant for FIELD Evaluation. The system helps the user select workload measures appropriate to his or her application from the large pool of currently available techniques. Both novices and those with some workload experience may benefit from using WC FIELDE, since the systems operation is entirely transparent and all rules involved in the decision process are available for the user to examine. WC FIELDE recommends several assessment methodologies in decreasing order of appropriateness, and provides additional information on each measure at the end of the program in the form of text files.

UAS integration into the NAS: an examination of baseline compliance in the current airspace system

As a result of the FAA Modernization and Reform Act of 2012, Unmanned Aerial Systems (UAS) are expected to be integrated into the National Airspace System (NAS) by 2015. Several human factors challenges need to be addressed before UAS can safely and routinely fly in the NAS with manned aircraft. Perhaps the most significant challenge is for the UAS to be non-disruptive to the air traffic management system. Another human factors challenge is how to provide UAS pilots with intuitive traffic information in order to support situation awareness (SA) of their airspace environment as well as a see-and-avoid capability comparable to manned aircraft so that a UAS pilot could safely maneuver the aircraft to maintain separation and collision avoidance if necessary. A simulation experiment was conducted to examine baseline compliance of UAS operations in the current airspace system. Researchers also examined the effects of introducing a Cockpit Situation Display (CSD) into a UAS Ground Control Station (GCS) on UAS pilot performance, workload and situation awareness while flying in a positively controlled sector. Pilots were tasked with conducting a highway patrol police mission with a Medium Altitude Long Endurance (MALE) UAS in L.A. Center airspace with two mission objectives: 1) to reroute the UAS when issued new instructions from their commander, and 2) to communicate with Air Traffic Control (ATC) to negotiate flight plan changes and respond to vectoring and altitude change instructions. Objective aircraft separation data, workload ratings, SA data, and subjective ratings regarding UAS operations in the NAS were collected. Results indicate that UAS pilots were able to comply appropriately with ATC instructions. In addition, the introduction of the CSD improved pilot SA and reduced workload associated with UAS and ATC interactions.

UAS Contingency Management: The Effect of Different Procedures on ATC in Civil Airspace Operations

UAS currently lack key capabilities required to routinely integrate with the current Air Traffic Management (ATM) system, including standardized and predictable procedures for managing off nominal or contingency events, especially those that are specifically related to UAS and their unique communications architecture [i.e., loss of the command and control communications link(s). A simulation experiment was conducted to examine the effects of a variety of currently-employed UAS contingency procedures on sector safety and efficiency, and Air Traffic Controller (ATC) workload. ATC participants were tasked with maintaining safe separation standards in a busy Terminal Radar Approach Control (TRACON) sector that included a single UAS. During different trials, the UAS would execute one of five contingency types, including one trial with no contingency (i.e., baseline), three different contingency procedures for the loss of command and control link, and one emergency landing procedure. Objective aircraft separation and sector throughput data, workload ratings, situation awareness ratings, and subjective ratings regarding the safety and efficiency of UAS operations in the NAS were collected. Results indicated that the simulated UAS contingency procedures had no significant impact on objective measures of safety and efficiency compared to the baseline. Further, there were no significant differences in subjective workload and situation awareness ratings between the baseline and any of the contingency procedures.

Measured Response for Multiple UAS in a Simulated NAS Environment

The U.S. Congress passed the FAA Modernization and Reform Act in 2012, which calls for a plan to integrate Unmanned Aircraft Systems (UAS) into the National Airspace System (NAS). For UAS to be allowed to operate in the NAS, they will be required to “act and respond as manned aircraft do.” FAA regulations require that pilots respond promptly to air traffic controller (ATCo) commands. However, they do not quantify what an acceptable latency is. We maintain that research needs to be carried out to identify an acceptable latency in UAS responding, known as the “measured response.” While the measured response can be broken down into several components, the present paper provides an overview of how latencies in verbal communication and command execution can impact Pilot-ATCo communications and acceptability ratings when single and multiple UAS are operating in a simulated NAS environment.

Crew Resource Management for Automated Teammates (CRM-A).

Crew Resource Management (CRM) is the application of human factors knowledge and skills to ensure that teams make effective use of all resources. This includes ensuring that pilots bring in opinions of other teammates and utilize their unique capabilities. CRM was originally developed 40 years ago in response to a number of airline accidents in which the crew was found to be at fault. The goal was to improve teamwork among airline cockpit crews. The notion of “team” was later expanded to include cabin crew and ground resources. CRM has also been adopted by other industries, most notably medicine. Automation research now finds itself faced with similar issues to those faced by aviation 40 years ago: how to create a more robust system by making full use of both the automation and its human operators. With advances in machine intelligence, processing speed and cheap and plentiful memory, automation has advanced to the point that it can and should be treated as a teammate to fully take advantage of its capabilities and contributions to the system. This area of research is known as Human-Autonomy Teaming (HAT). Research on HAT has identified reusable patterns that can be applied in a wide range of applications. These patterns include features such as bi-directional communication and working agreements. This paper will explore the synergies between CRM and HAT. We believe that HAT research has much to learn from CRM and that there are benefits to expanding CRM to cover automation.

Breaking Down Barriers to UAS Integration What are government agencies doing

This symposium brings together researchers from NASA, the FAA, DoD and academia who have been investigating human factors barriers to UAS integration into the NAS. Participants will describe NASA’s five year effort, work on defining minimum information elements for detect and avoid, acceptable levels of automation, acceptable time for UAS pilots to respond to ATC and DoD efforts on air born sense and avoid.