Lynne Martin

Evaluation of the Terminal Sequencing and Spacing system for Performance-Based Navigation arrivals

NASA has developed the Terminal Sequencing and Spacing (TSS) system, a suite of advanced arrival management technologies combining time-based scheduling and controller precision spacing tools. TSS is a ground-based controller automation tool that facilitates sequencing and merging arrivals that have both current standard ATC routes and terminal Performance-Based Navigation (PBN) routes, especially during highly congested demand periods. In collaboration with the FAA and MITREs Center for Advanced Aviation System Development (CAASD), TSS system performance was evaluated in human-in-the-loop (HITL) simulations with currently active controllers as participants. Traffic scenarios had mixed Area Navigation (RNAV) and Required Navigation Performance (RNP) equipage, where the more advanced RNP-equipped aircraft had preferential treatment with a shorter approach option. Simulation results indicate the TSS system achieved benefits by enabling PBN, while maintaining high throughput rates-10% above baseline demand levels. Flight path predictability improved, where path deviation was reduced by 2 NM on average and variance in the downwind leg length was 75% less. Arrivals flew more fuel-efficient descents for longer, spending an average of 39 seconds less in step-down level altitude segments. Self-reported controller workload was reduced, with statistically significant differences at the p<;0.01 level. The RNP-equipped arrivals were also able to more frequently capitalize on the benefits of being “Best-Equipped, Best-Served” (BEBS), where less vectoring was needed and nearly all RNP approaches were conducted without interruption.

Investigating the impact of off-nominal events on high-density ‘green’ arrivals

Trajectory-based controller tools developed to support a schedule-based terminal-area air traffic management (ATM) concept have been shown effective for enabling ‘green’ arrivals along Area Navigation (RNAV) routes in moderately high-density traffic conditions. A recent human-in-the-loop simulation investigated the robustness of the concept and tools to off-nominal events-events that lead to situations in which runway arrival schedules require adjustments and controllers can no longer use speed control alone to impose the necessary delays. Study participants included a terminal-area Traffic Management Supervisor responsible for adjusting the schedules. Sector-controller participants could issue alternate RNAV transition routes to absorb large delays. The study also included real-time winds/wind-forecast changes. The results indicate that arrival spacing accuracy, schedule conformance, and tool usage and usefulness are similar to that observed in simulations of nominal operations. However, the time and effort required to recover from an off-nominal event is highly context-sensitive, and impacted by the required schedule adjustments and control methods available for managing the evolving situation. The research suggests ways to bolster the off-nominal recovery process, and highlights challenges related to using human-in-the-loop simulation to investigate the safety and robustness of advanced ATM concepts.

Development and evaluation of the terminal precision scheduling and spacing system for off-nominal condition operations

NASA has developed a capability for terminal area precision scheduling and spacing (TAPSS) to increase airport throughput and the use of fuel-efficient arrival procedures during periods of peak traffic congestion at high-throughput airports. This advanced technology represents NASAs current concept for the NextGen terminal metering desired capability. A series of high-fidelity human-in-the-loop simulation experiments were conducted to evaluate the performance of the TAPSS system during off-nominal conditions, specifically aircraft executing missed-approach and go-around procedures after transitioning to the final approach fix during an attempted landing. Each simulation run contained 2-4 missed approaches during a highly congested 60-minute period. The TAPSS system was adapted to arrival operations for the Los Angeles International airport (LAX). It was also enhanced to support automated missed-approach processing and procedures. The experiments evaluated the utility of the missed approach enhanced automation features by comparing system performance and controller workload with and without the enhancements. The simulated traffic throughput exceeded that of the current LAX operations with two runways in instrument meteorological conditions (IMC) by 10%. The results showed that when using the enhanced automation, the controllers could maintain the higher throughput levels with more consistent and predictable routing in the final operations but with increased vectoring and off-route aircraft in the feeder positions. Controller workload results indicated a preference for the automation enhancement especially as the numbers of missed approaches increased from 2 to 4 during the 60-minute evaluation period.

Human-automation cooperation for separation assurance in future NextGen environments

A 2012 Human-In-The-Loop air traffic control simulation investigated a gradual paradigm-shift in the allocation of functions between operators and automation. Air traffic controllers staffed five adjacent high-altitude en route sectors and, during the course of a two-week experiment, worked traffic under different function-allocation approaches aligned with four increasingly mature NextGen operational environments. These NextGen time-frames ranged from near current-day operations to nearly fully-automated control in which the ground systems automation was responsible for detecting conflicts, issuing strategic and tactical resolutions, and alerting the controller to exceptional circumstances. Results indicate that overall performance was best in the most automated NextGen environment. Safe operations were achieved in this environment for twice todays peak airspace capacity, while being rated by the controllers as highly acceptable. However, results show that sector operations were not always safe; separation violations did in fact occur. This paper will describe in detail the simulation conducted, as well discuss important results and their implications.

Evaluation of the Terminal Area Precision Scheduling and Spacing system for Performance-Based Navigation arrivals

In 2012, NASA and FAA jointly conducted a human-in-the-loop air traffic simulation to evaluate the utility of the Terminal Area Precision Scheduling and Spacing (TAPSS) system for supporting Performance-Based Navigation arrival operations during periods of congestion at a mid-sized airport. The TAPSS system is a trajectory-based strategic planning and tactical control tool that was developed to efficiently manage arrivals. For this study, the TAPSS system was enhanced to handle Required Navigation Performance arrivals. A baseline case, where none of the TAPSS systems advisories were provided, was run along with two different configurations of the TAPSS system with differing sets of controller advisory tools. The engineering data indicate that the TAPSS system has a potential to enable efficient Performance-Based Navigation arrival operations. The participating controllers found the TAPSS systems advisories useful. When controllers were given the full set of TAPSS advisory tools, 90% of Required Navigation Performance arrivals stayed on-path as compared to 87% in the baseline case, the average extra track distance of Area Navigation arrivals decreased by 36%, and the average number of controller voice communications decreased by 13%.

Acceptability and effects of tools to assist with controller managed spacing in the terminal area

In a human-in-the-loop simulation, a scheduler delivered aircraft to meter fixes in the Los Angeles terminal area with a -60 to +30 second accuracy. This study investigated whether, and how well, controllers could control aircraft to land them as close to their scheduled time of arrival (STA) as possible using speed control alone. Controllers were assigned one of three levels of tools to assist them but had to compensate for errors in the forecast winds that had not been taken into account by the scheduler. Results show that speed clearances were sufficient under all conditions to maneuver aircraft closer to their STAs. From participant reports, this form of control incurred manageable workload and two of the three levels of tools were deemed easy to use.

Exploring workload factors across future environments

A human-in-the-loop simulation was conducted that examined separation assurance across four progressive future time frames. Decision support, traffic density, separation assurance roles and responsibilities, and aircraft equipage mix were varied across conditions. In a near-term condition, these factors were set to approximate current day operations. In contrast, the most far-term condition involved two times current traffic, full air-ground data communications equipage, and automated conflict resolution working independently. The variation across the four conditions provided an opportunity to explore the pattern of reported controller workload, and what factors contributed to any observed differences. Despite increasing levels of traffic, results showed that mean workload ratings did not differ across conditions with the exception of the furthest term condition which was significantly lower. However, additional analyses were conducted that examined the relationship between workload and the varying traffic characteristics per condition. Although each condition had different significant contributors to workload, the one consistent contributor to workload across each condition was the number of conflicts. This result highlights the importance of work being done to develop the concepts and automation necessary to progressively balance the allocation of separation assurance functions between automation and the air traffic controller of the future.

Flying schedule-matching descents to explore flight crews' perceptions of their load and task feasibility

Recent studies at NASA Ames Research Center have investigated the development and use of ground-based (air traffic controller) tools to manage and schedule air traffic in future terminal airspace. No complementary studies have investigated the impacts that such tools (and concepts) could have on the flight-deck (although some independent studies have researched impacts of other ground tools). To begin to redress the balance within the project, an exploratory study investigated the procedures and actions of ten Boeing-747–400 crews as they flew eight continuous descent approaches in the Los Angeles terminal airspace, with the descents being controlled using speed alone. Although the study was exploratory in nature, four variables were manipulated: speed changes, route constraints, clearance phraseology, and winds. Despite flying the same scenarios with the same events and timing, there was at least a 50 second difference in the time it took crews to fly the approaches. This variation is the product of a number of factors but highlights potential difficulties for scheduling tools that would have to accommodate this amount of natural variation in descent times. The primary focus of this paper is the potential impact of ground scheduling tools on the flight crews performance and procedures. Crews reported “moderate to low” workload, on average; however, short periods of intense and high workload were observed. The non-flying pilot often reported a higher level of workload than the flying-pilot, which may be due to their increased interaction with the Flight Management Computer, when using the aircraft automation to assist with managing the descent clearances. It is concluded that ground-side tools and automation may have a larger impact on the current-day flight-deck than was assumed and that studies investigating this impact should continue in parallel with controller support tool development.

NextGen Technologies on the FAA's Standard Terminal Automation Replacement System

This paper describes the integration, evaluation, and results from a high-fidelity human-in-the-loop (HITL) simulation of key NASA Air Traffic Management Technology Demonstration - 1 (ATD-1) technologies implemented in an enhanced version of the FAAs Standard Terminal Automation Replacement System (STARS) platform. These ATD-1 technologies include: (1) a NASA enhanced version of the FAAs Time-Based Flow Management, (2) a NASA ground-based automation technology known as controller-managed spacing (CMS), and (3) a NASA advanced avionics airborne technology known as flight-deck interval-management (FIM). These ATD-1 technologies have been extensively tested in large-scale HITL simulations using general-purpose workstations to study air transportation technologies. These general-purpose workstations perform multiple functions and are collectively referred to as the Multi-Aircraft Control System (MACS). Researchers at NASA Ames Research Center and Raytheon collaborated to augment the STARS platform by including CMS and FIM advisory tools to validate the feasibility of integrating these automation enhancements into the current FAA automation infrastructure. NASA Ames acquired three STARS terminal controller workstations, and then integrated the ATD-1 technologies. HITL simulations were conducted to evaluate the ATD-1 technologies when using the STARS platform. These results were compared with the results obtained when the ATD-1 technologies were tested in the MACS environment. Results collected from the numerical data show acceptably minor differences, and, together with the subjective controller questionnaires showing a trend towards preferring STARS, validate the ATD-1/STARS integration.

Adapting a Human – Automation Trust Scale to an Air Traffic Management Environment

In response to the demand of future air traffic environments potentially exceeding human operator capabilities, the process of integrating automated decision-making tools into the air traffic management system is underway. However, the current system is lacking a validated standard for measuring a critical element of the human-automation partnership trust. Specifically needed is a valid scale appropriate for and tested on air traffic controllers. To remedy this issue, a two-phase modification of the Human Automation Trust Scale by Jian, Bisantz and Drury (2000) was deployed at the Airspace Operations Laboratory at NASA Ames during 2013. Applied to two different human-in-the-loop experiments, the following results include a scale that supports understanding underlying trust attitude in air traffic controllers while maintaining a high inter-item reliability score. The use of this assessment method when testing new air traffic management tools can assist in understanding potential pitfalls for tool use and implementation. INTRODUCTION The next generation of air traffic control in the United States (NextGen) is evolving into an integrated humanautomation environment, with automation generated information becoming a critical contributor to decisionmaking (Joint Planning and Development Office, 2012). As this integration continues, it becomes increasingly necessary to appropriately assess the human-automation relationship as it pertains to decision-making and safety critical tasks. Underlying trust is one component that has been identified as a key (though not sole) contributor to intent to use and actual usage of an automated system (Lee & Moray, 1994). As such, developing a reliable method for measuring underlying trust is necessary in the assessment of human-automation integration. In the current air traffic control domain where most systems are very safe and the goal is near-perfect performance i.e., delivering aircraft safely and on time, the general underlying trust attitude (Lee & See, 2004) of a controller in regards to an automated system is of as much concern as their ability to detect envrionmentally (such as an incorrect weather forecast) induced inaccuracies / unreliabilities requiring a change in automated tool use by a controller (Kirlik,1993). As such, measuring actual general underlying trust of controllers in an automated system necessitates a method which is not highly sensitive to direct experimental manipulation of envrionmental factors impacting the current accuracy of the automation, but instead responds to their underlying attitude, knowledge in and trust of the system. To this end, in 2013, researchers in the Airspace Operations Laboratory (AOL) at NASA Ames employed modified versions of the Human-Automation Trust Scale (HAT) (Jian, Bisantz, & Drury, 2000) during two human-inthe-loop simulations to assess its efficacy in an air traffic management (ATM) environment. HAT was selected specifically for its empirically based assessment method, and its track record of use in different automated domains (Montague, Kleiner, & Winchester 3rd, 2009; Wang, Jamiseon, & Hollands, 2009). Due to the simulations’ highly specific domain –ATM -, HAT was modified to address an air traffic management environment while retaining the same, or synonyms of, key words in the original scale. It was the concern of the experimenters that presenting the scale without modification would be inapplicable to this technology and population. Therefore, the first study conducted in January 2013 retained seven of the original twelve scale items. Study 2 in September 2013 used the findings from Study 1 and presented eight of the original twelve items. The adaptations and subsequent performance of the HAT scale in both experimental environments is the focus of this paper. METHODS A full report on the simulations cited in this paper can be found in Mercer et al. (2013) and Callantine, Hunt, & Prevot (2014) for Study 1 and Study 2, respectively. As Study 1 was conducted first, adaptations made to the HAT scale were further modified before use in Study 2. HAT’s presentation for both experiments involved: (a) a randomized array using an internet-based survey software suite , (b)a seven point Likert style scale, (c) identical scale anchor points – ‘not at all’ [1], ‘moderately’ [4], ‘very’ [4], and (d) the instructions “Please indicate your current thoughts about the automation for the following:”. All participants were retired air traffic controllers, both male and female, and over forty years of age. Experimenters chose two disparate studies of ATM tools in order to evaluate the usefulness of the modified HAT scale. Test Environments Study 1 Test Environment: Study 1’s test environment consisted of two en route sectors (one-high altitude and one low-altitude) run by two parallel teams simultaneously in the lab for four total participants. Based on historical traffic of the Atlanta airspace, traffic scenarios included flows of arrival aircraft feeding into the northwest meter-fix of Atlanta’s Terminal Radar Approach Control (TRACON). While the majority of the aircraft were Atlanta arrivals, the simulation also included several over-flights. The controller teams metered traffic with a delivery goal of +/20 seconds at the meter-fix. Confederate controllers staffed all necessary adjacent airspace. The aircraft simulated in this environment were equipped with Flight Management Systems (FMS) and Automatic Dependent Surveillance-Broadcast -out capabilities (ADS-B-out). Controllers issued all instructions to the pilots via voice communications. Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting 2014 26