Christopher D. Cabrall

Comparison of Ground-Based and Airborne Function Allocation Concepts for NextGen Using Human-In-The-Loop Simulations

Investigation of function allocation for the Next Generation Air Transportation System is being conducted by the National Aeronautics and Space Administration (NASA). To provide insight on comparability of different function allocations for separation assurance, two human-in-the-loop simulation experiments were conducted on homogeneous airborne and ground-based approaches to four-dimensional trajectory-based operations, one referred to as ground-based automated separation assurance (groundbased) and the other as airborne trajectory management with self-separation (airborne). In the coordinated simulations at NASA s Ames and Langley Research Centers, controllers for the ground-based concept at Ames and pilots for the airborne concept at Langley managed the same traffic scenarios using the two different concepts. The common scenarios represented a significant increase in airspace demand over current operations. Using common independent variables, the simulations varied traffic density, scheduling constraints, and the timing of trajectory change events. Common metrics were collected to enable a comparison of relevant results. Where comparisons were possible, no substantial differences in performance or operator acceptability were observed. Mean schedule conformance and flight path deviation were considered adequate for both approaches. Conflict detection warning times and resolution times were mostly adequate, but certain conflict situations were detected too late to be resolved in a timely manner. This led to some situations in which safety was compromised and/or workload was rated as being unacceptable in both experiments. Operators acknowledged these issues in their responses and ratings but gave generally positive assessments of the respective concept and operations they experienced. Future studies will evaluate technical improvements and procedural enhancements to achieve the required level of safety and acceptability and will investigate the integration of airborne and ground-based capabilities within the same airspace to leverage the benefits of each concept.

An Overview of Current Capabilities and Research Activities in the Airspace Operations Laboratory at NASA Ames Research Center

The Airspace Operations Laboratory at NASA Ames conducts research to provide a better understanding of roles, responsibilities, and requirements for human operators and automation in future air traffic management (ATM) systems. The research encompasses developing, evaluating, and integrating operational concepts and technologies for near-, mid-, and far-term air traffic operations. Current research threads include efficient arrival operations, function allocation in separation assurance and efficient airspace and trajectory management. The AOL has developed powerful air traffic simulation capabilities, most notably the Multi Aircraft Control System (MACS) that is used for many air traffic control simulations at NASA and its partners in government, academia and industry. Several additional NASA technologies have been integrated with the AOLs primary simulation capabilities where appropriate. Using this environment, large and small-scale system-level evaluations can be conducted to help make near-term improvements and transition NASA technologies to the FAA, such as the technologies developed under NASA’s Air Traffic Management Demonstration-1 (ATD-1). The AOL’s rapid prototyping and flexible simulation capabilities have proven a highly effective environment to progress the initiation of trajectory-based operations and support the mid-term implementation of NextGen. Fundamental questions about accuracy requirements have been investigated as well as realworld problems on how to improve operations in some of the most complex airspaces in the US. This includes using advanced trajectory-based operations and prototype tools for coordinating arrivals to converging runways at Newark airport and coordinating departures and arrivals in the San Francisco and the New York metro areas. Looking beyond NextGen, the AOL has started exploring hybrid human/automation control strategies as well as highly autonomous operations in the air traffic control domain. Initial results indicate improved capacity, low operator workload, good situation awareness and acceptability for controllers teaming with autonomous air traffic systems. While much research and development needs to be conducted to make such concepts a reality, these approaches have the potential to truly transform the airspace system towards increased mobility, safe and efficient growth in global operations and enabling many of the new vehicles and operations that are expected over the next decades. This paper describes how the AOL currently contributes to the ongoing air transportation transformation.

Initial Investigations of Controller Tools and Procedures for Schedule-Based Arrival Operations with Mixed Flight-Deck Interval Management Equipage

National Aeronautics and Space Administration (NASA) Air Traffic Management Demonstration-1 (ATD-1) is a multi-year effort to demonstrate high-throughput, fuel-efficient arrivals at a major United States airport using NASA-developed scheduling automation, controller decision-support tools, and Automatic Dependent Surveillance–Broadcast (ADS-B)-enabled Flight-Deck Interval Management (FIM) avionics. First-year accomplishments include the development of a concept of operations for managing scheduled arrivals flying Optimized Profile Descents with equipped aircraft conducting FIM operations, and the integration of laboratory prototypes of the core ATD-1 technologies. Following each integration phase, a human-in-the-loop simulation was conducted to evaluate and refine controller tools, procedures, and clearance phraseology. From a ground-side perspective, the results indicate the concept is viable and the operations are safe and acceptable. Additional training is required for smooth operations that yield notable benefits, particularly in the areas of FIM operations and clearance phraseology.

Analysis of Interactive Conflict Resolution Tool Usage in a Mixed Equipage Environment

A human-in-the-loop simulation was conducted that examined separation assurance concepts in varying levels of traffic density with mixtures of aircraft equipage and automation. This papers analysis focuses on one of the experimental conditions in which traffic levels were approximately fifty percent higher than today, and approximately fifty percent of the traffic within the test area were equipped with data communications (data comm) capabilities. The other fifty percent of the aircraft required control by voice much like today. Within this environment, the air traffic controller participants were provided access to tools and automation designed to support the primary task of separation assurance that are currently unavailable. Two tools were selected for analysis in this paper: 1) a pre-probed altitude fly-out menu that provided instant feedback of conflict probe results for a range of altitudes, and 2) an interactive auto resolver that provided on-demand access to an automation-generated conflict resolution trajectory. Although encouraged, use of the support tools was not required; the participants were free to use the tools as they saw fit, and they were also free to accept, reject, or modify the resolutions offered by the automation. This mode of interaction provided a unique opportunity to examine exactly when and how these tools were used, as well as how acceptable the resolutions were. Results showed that the participants used the pre-probed altitude fly-out menu in 14% of conflict cases and preferred to use it in a strategic timeframe on data comm equipped and level flight aircraft. The interactive auto resolver was also used in a primarily strategic timeframe on 22% of conflicts and that their preference was to use it on conflicts involving data comm equipped aircraft as well. Of the 258 resolutions displayed, 46% were implemented and 54% were not. The auto resolver was rated highly by participants in terms of confidence and preference. Factors such as aircraft equipage, ownership, and location of predicted separation loss appeared to play a role in the decision of controllers to accept or reject the auto resolvers resolutions.

Evaluation of High Density Air Traffic Operations with Automation for Separation Assurance, Weather Avoidance and Schedule Conformance

In this paper we discuss the development and evaluation of our prototype technologies and procedures for far-term air traffic control operations with automation for separation assurance, weather avoidance and schedule conformance. Controller-in-the-loop simulations in the Airspace Operations Laboratory at the NASA Ames Research Center in 2010 have shown very promising results. We found the operations to provide high airspace throughput, excellent efficiency and schedule conformance. The simulation also highlighted areas for improvements: Short-term conflict situations sometimes resulted in separation violations, particularly for transitioning aircraft in complex traffic flows. The combination of heavy metering and growing weather resulted in an increased number of aircraft penetrating convective weather cells. To address these shortcomings technologies and procedures have been improved and the operations are being re-evaluated with the same scenarios. In this paper we will first describe the concept and technologies for automating separation assurance, weather avoidance, and schedule conformance. Second, the results from the 2010 simulation will be reviewed. We report human-systems integration aspects, safety and efficiency results as well as airspace throughput, workload, and operational acceptability. Next, improvements will be discussed that were made to address identified shortcomings. We conclude that, with further refinements, air traffic control operations with ground-based automated separation assurance can routinely provide currently unachievable levels of traffic throughput in the en route airspace.

Validity and reliability of naturalistic driving scene categorization Judgments from crowdsourcing

A common challenge with processing naturalistic driving data is that humans may need to categorize great volumes of recorded visual information. By means of the online platform CrowdFlower, we investigated the potential of crowdsourcing to categorize driving scene features (i.e., presence of other road users, straight road segments, etc.) at greater scale than a single person or a small team of researchers would be capable of. In total, 200 workers from 46 different countries participated in 1.5days. Validity and reliability were examined, both with and without embedding researcher generated control questions via the CrowdFlower mechanism known as Gold Test Questions (GTQs). By employing GTQs, we found significantly more valid (accurate) and reliable (consistent) identification of driving scene items from external workers. Specifically, at a small scale CrowdFlower Job of 48 three-second video segments, an accuracy (i.e., relative to the ratings of a confederate researcher) of 91% on items was found with GTQs compared to 78% without. A difference in bias was found, where without GTQs, external workers returned more false positives than with GTQs. At a larger scale CrowdFlower Job making exclusive use of GTQs, 12,862 three-second video segments were released for annotation. Infeasible (and self-defeating) to check the accuracy of each at this scale, a random subset of 1012 categorizations was validated and returned similar levels of accuracy (95%). In the small scale Job, where full video segments were repeated in triplicate, the percentage of unanimous agreement on the items was found significantly more consistent when using GTQs (90%) than without them (65%). Additionally, in the larger scale Job (where a single second of a video segment was overlapped by ratings of three sequentially neighboring segments), a mean unanimity of 94% was obtained with validated-as-correct ratings and 91% with non-validated ratings. Because the video segments overlapped in full for the small scale Job, and in part for the larger scale Job, it should be noted that such reliability reported here may not be directly comparable. Nonetheless, such results are both indicative of high levels of obtained rating reliability. Overall, our results provide compelling evidence for CrowdFlower, via use of GTQs, being able to yield more accurate and consistent crowdsourced categorizations of naturalistic driving scene contents than when used without such a control mechanism. Such annotations in such short periods of time present a potentially powerful resource in driving research and driving automation development.

Designing Scenarios for Controller-in-the-Loop Air Traffic Simulations

Well prepared traffic scenarios contribute greatly to the success of controller-in-the-loop simulations. This paper describes each stage in the design process of realistic scenarios based on real-world traffic, to be used in the Airspace Operations Laboratory for simulations within the Air Traffic Management Technology Demonstration 1 effort. The steps from the initial analysis of real-world traffic, to the editing of individual aircraft records in the scenario file, until the final testing of the scenarios before the simulation conduct, are all described. The iterative nature of the design process and the various efforts necessary to reach the required fidelity, as well as the applied design strategies, challenges, and tools used during this process are also discussed.

Design and Evaluation of a Corridors-in-the-sky Concept: The Benefits and Feasibility of Adding Highly Structured Routes to a Mixed Equipage Environment

A human-in-the-loop simulation of a Corridors-in-the-sky concept was conducted that focused on investigating the potential benefits and feasibility of the concept with human operators in a realistic environment. In this simulation, the definition of “corridors” was changed from meaning separate corridor airspace with dedicated corridor controllers to highly structured routes with potentially common speeds and avionics equipage. The change in definition allowed the concept to be realizable within the mid-term Next Generation Air Transportation System (NextGen) timeframe and mitigated many of the feasibility issues that were identified in earlier research. Feasibility and benefits were tested through the variance of traffic levels within the test airspace (High and Max). In the High Traffic condition, the radar (R-side) and supporting data (D-side) controllers managed a high level of traffic without aircraft being rerouted out of the congested sectors. In the Max Traffic condition, a traffic flow manager and area supervisors worked together to reroute aircraft out of the congested sectors. Four different procedures were tested with different corridor structures: (1) no specific corridors (No Corridors), (2) only equipped aircraft within corridors (Equipped in Corridors), (3) only unequipped aircraft within corridors (Unequipped in Corridors), and (4) a mix of both equipped and unequipped aircraft within corridors (Mixed in Corridors). Surrounding non-corridor traffic consisted of a 50/50 mix of Data Comm and non-Data Comm equipped aircraft in all conditions. The results of the study indicate that the Equipped in Corridors condition showed the greatest benefits with the highest levels of throughput and the lowest reported workload relative to the other conditions. In contrast, the Unequipped in Corridors condition showed little throughput or workload benefits relative to the No Corridors condition. The results for the Mixed in Corridors condition fell in between the values observed for Equipped in Corridors and No Corridors. Feedback from the participants revealed that the observed reduction in benefits when unequipped aircraft were in the corridors was a result of the workload associated with the communications and monitoring required for the unequipped corridor aircraft as well as the display clutter of their data blocks. In addition, the study showed that the concept was feasible and was well received by the participants. Service for equipage was also shown to be feasible with fewer Data Comm equipped aircraft rerouted than non-Data Comm equipped aircraft.

The 4D LINT Model of Function Allocation: Spatial-Temporal Arrangement and Levels of Automation

Human factors researchers are well familiar with Sheridan and Verplank’s (1978) ‘levels of automation’. Although this automation dimension has proved useful, the last decade has seen a vast increase of automation in different forms, especially in transportation domains. To capture these and future developments, we propose an extended automation taxonomy via additional dimensions. Specifically, we propose a 4D LINT representation for vehicle operation regarding control across multiple simultaneous dimensions of (1) Location (from local to remote), (2) Identity (between human and computer), (3) Number of agents (degree of centralization of control), as well as (4) adaptive optimization over Time. Our model aims to provide guidance and support in communicable ways to allocation authority agents (whether human or computer) in optimized supervisory outer loop control of complex and intelligent dynamic systems for more efficient, safe, and robust transportation operations.

Current Insights in Human Factors of Automated Driving and Future Outlook Towards Tele-Operated Remote Driving Services

Across the automotive industry, manufacturers have recently released various Partial Automation systems (SAE Level 2) which allow simultaneous/combined execution of both lateral and longitudinal vehicle control at the same time, yet still require active human supervision/engagement. Current reactive trends will be reviewed across major automotive players regarding differences in terminology, HMI input/outputs, and escalation intervals. Scholarly research is also reviewed pertaining to proactive strategies for driver engagement. Additionally, human factors research and findings will be presented regarding recommendations for situation awareness, human machine interfaces, TOR, as well as shared control concepts. The tutorial will conclude with discussion and brainstorming around outlook toward tele-operated remote driving services (Tele-Driving); what they have to offer beyond assisted/automated driving, autonomous vehicles, and ride-hailing/car-sharing paradigms; as well as the design/conduct of human factors research regarding Tele-Driving.