Miwa Hayashi

Hidden Markov model analysis for space shuttle crewmembers' scanning behavior

The paper describes the application of a novel analysis technique employing hidden Markov models (HMMs) to analyze complex sequences of visual fixations and understand the supervisory monitoring strategies of space shuttle cockpit crewmembers. While HMM analysis has been shown to be useful for understanding the scanning behavior of air craft pilots engaging in manual flight control, it has not yet been applied to the supervisory monitoring context. The paper presents a proof-of-concept demonstration by applying HMM analysis to the eye-movement data of a veteran Commander astronaut performing simulated ascent-phase operations. The paper also demonstrates how the resulting HMM parameters can be used to detect deviations from regular scan patterns.

CHARACTERIZING SCAN PATTERNS IN A SPACECRAFT COCKPIT SIMULATOR: EXPERT VS. NOVICE PERFORMANCE

Operating a spacecraft is a complex and demanding task that requires years of training and constant monitoring of both navigation and systems parameters. By examining differences in scanning between “expert” and “novice” operators, we can develop cognitive models of scanning behavior or enhance training. In the Intelligent Spacecraft Interface Systems (ISIS) laboratory, we measure eye movements and record performance parameters in a part-task space shuttle cockpit simulator. We trained airline transport pilots (as our “novice” group) on fundamentals of flying an ascent (”launch-to-orbit”) in the space shuttle. We tested three levels of malfunctions occurring during a trial—none (nominal), one malfunction, or three malfunctions—on both pilots and astronauts (our “expert” group). Astronauts had fewer errors and faster reaction times. Eye movement analyses showed that both astronauts and pilots similarly modified their scan strategies depending on the flight segment and how many malfunctions occurred during a trial.

Computer vision based surveillance concept for airport ramp operations

Current research develops a vision-based surveillance system concept suitable for airport ramp area operations. The surveillance approach consists of computer vision algorithms operating on video streams from surveillance cameras for detecting aircraft in images and localizing them. Rough order of magnitude estimates of the number of cameras required to cover the ramp area at a sample airport (Dallas/Fort Worth International Airport) were obtained. Two sets of algorithms with complimentary features were developed to detect an aircraft in a given image. The first set of algorithms was based on background subtraction, a popular computer-vision approach, for change detection in video streams. The second set was a supervised-learning approach based on a model learned from a database of images. The Histogram of Oriented Gradient (HOG) feature was used for classification with Support Vector Machines (SVMs). Then, an algorithm for matching aircraft in two different images was developed based on an approximate aircraft localization algorithm. Finally, stereo-vision algorithms were used for 3D-localization of the aircraft. A 1:400 scale model of a realistic airport consisting of a terminal building, jet bridges, ground marking, aircraft, and ground vehicles was used for testing the various algorithms. Aircraft detection was demonstrated using static and moving aircraft images, single and multiple aircraft images, and occluded aircraft images. Preliminary testing using the in-house setup demonstrated 3D localization accuracy of up to 30 ft.

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Flights often experience large delays when they are routed around weather. Multi-flight common route advisories provide time-saving re-routes for groups of flights whose current weather-avoidance routes have become outdated because the weather has dissipated and/or moved away. A laboratory evaluation of these advisories was conducted by five subject matter experts having extensive experience in traffic flow management operations. These experts provided a total of 200 data sets in the airspace of Fort Worth Center and Houston Center. The tool identifies time-saving route changes taking into account flight plans, wind fields, and the spatial-temporal evolution of predicted convective weather. The tool is not designed to account for complex operational factors such as non-standard sector traversal, and hence a relatively low percentage of advisories initially generated by the tool were rated as acceptable (55% for Fort Worth Center and 22% for Houston Center). However, a high percentage of advisories were rated as acceptable (86% for Fort Worth Center and 75% for Houston Center) after the subject matter experts used the tool’s user interface to make route modifications that accounted for relevant operational factors. The workload associated with using the tool was rated as low for 80% of Fort Worth Center advisories and 56% of Houston Center advisories. On average, Houston Center advisories provided larger time savings but were more complex, hence they were more challenging to evaluate and received less favorable ratings. The results of this evaluation make a good case for human-automation teaming to design weather re-routes for delay recovery.

Operator Performance Evaluation of Fault Management Interfaces for Next-Generation Spacecraft

In the cockpit of the NASAs next generation of spacecraft, most of vehicle commanding will be carried out via electronic interfaces instead of hard cockpit switches. Checklists will be also displayed and completed on electronic procedure viewers rather than from paper. Transitioning to electronic cockpit interfaces opens up opportunities for more automated assistance, including automated root-cause diagnosis capability. The paper reports an empirical study evaluating two potential concepts for fault management interfaces incorporating two different levels of automation. The operator performance benefits produced by automation were assessed. Also, some design recommendations for spacecraft fault management interfaces are discussed.

Human-in-the-Loop Evaluation of Dynamic Multi-Flight Common Route Advisories

Flights often experience large delays when they are routed around weather. Multi-flight common route advisories provide delay recovery by suggesting time-saving re-routes for groups of flights whose current weather-avoidance routes have become outdated because the weather has dissipated and/or moved away. A laboratory evaluation of these advisories was conducted by four subject matter experts having extensive experience in traffic flow management operations. These experts provided a total of 120 data points in the airspace of Houston Center. The multi-flight common route tool provides time-saving route change advisories taking into account flight plans, wind fields, and the spatio-temporal evolution of predicted convective weather. It is not designed to account for complex operational factors such as non-standard sector traversal and interactions with local traffic management initiatives; hence a relatively low percentage (37%) of advisories generated by the tool were rated as acceptable. However, a high percentage (81%) of advisories were rated as acceptable after the subject matter experts used the tool’s user interface to make route modifications that accounted for relevant operational factors not considered by the tool. The workload associated with using the tool, as measured by the NASA Task Load Index, was quite low (1.1 on a scale of 0 to 10). The results of this evaluation make a good case for human-automation teaming to design operationally valid weather re-routes for delay recovery.

Laboratory Evaluation of Dynamic Routing of Air Traffic in an En Route Arrival Metering Environment [STUB]

Arrival air traffic operations in the presence of convective weather are subject to uncertainty in aircraft routing and subsequently in flight trajectory predictability. Current management of arrival operations in weather-impacted airspace results in significant flight delay and suspension of arrival metering operations. The Dynamic Routing for Arrivals in Weather (DRAW) concept provides flight route amendment advisories to Traffic Management Coordinators to mitigate the impacts of convective weather on arrival operations. DRAW provides both weather conflict and schedule information for proposed route amendments, allowing air traffic managers to simultaneously evaluate weather avoidance routing and potential schedule and delay impacts. Subject matter experts consisting of retired Traffic Management Coordinators and retired Sector Controllers with arrival metering experience participated in a simulation study of Fort Worth Air Route Traffic Control Center arrival operations. Data were collected for Traffic Management Coordinator and Sector Controller participants over three weeks of simulation activities in October, 2017. Traffic Management Coordinators reported acceptable workload levels, a positive impact on their ability to manage arrival traffic while using DRAW, and initiated weather mitigation reroutes earlier while using DRAW. Sector Controllers also reported acceptable workload levels while using DRAW.

Cardiac-Activity Measures for Assessing Airport Ramp-Tower Controller's Workload

Heart rate (HR) and heart rate variability (HRV) potentially offer objective, continuous, and non-intrusive measures of human-operator’s mental workload. Such measurement capability is attractive for workload assessment in complex laboratory simulations or safety-critical field testing. The present study compares mean HR and HRV data with self-reported subjective workload ratings collected during a high-fidelity human-in-the-loop simulation of airport ramp traffic control operations, which involve complex cognitive and coordination tasks. Mean HR was found to be weakly sensitive to the workload ratings, while HRV was not sensitive or even contradictory to the assumptions. Until more knowledge on stress response mechanisms of the autonomic nervous system is obtained, it is recommended that these cardiac-activity measures be used with other workload assessment tools, such as subjective measures.