Scott Galster

AIR TRAFFIC CONTROLLER PERFORMANCE AND WORKLOAD UNDER MATURE FREE FLIGHT: CONFLICT DETECTION AND RESOLUTION OF AIRCRAFT SELF-SEPARATION

The effects of conflict detection and self-separating aircraft resolution on the mental workload and performance of en-route air traffic controllers were examined. An air traffic control simulator was used to manipulate traffic loads and traffic complexity. A mature stage of free flight was simulated by having controllers monitor self-separating aircraft. Four 30-min scenarios were created to combine moderate (11 aircraft) and heavy traffic loads (17 aircraft) in a 50-mile radius sector with the presence or absence of self-separating and conflicting aircraft. Conflicts (defined as a loss of separation of 5 nm laterally and 1,000 ft vertically) were indicated to the controller by the appearance of a red circle around each of the aircraft involved. A self-separation event was defined as an evasive maneuver (either altitude or speed change) made by 1 aircraft to avoid a potential conflict with another aircraft. Performance and workload measurements indicated that controllers had difficulty both in detecting conflicts and in recognizing self-separating events in a timely manner in saturated airspace. Controller mental workload also increased, as indexed both by subjective and secondary task measures. Implications for the design of automated tools to support controllers under free flight environments are discussed.

Sensing, assessing, and augmenting threat detection: behavioral, neuroimaging, and brain stimulation evidence for the critical role of attention.

Rapidly identifying the potentially threatening movements of other people and objects—biological motion perception and action understanding—is critical to maintaining security in many civilian and military settings. A key approach to improving threat detection in these environments is to sense when less than ideal conditions exist for the human observer, assess that condition relative to an expected standard, and if necessary use tools to augment human performance. Action perception is typically viewed as a relatively “primitive,” automatic function immune to top-down effects. However, recent research shows that attention is a top-down factor that has a critical influence on the identification of threat-related targets. In this paper we show that detection of motion-based threats is attention sensitive when surveillance images are obscured by other movements, when they are visually degraded, when other stimuli or tasks compete for attention, or when low-probability threats must be watched for over long periods of time—all features typical of operational security settings. Neuroimaging studies reveal that action understanding recruits a distributed network of brain regions, including the superior temporal cortex, intraparietal cortex, and inferior frontal cortex. Within this network, attention modulates activation of the superior temporal sulcus (STS) and middle temporal gyrus. The dorsal frontoparietal network may provide the source of attention-modulation signals to action representation areas. Stimulation of this attention network should therefore enhance threat detection. We show that transcranial Direct Current Stimulation (tDCS) at 2 mA accelerates perceptual learning of participants performing a challenging threat-detection task. Together, cognitive, neuroimaging, and brain stimulation studies provide converging evidence for the critical role of attention in the detection and understanding of threat-related intentional actions.

Real-Time Workload Assessment as a Foundation for Human Performance Augmentation

While current military systems are functionally capable of adaptively aiding human operators, the effectiveness of this capability depends on the availability of timely, reliable assessments of operator states to determine when and how to augment effectively. This paper describes a response to the technical challenges associated with establishing a foundation for reliable and effective adaptive aiding technologies. The central component of this approach is a real-time, model-based classifier and predictor of operator state on a continuous high resolution (0-100) scale. Using operator workload as a test case, our approach incorporates novel methods of integrating physiological, behavioral, and contextual factors for added precision and reliability. Preliminary research conducted in the Air Force Multi Attribute Task Battery (AF_MATB) illustrates the added value of contextual and behavioral data for physiological-derived workload estimates, as well as promising trends in the classification accuracy of our approach as the basis for employing adaptive aiding strategies.

Computational integration of nanoscale physical biomarkers and cognitive assessments for Alzheimer’s disease diagnosis and prognosis

Protein properties of AD patients can be computationally integrated with behavioral assessments for AD diagnosis and prognosis. With the increasing prevalence of Alzheimer’s disease (AD), significant efforts have been directed toward developing novel diagnostics and biomarkers that can enhance AD detection and management. AD affects the cognition, behavior, function, and physiology of patients through mechanisms that are still being elucidated. Current AD diagnosis is contingent on evaluating which symptoms and signs a patient does or does not display. Concerns have been raised that AD diagnosis may be affected by how those measurements are analyzed. Unbiased means of diagnosing AD using computational algorithms that integrate multidisciplinary inputs, ranging from nanoscale biomarkers to cognitive assessments, and integrating both biochemical and physical changes may provide solutions to these limitations due to lack of understanding for the dynamic progress of the disease coupled with multiple symptoms in multiscale. We show that nanoscale physical properties of protein aggregates from the cerebral spinal fluid and blood of patients are altered during AD pathogenesis and that these properties can be used as a new class of “physical biomarkers.” Using a computational algorithm, developed to integrate these biomarkers and cognitive assessments, we demonstrate an approach to impartially diagnose AD and predict its progression. Real-time diagnostic updates of progression could be made on the basis of the changes in the physical biomarkers and the cognitive assessment scores of patients over time. Additionally, the Nyquist-Shannon sampling theorem was used to determine the minimum number of necessary patient checkups to effectively predict disease progression. This integrated computational approach can generate patient-specific, personalized signatures for AD diagnosis and prognosis.

System decision framework for augmenting human performance using real-time workload classifiers

The high volume of information available to human operators and increasing scale of work can become unmanageable due to the complexity found in a variety of domains. The need for precise, continuous assessment of human operator performance and state is important to identify when, and how, interventions should be delivered. One challenge that requires attention is the need for intelligent model-driven systems that identify specifically when some form of augmentation is needed while work is performed. Our current research and development efforts seek to fill this need by following the Sense-Assess-Augment (S-A-A) framework. We utilize the Performance Measurement Engine (PM Engine™) and the Functional State Estimation Engine (FuSE2) to derive second-by-second measurements of performance and human operator state to identify the specific points in time where performance decrements occur due to high workload. These human state patterns can be computationally modeled via the Performance Augmentation Cueing Engine in Real-time (PACER) to provide the decision logic necessary to predict when performance decrements are likely to occur. In this paper, we describe the methods used to collect our initial data set and explore the complex relationships between cognitive workload and primary task performance.

Using Context to Optimize a Functional State Estimation Engine in Unmanned Aircraft System Operations

As UAS operations continue to expand, the ability to monitor real-time cognitive states of human operators would be a valuable asset. Although great strides have been made toward this capability using physiological signals, the inherent noisiness of these data hinders its readiness for operational deployment. We theorize the addition of contextual data alongside physiological signals could improve the accuracy of cognitive state classifiers. In this paper, we review a cognitive workload model development effort conducted in a simulated UAS task environment at the Air Force Research Laboratory (AFRL). Real-time workload model classifiers were trained using three levels of physiological data inputs both with and without context added. Following the evaluation of each classifier using four model evaluation metrics, we conclude that by adding contextual data to physiological-based models, we improved the ability to reliably measure real-time cognitive workload in our UAS operations test case.

Establishing an instrumented training environment for simulation-based training of health care providers: An initial proof of concept

Objective: Several decades of armed conflict at a time of incredible advances in medicine have led to an acknowledgment of the importance of cognitive workload and environmental stress in both war and the health care sector. Recent advances in portable neurophysiological monitoring technologies allow for the continuous real-time measurement and acquisition of key neurophysiological signals that can be leveraged to provide high-resolution temporal data indicative of rapid changes in functional state, (i.e., cognitive workload, stress, and fatigue). Here, we present recent coordinated proof of concept pilot project between private industry, the health sciences, and the USA government where a paper-based self-reporting of workload National Aeronautics and Space Administration Task Load Index Scale (NASA TLX) was successfully converted to a real-time objective measure through an automated cognitive load assessment for medical staff training and evaluation (ACLAMATE). Methods: These real-time objective measures were derived exclusively through the processing and modeling of neurophysiological data. This endeavor involved health care education and training with real-time feedback during high fidelity simulations through the use of this artificial modeling and measurement approach supported by Aptima Corporations FuSE2, SPOTLITE, and PM Engine technologies. Results: Self-reported NASA TLX workload indicators were converted to measurable outputs through the development of a machine learning-based modeling approach. Workload measurements generated by this modeling approach were represented as a NASA TLX anchored scale of 0–100 and were displayed on a computer screen numerically and visually as individual outputs and as a consolidated team output. Conclusions: Cognitive workloads for individuals and teams can be modeled through use of feed forward back-propagating neural networks thereby allowing healthcare systems to measure performance, stress, and cognitive workload in order to enhance patient safety, staff education, and overall quality of patient care. The following core competencies are addressed in this article: Medical Knowledge, Interpersonal Skills, Patient Care, and Professionalism.

An Empirical Study of Human-Robotic Teams with Three Levels of Autonomy

Three search-and-identify experiments were conducted using the RoboFlag testbed to investigate the performance of human-robotic teams with different levels of autonomy. The first set of experiments required human operators to manually control robot trajectories by setting waypoints. In the second set, operators controlled robots via five pre-programmed plays. The third set allowed operators to operate waypoint control and automated plays, and performance feedback was provided to the user in real-time and/or at the end of each trial. Five performance metrics were analyzed across all experiments: game time, idle time, tag events, target location uncertainty, and target identity uncertainty. Performance increased with respect to all metrics when automated plays were available, and the addition of performance feedback in the third set of experiments further improved game time and idle time. Human efficiency and mission effectiveness are discussed with respect to autonomy level, form of performance feedback, and mission configuration.

Towards Probabilistic Operator-Multiple Robot Decision Models

Coupled operator-multiple vehicle systems are modelled in a unified framework using probabilistic graphs to yield a methodology for analyzing semi-autonomous systems. The framework uses conditional probabilistic dependencies between all elements, leading to a Bayesian network (BN) with probabilistic evaluation capability. Vehicle attitude/navigation states and target/classification states can be evaluated using nonlinear estimators such as the EKF, multiple model filter, information filter, or other approaches. Discrete operator decisions are being modeled as Bayesian network blocks, with conditional dependencies on the vehicle and tracking estimators. Initial decision models use combinations of softmax and discrete probability distributions.

Collaboration technologies for tactical command & control: Performance, workload, and situation awareness

Future tactical command and control (C2) will undoubtedly be affected by a shift toward network-centric warfare (NCW), a concept of operations that relies on sophisticated information and communication technologies for enabling real-time collaboration and heightened shared situational awareness among geographically-distributed individuals and teams. One of the necessary components of NCW is the availability of appropriate and effective collaboration tools and an understanding of how they affect individual and team performance efficiency, problem solving, decision making, situation awareness, workload, and communication. This is especially true given that these teams will be faced with making high-stake decisions under non-optimal conditions characterized by time stress, incomplete and/or inaccurate information, rapidly changing situations, and uncertainty. To date, the majority of research on collaboration technologies has been limited to face-to-face collaboration versus computer-mediated collaboration, with much of it showing an advantage for face-to-face collaboration in terms of speed and quality of the teampsilas decisions. Moreover, it has been noted that theoretical models of collaborative decision making, frameworks and models of collaboration technologies and processes, and empirical data describing the impacts of collaboration technology on team processes are lacking. Accordingly, the Air Force Research Laboratorypsilas Collaborative Tools for Tactical Command and Control Research Program assembled a multi-university research consortium to examine the impact of collaboration technologies on selection, workload, and team decision making in tactical command and control teams in simulated network-centric environments. This proposed panel comprises consortium members and represents a range of world-renown experts in mental workload assessment, cognitive neuroscience, human-system interaction, computer supported collaboration, humans and automation, and team decision making and team performance. The purpose of the panel will be to report and discuss the research that was conducted as part of the 18-month consortium project.