Dev Minotra

Using the Neocities 3.1 Simulation to Study and Measure Team Cognition

This paper provides a detailed explanation of the link between NeoCITIES, a crisis management simulation of emergency response teams, and team cognition. Descriptions of the NeoCITIES simulation structure, interface, and modifications are provided, along with its functionality in effectively studying team cognition. The paper focuses on three commonly examined constructs within the team cognition literature, namely, team situation awareness, team mental models, and information sharing.

Finding Common Ground: Situation Awareness and Cognitive Work Analysis

The article “Situation Awareness Misconceptions and Misunderstandings” (Endsley, 2015) discusses common fallacies noted in the literature in the interpretation of Endsley’s 1995 model of situation awareness (SA; Endsley, 1995). The clarifications presented in the article provide a more complete and comprehensible explanation of the model. As in the complex domains that we study, our attention suffers from limitations as we simplify each other’s work in our eager and well-intentioned quest to attack new and exciting problems. Dr. Endsley is to be thanked for giving us pause to rebuild our own SA of SA in her thoughtful article. SA has, for many years, been a powerful and influential construct. In our own work in cognitive work analysis (CWA), we have viewed SA as a complementary framework that challenges and drives CWA. Without doubt, the output of a CWA-based design process should be the design of a system that promotes better SA and performance (Burns et al., 2008). CWA and goaldirected task analysis may organize the world in slightly different dimensions, but the overall intent is the same: to create systems that support human decision making as well as we can. Indeed, it is this common intent that unites us in our field. To advance cognitive engineering, there are times when we must challenge each other, critique each other’s models, hunt for flaws, and identify promising new directions. Assuredly, this helps us progress, strengthen our methods, and deepen our understanding. This article clearly identifies such activity and responds to it. We are all better for this exercise, as it challenges both SA and all our approaches to grow and deepen. Acknowledging this, we would like to change our perspective to a larger one and discuss challenges facing cognitive engineering as a whole. In these challenges, our existing methods, SA, CWA, and other approaches must adapt and grow. Advancements in intelligent systems and automation have increased in the amount of data produced by information systems and yet placed the human in new roles. In many cases, these roles are partially in the loop and partially out of the loop and may involve supervisory control or may have the human working in systems with very little supervisory control at all because the automation is largely nontransparent. We outline three core areas of challenges: self-awareness and self-regulation, memory failures or performance with incorrect SA, and design for unstructured environments.

Understanding safe performance in rapidly evolving systems: a risk management analysis of the 2010 US financial market Flash Crash with Rasmussen's risk management framework

ABSTRACT Systemic events or widespread disruptions in financial markets like the Flash Crash are a public concern as they jeopardise investors’ confidence in financial markets and result in financial losses for market participants. Federal regulations for financial markets do not keep pace with the evolution and growth of financial technology including advances in algorithmic and high-frequency trading (HFT). We report an analysis of the Flash Crash which occurred on 6 May 2010 using Rasmussens 1997 risk management framework. While the framework has been validated on a number of well-documented accidents, our work examines the framework in the context of a large-scale adverse event associated with complex technologies and automated systems capable of evolving over time. Our contribution is a set of implications of the Flash Crash associated with Rasmussens propositions. These implications would inform regulators and risk assessment methods for rapidly evolving complex socio-technical systems.

Capturing human cognition in cyber-security simulations with NETS

As new technologies are employed to improve cyber-security, designers often make claims that their interfaces improve usability, improve situation awareness, reduce cognitive load, or improve team cognition. However, the assessment of these claims is complex because these higher cognitive functions are often difficult to measure. This paper outlines a three human-subjects experiments, using the NETS simulation engine, to explore human cognition in simulated cyber-security environments. First, the salient features of the NETS simulator are described. Second, the three experiments are each outlined to include a description the individual cognitive measurements that were assessed. Next, a summarization of results from all three experiments will compare the important findings. This paper concludes with a discussion of key factors that should be considered when attempting to quantify differences in human cognition when presented with novel visualization approaches in a cyber-security context.

An Empirical Study on a Locality Based Heuristic in Multi-Agent Constraint Satisfaction

Previous work on multi-agent oriented constraint-satisfaction shows that ERA is competitively suitable in solving the n-queen problem, and is capable of providing approximate solutions to different types of CSPs in about two or three time steps. Challenges remain in developing more efficient ERA based systems for solving CSPs beyond just approximation. In this paper, a modified ERA algorithm, involving a locality based heuristic and a random tie-breaking method, is proposed and tested. Experimental results have shown significant performance improvements, indicating some of the key characteristics and strengths of ERA as a multi-agent solution search process.

Eliciting Knowledge from Helicopter Pilots: Recommendations for Revising the ACTA Method for Helicopter Landing Tasks

The task of landing helicopters on ships is cognitively complex and is bounded by several limitations for safe operation. To design technologies to support helicopter pilots in shipboard operations, a better understanding of cognitive processes underlying helicopter piloting in shipboard landing maneuvers is required. Limitations in prior work on ship-based helicopter pilots motivate more studies to better understand how pilots compensate for difficulties associated with ship motion, air turbulence among other factors. We report a preliminary study that used the Applied Cognitive Task Analysis (ACTA) method. We interviewed three participants who have ground-based helicopter piloting experience. The results indicate that ACTA can be used in eliciting knowledge from helicopter pilots. However, we have identified several caveats with the use of the method for this application. The revised method will be used to interview helicopter pilots with shipboard landing experience. Our study has identified a number of cognitive elements in the task which are associated with task management, divided attention, and noticing anomalies.

Protocol for Usability Testing and Validation of the ISO Draft International Standard 19223 for Lung Ventilators

Background Clinicians, such as respiratory therapists and physicians, are often required to set up pieces of medical equipment that use inconsistent terminology. Current lung ventilator terminology that is used by different manufacturers contributes to the risk of usage errors, and in turn the risk of ventilator-associated lung injuries and other conditions. Human factors and communication issues are often associated with ventilator-related sentinel events, and inconsistent ventilator terminology compounds these issues. This paper describes our proposed protocol, which will be implemented at the University of Waterloo, Canada when this project is externally funded. Objective We propose to determine whether a standardized vocabulary improves the ease of use, safety, and utility as it relates to the usability of medical devices, compared to legacy medical devices from multiple manufacturers, which use different terms. Methods We hypothesize that usage errors by clinicians will be lower when standardization is consistently applied by all manufacturers. The proposed study will experimentally examine the impact of standardized nomenclature on performance declines in the use of an unfamiliar ventilator product in clinically relevant scenarios. Participants will be respiratory therapy practitioners and trainees, and we propose studying approximately 60 participants. Results The work reported here is in the proposal phase. Once the protocol is implemented, we will report the results in a follow-up paper. Conclusions The proposed study will help us better understand the effects of standardization on medical device usability. The study will also help identify any terms in the International Organization for Standardization (ISO) Draft International Standard (DIS) 19223 that may be associated with recurrent errors. Amendments to the standard will be proposed if recurrent errors are identified. This report contributes a protocol that can be used to assess the effect of standardization in any given domain that involves equipment, multiple manufacturers, inconsistent vocabulary, symbology, audio tones, or patterns in interface navigation. Second, the protocol can be used to experimentally evaluate the ISO DIS 19223 for its effectiveness, as researchers around the world may wish to conduct such tests and compare results.

Guidelines and Caveats for Manipulating Expectancies in Experiments Involving Human Participants

In a number of domains involving decision making under high time pressure, the appearance of an unexpected critical event can startle operators causing errors that may potentially lead to a chain reaction of unfavorable events. Simulating such situations in a laboratory requires overcoming several challenges tied to human responses to unexpected and rare critical events. We discuss cases of experimental studies that have incorporated simulations, critical events, interruptions, predictive displays and ecological displays involving naïve and expert participants. We spell out guidelines and caveats pertaining to training, expertise, temporal variables, critical event frequencies, scenario design, mediator and moderator variables to name a few. The guidelines and caveats may inform researchers in the design of generalizable and reproducible experiments and in the analysis of data.