Stephanie M. Doane

Memory Processes of Flight Situation Awareness: Interactive Roles of Working Memory Capacity, Long-Term Working Memory, and Expertise

This research examined the role of working memory (WM) capacity and long-term working memory (LT-WM) in flight situation awareness (SA). We developed spatial and verbal measures of WM capacity and LT-WM skill and then determined the ability of these measures to predict pilot performance on SA tasks. Although both spatial measures of WM capacity and LT-WM skills were important predictors of SA performance, their importance varied as a function of pilot expertise. Spatial WM capacity was most predictive of SA performance for novices, whereas spatial LTWM skill based on configurations of control flight elements (attitude and power) was most predictive for experts. Furthermore, evidence for an interactive role of WM and LT-WM mechanisms was indicated. Actual or potential applications of this research include cognitive analysis of pilot expertise and aviation training.

Roles of working memory capacity and long-term working memory skill in complex task performance.

In the present research, we examined the relative roles of domain-general and domain-specific individual difference characteristics in complex cognitive task performance. Specifically, we examined the impact both of working memory (WM) capacity and of acquired skills used to encode presented information in an accessible form in long-term working memory (LTWM) on performance in a complex aviation task environment. Measures of WM capacity and LTWM skill served as performance predictors. A criterion measure of task performance was related to the predictor measures. The results indicated that an increase in LTWM skill decreases the role of WM capacity as the determinant of complex task performance, although both measures are important performance predictors. We discuss how the two distinct WM constructs coexist and interact to support complex task performance.

Pilot Ability to Anticipate the Consequences of Flight Actions as a Function of Expertise

The study offers insights into pilot ability to anticipate consequences of actions and how this ability changes with experience. Novice and expert pilots completed trials in which 3 screens depicted a control movement (or control movements), a cockpit flight situation, or a change in flight situation. Changes depicted in the 3rd screen of each trial were consistent, inconsistent with the mental model of the effect of the control movement or movements, or inconsistent with the application of the control movement(s) to the current flight situation. Pilots indicated whether the depicted change was inconsistent or consistent with their expectations, and accuracy of consistency judgments was greater for mental-model than for situation-model inconsistent statements. Experts are more accurate than novices, particularly for trials that involve multiple, meaningfully related control movements. Expert ability to organize information into meaningful units appears to facilitate future flight state projections, and projection failures appear to result from situation- rather than mental-model failures. Actual or potential applications of this research include analysis of flight situation awareness and flight performance errors.

ADAPT: A Predictive Cognitive Model of User Visual Attention and Action Planning

We present a computational cognitive model of novice and expert aviation pilot action planning called ADAPT that models performance in a dynamically changing simulated flight environment. We perform rigorous tests of ADAPTs predictive validity by comparing the performance of individual human pilots to that of their respective models. Individual pilots were asked to execute a series of flight maneuvers using a flight simulator, and their eye fixations and control movements were recorded in a time-synched database. Computational models of each of the 25 individual pilots were constructed, and the individual models simulated execution of the same flight maneuvers performed by the human pilots. The time-synched eye fixations and control movements of individual pilots and their respective models were compared, and rigorous tests of ADAPTs predictive validity were performed. The model explains and predicts a significant portion of pilot visual attention and control movements during flight as a function of piloting expertise. Implications for adaptive training systems are discussed.

Prompt comprehension in UNIX command production.

We hypothesize that a cognitive analysis based on the construction-integration theory of comprehension (Kintsch, 1988) can predict what is difficult about generating complex composite commands in the UNIX operating system. We provide empirical support for assumptions of the Doane, Kintsch, and Poison (1989, 1990) construction-integration model for generating complex commands in UNIX. We asked users whose UNIX experience varied to produce complex UNIX commands, and then provided help prompts whenever the commands that they produced were erroneous. The help prompts were designed to assist subjects with respect to both the knowledge and the memory processes that our UNIX modeling efforts have suggested are lacking in less expert users. It appears that experts respond to different prompts than do novices. Expert performance is helped by the presentation of abstract information, whereas novice and intermediate performance is modified by presentation of concrete information. Second, while presentation of specific prompts helps less expert subjects, they do not provide sufficient information to obtain correct performance. Our analyses suggest that information about the ordering of commands is required to help the less expert with both knowledge and memory load problems in a manner consistent with skill acquisition theories.

Comprehension-Based Skill Acquisition

We present a comprehension-based computational model of UNIX user skill acquisition and performance in a training context (UNICOM). The work extends a comprehension-based theory of planning to account for skill acquisition and learning. Individual models of 22 UNIX users were constructed and used to simulate user performance on successive command production problems in a training context. Comparisons of model and the human empirical data result in a high degree of agreement, validating the ability of UNICOM to predict user response to training.

Evaluating Comprehension-Based User Models: Predicting Individual User Planning and Action

Described is a program of research that uses rigorous methods to evaluate models of user cognition and action based on the construction-integration architecture of comprehension (Doane and Sohn, 2000; Kintsch, 1988; 1998). The models interrelate user environmental information, background knowledge, and current goals, and then spread activation throughout the interrelated information to simulate UNIX user command productions, aviation pilot eye fixations and control movements during flight, and army personnel intelligence planning. Models of individuals in the complex interactive environments are tested for descriptive as well as predictive validity. Comparisons of model and human empirical data have resulted in a high degree of agreement, validating the ability of the comprehension-based architecture to support models that can predict user performance. Evaluation methods are detailed and the importance of evaluative rigor is discussed.

Modeling user action planning: A comprehension based approach

We review our efforts to model user command production in an attempt to characterize the knowledge users of computers have at various stages of learning. We modeled computer users with a system called NETWORK (Mannes and Kintsch, 1988; 1991) and modeled novice, intermediate, and expert UNIX command production data collected by Doane et al. (1990b) with a system called UNICOM (Doane et al., 1989a; 1991). We use the construction-integration theory of comprehension proposed by Kintsch (1988) as a framework for our analyses. By focusing on how instructions activate the knowledge rele/ant to the performance of the specified task, we have successfully modeled major aspects of correct user performance by incorporating in the model knowledge about individual commands and knowledge that allows the correct combination of elementary commands into complex, novel commands. Thus, experts can be modeled in both NETWORK and in UNICOM. We further show that salient aspects of novice and intermediate performance can be described by removing critical elements of knowledge from the expert UNICOM model. Results suggest that our comprehension-based approach has promise for understanding user interactions and implications for system design are discussed.

Individual Differences in Object Localization in Virtual Environments

The present research examined the effects of disorientation on human ability to locate objects in space in a virtual environment (VE). Participants were asked to memorize the location of virtual objects, and then they were asked to indicate where the objects were located while they were inside a virtual chamber. This procedure was repeated in both eyesclosed and disoriented conditions. Subject pointing responses were used to measure memory for the relative location of objects in virtual space. This method was extended from previous research in a real-world setting. The results showed systematic individual differences in the effects of disorientation on the ability to locate objects in space. Further, the use of strategies played a role in object localization ability, but strategy use was affected by the individual differences in spatial representations used by subjects.

Cognitive engineering studies of DSS and dealing with uncertainty in load for real-time adaptive power system reconfiguration

The quality of a decision made by human operators in a real-time reconfiguration of the power system can be improved by providing him/her with a decision support system (DSS). This paper discusses the quantitative cognitive engineering of a decision support system to improve the situational awareness of the operator working in a real-time environment. The quality of decision results in reconfiguration of a power system network is compared with and without the decision support system. Within the reconfiguration activities, lack of awareness of the uncertainty present in load data will restrict the reconfiguration algorithm to give a new optimized power system topology. Uncertainty due to faulty metering is duly analyzed and a fuzzy based approach of dealing with uncertainty is explained.