Haydee M. Cuevas

Fostering constructive cognitive and metacognitive activity in computer-based complex task training environments

In this paper, we describe the results of our programmatic research efforts aimed at investigating the use of interactive computer-based training technology to support knowledge acquisition and integration for complex task training environments. We present the theoretical rationale for our efforts and briefly describe the successive iterations of our investigation. Based upon the significant findings in our studies, we then present, within the following areas, a set of propositions for the design and evaluation of computer-based training programs for complex systems: (1) Diagrams and knowledge integration; (2) Multimedia and training evaluation; (3) Mental model development and training evaluation; (4) Instructional efficiency and training evaluation; (5) Fostering constructive metacognitive processes; (6) Supporting individual differences in ability. These propositions are presented in the context of the related theoretical rationale drawn from the literature and the relevant empirical support from the body of research conducted within our collective research efforts.

A Picture Is Worth a Thousand Connections: The Facilitative Effects of Diagrams on Mental Model Development and Task Performance.

This study investigated the influence of training format on knowledge structure development and how differing knowledge structure measurement techniques can be used to best assess training effectiveness. The presence of diagrams was manipulated in a computer-based training tutorial designed to assess mental model development and knowledge acquisition. Participants presented with diagrams were more likely to accurately draw connections across modules of the training. Diagrams also facilitated performance on measures of integrative knowledge but not on measures of declarative knowledge. The results are discussed in the context of mental model theory.

Collaborative planning and situation awareness in Army command and control

We conducted a theoretical investigation of a complex command and control (C2) operation—the manoeuvres planning processes in Army land-battle situations, to improve understanding of how technology can best be designed to support planning and course of action development. We drew upon results from cognitive task analyses and interviews with subject matter experts and insights gleaned from observations of Army training exercises and experiments to make inferences on the C2 activities carried out in preparation for tactical manoeuvres. In this paper, we summarize several critical human factors issues associated with planning in a rapidly evolving environment, as identified in our investigation, and describe system design concepts aimed at addressing these challenges to distributed collaborative planning of C2 activities. We conclude with implications for the application of these findings to other C2 domains.

Training individuals for distributed teams: problem solving assessment for distributed mission research §

In this paper we describe an effort investigating the feasibility and utility of cognitively diagnostic assessment of problem solving when training for distributed team tasks. We utilized computer-based knowledge elicitation methods to assess both relational problem solving, requiring the semantic integration of concepts, and dynamic problem solving, requiring the ability to integrate and apply these concepts. Additionally, we addressed how metacognitive processes interact with learning outcomes when training for complex synthetic task environments. We find first, that multiple methods of assessing problem solving performance are diagnostic of knowledge acquisition for a complex synthetic team task, and second, that general metacomprehension predisposition is related to metacomprehension accuracy in synthetic task environments.

Information Alignment and Task Coordination in Organizations: An 'Information Clutch' Metaphor

Abstract Despite their rising popularity, distributed teams face a number of collaboration challenges that may potentially hinder their ability to productively coordinate their resources, activities, and information, often in dynamic and uncertain task environments. In this paper, we focus principally on the criticality of information alignment for supporting coordinated task performance in complex operational environments. As organizations become more expertise, geographically, and temporally distributed, appropriate alignment and coordination among distributed team members becomes more critical for minimizing the occurrence of information flow failures, poor decision-making, and degraded team performance. We first describe these coordination processes using the metaphor of an ‘information clutch’ that allows for smooth transitions of task priorities and activities in expert teams. We then present two case study examples that illustrate the potentially significant impact of information sharing and information alignment on productivity and coordination in organizations. We conclude with a discussion of future directions in this area.

Predicting Cognitive Readiness of Deploying Military Medical Teams

Cognitive readiness can be defined as “possessing the psychological (mental) and sociological (social) knowledge, skills, and attitudes (KSAs) that individuals and team members need to sustain competent professional performance and mental wellbeing in the dynamic, complex, and unpredictable environments of military operations.” Determining if medical personnel are cognitively ready to perform their job poses a considerable challenge to the research community both in terms of understanding what is meant by being cognitively ready and in terms of developing methods to actually assess it. Accordingly, as part of a government-sponsored research program, we set out to gain a better understanding of what is meant by being “cognitively ready” for military medical teams as well as develop a tool for predicting cognitive readiness. In this paper, we describe the design, development, and initial user testing of our Medical Cognitive REadiness Survey Tool (M-CREST).

Improving Situation Awareness through Cross-Training

This study investigated how cross-training, particularly in a leadership role, may assist individuals in better understanding the task requirements of their fellow team members, and, thereby, increase their shared situation awareness. Data was collected from a training exercise at the Joint Personnel Recovery Agency. Participants were assigned to one of 4 teams (Navy, Army, Special Ops, or Joint Service) and completed a simulated exercise designed to mimic real life events in a recovery center. Each player was rotated though the various positions and teams such that everyone had a chance to be a team director (lead person) and a team member in each of the 4 teams. Situation awareness was measured during the exercise using the SAGAT technique. Overall, results suggest that cross-training may lead to improved situation awareness. Participants, on average, exhibited greater situation awareness following experience in the director role than prior to director experience.

Virtual Field Trips: Synthetic Experiences and Learning

Founded on the principles of experiential learning and anchored instruction, Virtual Field Trips utilize state-of-the art technologies to create immersive, multi-sensory, interactive experiences with real world environments. Virtual Field Trips are designed to be an integral part of a technology-enabled educational system to teach targeted material and motivate students.

The Facilitative Effects of Diagrams on Scaffolding Knowledge Acquisition and Metacognition in Low Verbal Ability Learners

This study investigated the differential benefit of diagrams as a learning aid for participants of differing ability levels. Diagrams facilitated the acquisition of conceptual knowledge but had no effect on declarative knowledge acquisition. Additionally, diagrams increased metacognitive accuracy. More importantly, the effect on knowledge acquisition and metacognitive accuracy was found to be strongest for participants with low verbal ability. Finally, incorporating diagrams into the training resulted in improved instructional efficiency (i.e., higher level of performance was achieved with less mental effort). Implications for incorporating findings on individual differences into training system design are discussed.

Investigating Individual Differences and Instructional Efficiency in Computer-Based Training Environments

This study assessed the extent to which a guided learner-generated questioning strategy could facilitate the acquisition of task-relevant knowledge and improve the instructional efficiency of a computer-based training program for a complex dynamic distributed decision-making task. This study also investigated how individual differences in verbal comprehension ability may interact with this instructional strategy to impact post-training outcomes. Overall, results highlighted the importance of learner aptitudes in complex task training and also showed that the effect of the instructional strategy on knowledge acquisition and the training programa” instructional efficiency was strongest for learners with low verbal comprehension ability. Implications for the design of adaptive learning systems are discussed