Carolyn Prince

SITUATION AWARENESS IN TEAM PERFORMANCE: IMPLICATIONS FOR MEASUREMENT AND TRAINING

Situation awareness has long been recognized as an important variable in aviation performance. Research to date has focused on identifying characteristics of situation awareness for individuals, not on the behaviors and processes associated with team situation awareness. The purpose of this review is to delineate and identify characteristics of team situation awareness. In addition, implications are discussed and research questions are outlined that target the measurement and training of situation awareness in teams.

DOES CRM TRAINING IMPROVE TEAMWORK SKILLS IN THE COCKPIT?: TWO EVALUATION STUDIES

Crew resource management (CRM) has been the most widely used instructional strategy to enhance teamwork skills within aviation. However, few studies have provided empirical evidence that this strategy results in increased use of teamwork behaviors. The current paper describes 2 evaluation studies of CRM training that were conducted with 96 naval aviators and followed a multiple assessment approach. Results indicated that highly experienced aviators benefited from the training, which focused on increasing the use of several critical teamwork skills. These rare evaluation data suggest that CRM training is a viable strategy for improving teamwork in the cockpit. Implications for team training are discussed.

A Methodology for Enhancing Crew Resource Management Training

Human error is an ever-present threat to the safe conduct of flight. Recently, applied psychologists have developed an intervention, crew resource management (CRM) training, designed to help prevent human error in the cockpit. However, as it is commonly applied within the aviation community, CRM lacks standardization in content, design, delivery, and evaluation. This paper presents a discussion of an applied program of research aimed at developing a methodology for the design and delivery of CRM training within the Navy. This long-term, theoretically based program of aviation team research included identification of skills to be trained, development of performance measures, application of instructional design principles, and evaluation of the training delivery. Our conclusion indicates that a systematic methodology for developing CRM training can result in better performance in the cockpit. Actual or potential applications of this research include any task environment in which teams are interdependent.

Games teams play: A method for investigating team coordination and performance

Teams are playing an increasingly important role in the workplace. However, reviews of the team performance literature have suggested that there are serious deficiencies in our understanding of team processes and performance (e.g., Dyer, 1984). These difficulties may be attributable, in part, to the lack of laboratory methodologies to investigate team performance. This paper describes the use of low-fidelity simulations as a potentially useful paradigm for researching team coordination and performance. This paradigm is advantageous in that it offers relatively high levels of experimental control and task representation at a low cost.

Can PC-Based Systems Enhance Teamwork in the Cockpit?

We used a personal computer (PC)-based flight simulator to develop a crew resource management (CRM) training module. Two-person teams practiced with the simulator and also received feedback from an instructor about their performance. We evaluated training effectiveness by comparing trained teams (N = 24) to control teams (N = 24) in a high-fidelity simulator. Raters who were blind to condition provided evaluations of the teams on both CRM and technical proficiency. The results show positive transfer of training of CRM skills from the PC-based system to the high-fidelity simulator. This study supports the effectiveness of relatively inexpensive PC-based systems for CRM training.

Measurement of Team Situation Awareness in Low Experience Level Aviators

Abstract Two measurement methods for situation awareness were tested on 2-person aircrews and compared for the information they yielded on team situation awareness. Forty-one crews of low experience level military aviators flew 2 different scenarios, 1 in a full-mission simulator, a 1 in a low-fidelity trainer. Team situation awareness was measured by instructor and observers in the high-fidelity simulator scenario and by responses to questions on flight knowledge in a scenario in the low-fidelity trainer. Scores on both measures were found to be reliable. Team situation awareness scores based on flight knowledge collected in the low-fidelity scenario were significantly correlated with team performance in the high-fidelity simulation (r = .41, p< .05). The 2 team situation awareness scores (1 from the high-fidelity simulation and 1 from the low-fidelity simulation) were also significantly correlated (r = .43, p< .05). These findings and related information are discussed in relation to the use of the measurements in situation awareness training for low experience level pilots.

The reliability of instructor evaluations of crew performance: Good news and not so good news

Two instructors rated the crew performance of helicopter crews (N = 45) who flew a simulated mission in a full-motion simulator. The instructors were given dimension training and behavior-observation training before the ratings. Before making their ratings, instructors completed an observation form as they watched videotapes of the crews and then completed a form that helped link behaviors to dimensions. For their ratings, instructors recorded or evaluated crew behaviors using 3 types of items: (a) specific crew behaviors in response to scenario events (e.g., whether the crew kept out of icing conditions), (b) evaluations of crew responses to scenario events (e.g., overall handling of the icing problem), and (c) crew resource management (CRM) dimensions for the entire scenario (e.g., evaluations of decision making). Results showed (a) both interjudge agreement and internal consistency were high for evaluations of crew responses to scenario events, (b) interjudge agreement was low but internal consistency was high on CRM items and scales, and (c) interjudge agreement was high but internal consistency was low for specific observable behaviors. The results for the evaluations of crew responses to scenario events were very encouraging and showed reliability over time and over crews. Suggestions for improving the reliability for the other 2 item types are provided.

Rapidly Reconfigurable Event-Set Based Line Operational Evaluation Scenarios

An important cornerstone of the Advanced Qualification Program is the use of realistic flight simulations to train flight crews and evaluate their proficiency. This is achieved through the use of Line Operational Simulations (LOSs) which contain a number of realistic event sets that require flight crews to use the knowledge, skills, and abilities they gained in training in both technical and crew resource management areas. In the past, the development of LOS scenarios was complex, expensive, and time-consuming. As a result, the number of LOS scenarios used by any particular training organization was limited. This may have led, in some cases, to the LOS scenarios being compromised among flight crews, thus reducing the validity and reliability of the assessment process. In response to this problem, it has been suggested to create a methodology for quickly reconfiguring the content of flight simulator LOS scenarios. This paper provides a background for this new development and describes a research project that was begun in response to this need. A number of specific research questions are discussed which need to be answered before the methodology can be adopted by participants in the AQP program.

Team Process Measurement and Implications for Training

The purpose of this research was to establish the construct validity of a behaviorally anchored rating scale developed to measure team process behaviors. This scale contains six skills (i.e. leadership, assertiveness, decision making/mission analysis, situation awareness, communication, adaptability/flexibility) that were identified through a prior needs analysis with training specialists and subject matter experts. Student and instructor pilots (104 individuals, 51 teams) participated in two team tasks (simulated aviation tasks) which were designed to elicit the team process behaviors identified for the rating scale, and were rated on their behaviors. A multitrait-multimethod analysis on the resulting ratings (Campbell and Fiske, 1959) was conducted. Evidence of convergent and discriminant validity as well as some method bias were found when the method investigated was team task. Implications for the use of the team process scale in training are discussed.

The Influence of Experience and Organizational Goals on Leadership in the Military Cockpit

Despite the importance of leadership, little formalized leadership research has been conducted in the military cockpit. To explore how directive leadership is manifested in military helicopter cockpits, communications of 52 military pilots, flying a realistic simulator scenario in crews of 2, were analyzed. Based on information from general leadership research and theory, we focused the analyses on flight experience and organization goals. Pilots were assigned to 3 different groups, based on their experience level. Transcripts of communications from 3 portions of the flight were analyzed: from take-off to the first problem; from identification of the first problem to its solution and; from the discovery of the second problem to its solution. Analysis of variance showed there were no significant differences in commands given by the aircraft commanders in all 3 segments despite the experience level of the second pilot. There were significant differences in the commands given by the second pilot to the aircraft commander based on experience level and organizational goals. Although aircraft commanders were equally directive with their second pilots, no matter what their experience levels, during problem solving, they differed in their approach to the problems depending on the second pilots experience. We discuss these findings and suggest application to both training and additional research.