Brian T. Schreiber

Developing competency-based methods for near­ real-time air combat problem solving assessment*

Air combat scenarios present unique problems where the solution is not obvious. An advanced networked simulation environment can be used to train high-level cognitive air combat skills such as problem solving. A problem in the development of principled and construct oriented assessment is related to determining the level of specificity of the assessment. We present a detailed discussion of the definition of critical competencies associated with combat mission problem solving performance, and we describe our approach to develop and implement an embedded performance assessment system that maps outcomes to these competencies. Finally, we discuss the implications for our approach for comprehensive assessment and discuss our goals for an evaluation of the competency-based approach to assessment.

Distributed Mission Operations Within-Simulator Training Effectiveness

This study examined the effectiveness of distributed mission operations (DMO) training using objective and subjective measures. DMO consists of multiplayer networked environments that facilitate the training of higher order individual and team-oriented combat skills. Objective measures included performance assessments, and subjective measures included performance ratings by subject matter experts and pilot perceptions of DMO utility. Results indicated that DMO training improved pilot performance, most notably in the reduction of the number of enemy strikers reaching their target and the number of F-16 mortalities. Considerations of real-world reductions in loss of life and expenditures are discussed.

Challenges in transforming military training: Research and application of advanced simulation and training technologies and methods.

Welcome to this special issue! This issue is unique in several ways. First, we have an opening paper discussing some of the operational and integration issues being faced in transforming training—in this case directly by the USAF air combat community— but the issues described and discussed are pervasive to other applications and mission areas as well. Second, there are a series of wide-ranging empirical research papers focusing on a diverse set of issues and challenges and that discuss solutions of interest and promise. Finally, we have two internationally recognized experts in the field, Dr. Robert Foster and Dr. Dexter Fletcher, who provide us with their overall views on the papers in the issue as well as their perspectives regarding the future science, technology, and application roadmap for training and readiness. The major focus of this special issue is on empirical studies within the military context, or research that has direct potential application to the military simulation and training community. Since most of the papers in this special issue are empirical, they focus on specific research questions and issues. These focused questions serve as excellent examples of larger training and learning research topics/issues being addressed by agencies around the world today. Each of the papers in this special issue explores topics of relevance that need to be considered in transforming military training. Throughout the course of this special issue, we encourage the reader to ponder the topics presented in the individual papers and how those topics present unique challenges for transforming training in our military today. Here we have summarized some of those topics albeit at a very high level. The first paper provides an excellent overview of the rationale and stated objectives for transforming training in the military. Chapman and Colegrove describe many of the challenges facing the Air Force today in terms of maintaining readiness while investing in and leveraging research and practice to improve the efficiency and effectiveness of training in the 21st century. The paper discusses the traditional military training emphasis on individual, lower-level skill training on a frequency-oriented basis and the tension created with a more transformational view toward the future that embraces individual, team, and interteam skill training in a competency-based approach. Transforming Air Force training requires (a) more assets involved in dynamic training events (through methods such as Distributed Mission Operations, or DMO) and (b) assessing the proficiency of the warfighter, rather than simply counting number of events logged. The transformations stem largely from technological advancements that enable what was previously not possible (i.e., DMO) and the methods and processes that make up the framework of competency-based training (Mission Essential Competencies, or MECs). The authors close with a discussion of these key transformations on implications for the future in terms of needs for transforming training policies, the need to provide a classified training environment that accommodates program and operational security, and the technologies considered for incorporation. In the second paper, by Stanney, Cohn, Milham, Hale, Darken, and Sullivan, the authors discuss a method and study revolved around the training of spatial knowledge. Topics covered in this paper illustrate a more general challenge Winston Bennett, Jr., Brian Schreiber, Antoinette M. Portrey, and Herbert H. Bell, Air Force Research Laboratory, 711th Human Performance Wing, Human Effectiveness Directorate, Warfighter Readiness Research Division, Wright-Patterson AFB, Ohio. Correspondence concerning this article should be addressed to Winston Bennett, Jr., 711 HPW/RHAS, 2620 Q Street (Bldg 20852), Wright Patterson AFB, OH 454337955. E-mail: winston.bennett@wpafb.af.mil, winston. bennett@us.af.mil T hi s do cu m en t is co py ri gh te d by th e A m er ic an Ps yc ho lo gi ca l A ss oc ia tio n or on e of its al lie d pu bl is he rs .

Incidental Changes to Training Capabilities Due to Technological Improvements

Abstract : During the development and enhancement of simulator systems, new technologies are incorporated to improve desired capabilities. The degree to which changes to the simulator system improve training capabilities in anticipated areas is of primary interest; however, an evaluation of benefits from newly incorporated technologies should consider the impact on training capabilities in other, less directly related areas. The current study utilized methodology developed to identify and evaluate the fidelity and deficiencies of a simulation system. The process involved utilizing Mission Essential Competencies and Subject Matter Experts to determine the training capabilities of a simulation system. The work was applied to the development of a Deployable Tactical Trainer as a method of providing feedback on the system. Data from the process were obtained throughout the development of the simulator, and the current study presents a comparison of the evaluation of the system before and after a meaningful upgrade to the technology of the simulation system. At issue is determining whether or not the changes in the training capabilities were as expected. Incidental changes to training capabilities of the system were identified and are discussed, presenting an overall evaluation of the expected versus unexpected effects on the system. Implications for the importance of building a global awareness of the simulator system are discussed. The growing dependence on simulation systems for training increases the impact of changes to the system on pilot readiness.