Andrew M. Rose

Methodological Approaches For Simulator Evaluations

Publisher Summary This chapter discusses the prescribed and actual evaluations of training system effectiveness. It presents alternatives to transfer of training as criteria for training system effectiveness. The chapter also presents an emerging model for training system evaluations, the device effectiveness forecast technique. The armed forces of the United States rely on training devices and simulators as indispensable components of training. The design of training devices in theory is based on detailed and precise information about the training requirements to be met, physical and functional characteristics needed to satisfy requirements, intended use of the device, and its effectiveness and cost. However, unavoidable logistic demands during training device acquisition make empirical tests of effectiveness difficult to conduct. As a necessary consequence, appraisals of a particular design, or competing design alternatives, are primarily analytic. Formal analytic procedures, however, are inadequate or nonexistent. The bases on which device design decisions are made have not been articulated, nor is it clear what kinds of data or levels of data specificity are needed to support such decisions. There is a need, therefore, for analytic procedures, applicable during both early and later stages of device acquisition that permit prediction of the potential effectiveness of device designs.