Thomas Carolan

Effectiveness of Part-Task Training and Increasing-Difficulty Training Strategies A Meta-Analysis Approach

Objective: The objective was to conduct meta-analyses that investigated the effects of two training strategies, increasing difficulty (ID) and part-task training (PTT), on transfer of skills and the variables that moderate effectiveness of the strategies. Background: Cognitive load theory (CLT) provides a basis for predicting that training strategies reducing the intrinsic load of a task during training avail more resources to be devoted to learning. Two strategies that accomplish this goal, by dividing tasks in parts or by simplifying tasks in early training trials, have offered only mixed success. Method: A pair of complementary effect size measures were used in the meta-analyses conducted on 37 transfer studies employing the two training strategies: (a) a transfer ratio analysis on the ratio of treatment transfer performance to control transfer performance and (b) a Hedges’ g analysis on the standardized difference between treatment and control group means. Results: PTT generally produced negative transfer when the parts were performed concurrently in the whole transfer task but not when the parts were performed in sequence. Variable-priority training of the whole task was a successful technique. ID training was successful when the increases were implemented adaptively but not when increased in fixed steps. Both strategies provided evidence that experienced learners benefited less, or suffered more, from the strategy, consistent with CLT. Conclusion: PTT can be successful if the integrated parts are varied in the priority they are given to the learner. ID training is successful if the increases are adaptive. The fundamental elements of CLT are confirmed.

Costs and benefits of more learner freedom: meta-analyses of exploratory and learner control training methods.

Objective: Individual meta-analyses were conducted for six training methods as part of a U.S. Army basic research project. The objective was to identify evidence-based guidelines for the effectiveness of each training method, under different moderating conditions, for cognitive skill transfer in adult learning. Results and implications for two of these training methods, learner control (LC) and exploratory learning (EL), are discussed. LC provides learners with active control over training variables. EL requires learners to discover relationships and interactions between variables. Background: There is mixed evidence on the effectiveness of both LC and EL learning methods on transfer relative to more guided training methods. Cognitive load theory (CLT) provides a basis for predicting that training strategies that manage intrinsic load of a task during training and minimize extraneous load will avail more resources that can be devoted to learning. Method: Meta-analyses were conducted using a Hedges’s g analysis of effect sizes. Control conditions with little to no learner freedom were contrasted with treatment conditions manipulating more learner freedom. Results: Overall more LC was no different from training with limited or no learner control, and more EL was less effective than limited or no exploration; however, each can be effective under certain conditions. Both strategies have been more effective for cognitive skill learning than for knowledge recall tasks. LC exhibited more benefit to very near transfer, whereas EL’s benefit was to far transfer. Conclusion: Task type, transfer test, and transfer distance moderate the overall transfer cost of more learner freedom. Application: The findings are applicable to the development of instructional design guidelines for the use of LC and EL in adult skill training.

The Influence of Cognitive Load on Transfer With Error Prevention Training Methods A Meta-Analysis

Objective: The objective was to conduct research synthesis for the U.S. Army on the effectiveness of two error prevention training strategies (training wheels and scaffolding) on the transfer of training. Background: Motivated as part of an ongoing program of research on training effectiveness, the current work presents some of the program’s research into the effects on transfer of error prevention strategies during training from a cognitive load perspective. Based on cognitive load theory, two training strategies were hypothesized to reduce intrinsic load by supporting learners early in acquisition during schema development. Method: A transfer ratio and Hedges’ g were used in the two meta-analyses conducted on transfer studies employing the two training strategies. Moderators relevant to cognitive load theory and specific to the implemented strategies were examined. The transfer ratio was the ratio of treatment transfer performance to control transfer. Hedges’ g was used in comparing treatment and control group standardized mean differences. Both effect sizes were analyzed with versions of sample weighted fixed effect models. Results: Analysis of the training wheels strategy suggests a transfer benefit. The observed benefit was strongest when the training wheels were a worked example coupled with a principle-based prompt. Analysis of the scaffolding data also suggests a transfer benefit for the strategy. Conclusion: Both training wheels and scaffolding demonstrated positive transfer as training strategies. As error prevention techniques, both support the intrinsic load–reducing implications of cognitive load theory. Application: The findings are applicable to the development of instructional design guidelines in professional skill-based organizations such as the military.

Part Task Training Methods in Simulated and Realistic Tasks

Part task methods are widely used in training programs when the full target task is too complex or impractical due to cost or availability. However, part task training has had mixed success in transfer to the whole task, in comparison to whole task training. For unmanned systems, the goal is to create training that prepares trainees to control an actual vehicle once one is available. The present research compares part and whole task training options in a simulated training environment and a realistic transfer environment. The task is teleoperating a robotic device to detect and identify vehicles in an urban environment. Part task training on the real task yielded a significant benefit in first trial whole task transfer performance and in trials to achieve transfer criteria relative to training on the simulated environment only. While this gain came at a relative cost in overall training time, it produced a significant training effectiveness gain by reducing the training time required on the more expensive robotic system.

Cost Effective Approaches to Training Reconnaissance Tasks on Unmanned Systems

This study investigated the cost effectiveness of part task training for teaching operators to control an unmanned vehicle. It was assumed that initial training would be done via simulation and that learning would then be transferred to a live environment with a physical robot. Different combinations of part and whole task training in the simulation and live environment were tested. Findings supported the efficiency and transfer effectiveness of conducting part task training in the live environment prior to attempting the whole task in the live environment. Trainees spent time using relatively low-cost part task trainers. This translated to less time being used to achieve proficiency on the physical robot resulting in potential cost savings.

Error Training and Adaptive Remediation The Impact on Transfer Performance in a Complex Planning Task

An experiment investigated the relative effectiveness of three training strategies for improving transfer performance in a cognitively complex decision-making task. Error Management is an exploratory training approach in which trainees learn by exploring and making errors. Error Prevention training includes structured training on likely errors. Adaptive Remediation is error management training supplemented with practice scenarios tailored to the types of errors made. The study is an important addition to the literature because it examines training and transfer performance in a cognitively complex militarily-relevant task. Participants used a digital system to plan, execute, and monitor unmanned vehicles in reconnaissance missions. Transfer scenarios tested the ability to adapt what was learned when attempting new tasks and in an error prone environment. The results showed that transfer performance was better when participants did not complete adaptive remediation scenarios. It is believed that the adaptive remediation tasks encouraged participants to focus on single tasks and as a result they spent less time exploring the whole task. The implication is that the cost and effort required to create an adaptive remediation training program may not be justified in a cognitively complex task like the planning task used in the experiment.

Transfer from a simulation environment to a live robotic environment: are certain demographics better?

The ability to remotely operate an unmanned vehicle while simultaneously looking for suspicious targets and then classifying those targets is not a trivial skill. This study looked at different training approaches to make better use of simulation as a first training step. When transferring to a live environment, the operators could be grouped into two categories according to whether they passed live training criteria or not. There were clear performance differences between these groups. The group that failed to pass criteria had poorer performance overall, more SA errors, and spent more time in training. Post-hoc analysis showed differences in the demographics between those who passed and those that did not. Male participants and younger participants were more likely to achieve criteria. There were no differences in gaming experience and perceived sense of direction.