William C. McGaghie

Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review*

Review date: 1969 to 2003, 34 years. Background and context: Simulations are now in widespread use in medical education and medical personnel evaluation. Outcomes research on the use and effectiveness of simulation technology in medical education is scattered, inconsistent and varies widely in methodological rigor and substantive focus. Objectives: Review and synthesize existing evidence in educational science that addresses the question, ‘What are the features and uses of high-fidelity medical simulations that lead to most effective learning?’. Search strategy: The search covered five literature databases (ERIC, MEDLINE, PsycINFO, Web of Science and Timelit) and employed 91 single search terms and concepts and their Boolean combinations. Hand searching, Internet searches and attention to the ‘grey literature’ were also used. The aim was to perform the most thorough literature search possible of peer-reviewed publications and reports in the unpublished literature that have been judged for academic quality. Inclusion and exclusion criteria: Four screening criteria were used to reduce the initial pool of 670 journal articles to a focused set of 109 studies: (a) elimination of review articles in favor of empirical studies; (b) use of a simulator as an educational assessment or intervention with learner outcomes measured quantitatively; (c) comparative research, either experimental or quasi-experimental; and (d) research that involves simulation as an educational intervention. Data extraction: Data were extracted systematically from the 109 eligible journal articles by independent coders. Each coder used a standardized data extraction protocol. Data synthesis: Qualitative data synthesis and tabular presentation of research methods and outcomes were used. Heterogeneity of research designs, educational interventions, outcome measures and timeframe precluded data synthesis using meta-analysis. Headline results: Coding accuracy for features of the journal articles is high. The extant quality of the published research is generally weak. The weight of the best available evidence suggests that high-fidelity medical simulations facilitate learning under the right conditions. These include the following: providing feedback—51 (47%) journal articles reported that educational feedback is the most important feature of simulation-based medical education; repetitive practice—43 (39%) journal articles identified repetitive practice as a key feature involving the use of high-fidelity simulations in medical education; curriculum integration—27 (25%) journal articles cited integration of simulation-based exercises into the standard medical school or postgraduate educational curriculum as an essential feature of their effective use; range of difficulty level—15 (14%) journal articles address the importance of the range of task difficulty level as an important variable in simulation-based medical education; multiple learning strategies—11 (10%) journal articles identified the adaptability of high-fidelity simulations to multiple learning strategies as an important factor in their educational effectiveness; capture clinical variation—11 (10%) journal articles cited simulators that capture a wide variety of clinical conditions as more useful than those with a narrow range; controlled environment—10 (9%) journal articles emphasized the importance of using high-fidelity simulations in a controlled environment where learners can make, detect and correct errors without adverse consequences; individualized learning—10 (9%) journal articles highlighted the importance of having reproducible, standardized educational experiences where learners are active participants, not passive bystanders; defined outcomes—seven (6%) journal articles cited the importance of having clearly stated goals with tangible outcome measures that will more likely lead to learners mastering skills; simulator validity—four (3%) journal articles provided evidence for the direct correlation of simulation validity with effective learning. Conclusions: While research in this field needs improvement in terms of rigor and quality, high-fidelity medical simulations are educationally effective and simulation-based education complements medical education in patient care settings.

A critical review of simulation-based medical education research: 2003-2009.

Objectives  This article reviews and critically evaluates historical and contemporary research on simulation‐based medical education (SBME). It also presents and discusses 12 features and best practices of SBME that teachers should know in order to use medical simulation technology to maximum educational benefit.

Does Simulation-based Medical Education with Deliberate Practice Yield Better Results than Traditional Clinical Education? A Meta-Analytic Comparative Review of the Evidence

Purpose This article presents a comparison of the effectiveness of traditional clinical education toward skill acquisition goals versus simulation-based medical education (SBME) with deliberate practice (DP). Method This is a quantitative meta-analysis that spans 20 years, 1990 to 2010. A search strategy involving three literature databases, 12 search terms, and four inclusion criteria was used. Four authors independently retrieved and reviewed articles. Main outcome measures were extracted to calculate effect sizes. Results Of 3,742 articles identified, 14 met inclusion criteria. The overall effect size for the 14 studies evaluating the comparative effectiveness of SBME compared with traditional clinical medical education was 0.71 (95% confidence interval, 0.65–0.76; P < .001). Conclusions Although the number of reports analyzed in this meta-analysis is small, these results show that SBME with DP is superior to traditional clinical medical education in achieving specific clinical skill acquisition goals. SBME is a complex educational intervention that should be introduced thoughtfully and evaluated rigorously at training sites. Further research on incorporating SBME with DP into medical education is needed to amplify its power, utility, and cost-effectiveness.

Use of Simulation-Based Education to Reduce Catheter-Related Bloodstream Infections

BACKGROUND Simulation-based education improves procedural competence in central venous catheter (CVC) insertion. The effect of simulation-based education in CVC insertion on the incidence of catheter-related bloodstream infection (CRBSI) is unknown. The aim of this study was to determine if simulation-based training in CVC insertion reduces CRBSI. METHODS This was an observational education cohort study set in an adult intensive care unit (ICU) in an urban teaching hospital. Ninety-two internal medicine and emergency medicine residents completed a simulation-based mastery learning program in CVC insertion skills. Rates of CRBSI from CVCs inserted by residents in the ICU before and after the simulation-based educational intervention were compared over a 32-month period. RESULTS There were fewer CRBSIs after the simulator-trained residents entered the intervention ICU (0.50 infections per 1000 catheter-days) compared with both the same unit prior to the intervention (3.20 per 1000 catheter-days) (P = .001) and with another ICU in the same hospital throughout the study period (5.03 per 1000 catheter-days) (P = .001). CONCLUSIONS An educational intervention in CVC insertion significantly improved patient outcomes. Simulation-based education is a valuable adjunct in residency education.

What is feedback in clinical education

Objective  Feedback is important in clinical education. However, the medical education literature provides no consensual definition of feedback. The aim of this study is to propose a consensual, research‐based, operational definition of feedback in clinical education. An operational definition is needed for educational practice and teacher training, and for research into the effectiveness of different types of feedback.

Cognitive, social and environmental sources of bias in clinical performance ratings.

Background: Global ratings based on observing convenience samples of clinical performance form the primary basis for appraising the clinical competence of medical students, residents, and practicing physicians. This review explores cognitive, social, and environmental factors that contribute unwanted sources of score variation (bias) to clinical performance evaluations. Summary: Raters have a 1 or 2-dimensional conception of clinical performance and do not recall details. Good news is reported more quickly and fully than bad news, leading to overly generous performance evaluations. Training has little impact on accuracy and reproducibility of clinical performance ratings. Conclusions: Clinical performance evaluation systems should assure broad, systematic sampling of clinical situations; keep rating instruments short; encourage immediate feedback for teaching and learning purposes; encourage maintenance of written performance notes to support delayed clinical performance ratings; give raters feedback about their ratings; supplement formal with unobtrusive observation; make promotion decisions via group review; supplement traditional observation with other clinical skills measures (e.g., Objective Structured Clinical Examination); encourage rating of specific performances rather than global ratings; and establish the meaning of ratings in the manner used to set normal limits for clinical diagnostic investigations.

Mastery Learning of Advanced Cardiac Life Support Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice

AbstractBACKGROUND: Internal medicine residents must be competent in advanced cardiac life support (ACLS) for board certification. OBJECTIVE: To use a medical simulator to assess postgraduate year 2 (PGY-2) residents’ baseline proficiency in ACLS scenarios and evaluate the impact of an educational intervention grounded in deliberate practice on skill development to mastery standards. DESIGN: Pretest-posttest design without control group. After baseline evaluation, residents received 4, 2-hour ACLS education sessions using a medical simulator. Residents were then retested. Residents who did not achieve a research-derived minimum passing score (MPS) on each ACLS problem had more deliberate practice and were retested until the MPS was reached. PARTICIPANTS: Forty-one PGY-2 internal medicine residents in a university-affiliated program. MEASUREMENTS: Observational checklists based on American Heart Association (AHA) guidelines with interrater and internal consistency reliability estimates; deliberate practice time needed for residents to achieve minimum competency standards; demographics; United States Medical Licensing Examination Step 1 and Step 2 scores; and resident ratings of program quality and utility. RESULTS: Performance improved significantly after simulator training. All residents met or exceeded the mastery competency standard. The amount of practice time needed to reach the MPS was a powerful (negative) predictor of posttest performance. The education program was rated highly. CONCLUSIONS: A curriculum featuring deliberate practice dramatically increased the skills of residents in ACLS scenarios. Residents needed different amounts of training time to achieve minimum competency standards. Residents enjoy training, evaluation, and feedback in a simulated clinical environment. This mastery learning program and other competency-based efforts illustrate outcome-based medical education that is now prominent in accreditation reform of residency education.

Simulation-Based Training of Internal Medicine Residents in Advanced Cardiac Life Support Protocols: A Randomized Trial

Background: Internal medicine residents must be competent in Advanced Cardiac Life Support (ACLS) for board certification. Purpose: The purpose was to use a medical simulator to assess baseline proficiency in ACLS and determine the impact of an intervention on skill development. Method: This was a randomized trial with wait-list controls. After baseline evaluation in all residents, the intervention group received 4 education sessions using a medical simulator. All residents were then retested. After crossover, the wait-list group received the intervention, and residents were tested again. Performance was assessed by comparison to American Heart Association guidelines for treatment of ACLS conditions with interrater and internal consistency reliability estimates. Results: Performance improved significantly after simulator training. No improvement was detected as a function of clinical experience alone. The educational program was rated highly.

Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit.

Introduction: Interventions to reduce preventable complications such as catheter-related bloodstream infections (CRBSI) can also decrease hospital costs. However, little is known about the cost-effectiveness of simulation-based education. The aim of this study was to estimate hospital cost savings related to a reduction in CRBSI after simulation training for residents. Methods: This was an intervention evaluation study estimating cost savings related to a simulation-based intervention in central venous catheter (CVC) insertion in the Medical Intensive Care Unit (MICU) at an urban teaching hospital. After residents completed a simulation-based mastery learning program in CVC insertion, CRBSI rates declined sharply. Case-control and regression analysis methods were used to estimate savings by comparing CRBSI rates in the year before and after the intervention. Annual savings from reduced CRBSIs were compared with the annual cost of simulation training. Results: Approximately 9.95 CRBSIs were prevented among MICU patients with CVCs in the year after the intervention. Incremental costs attributed to each CRBSI were approximately

Effect of practice on standardised learning outcomes in simulation-based medical education

82,000 in 2008 dollars and 14 additional hospital days (including 12 MICU days). The annual cost of the simulation-based education was approximately