John Butter

Mastery Learning of Advanced Cardiac Life Support Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice

AbstractBACKGROUND: Internal medicine residents must be competent in advanced cardiac life support (ACLS) for board certification. OBJECTIVE: To use a medical simulator to assess postgraduate year 2 (PGY-2) residents’ baseline proficiency in ACLS scenarios and evaluate the impact of an educational intervention grounded in deliberate practice on skill development to mastery standards. DESIGN: Pretest-posttest design without control group. After baseline evaluation, residents received 4, 2-hour ACLS education sessions using a medical simulator. Residents were then retested. Residents who did not achieve a research-derived minimum passing score (MPS) on each ACLS problem had more deliberate practice and were retested until the MPS was reached. PARTICIPANTS: Forty-one PGY-2 internal medicine residents in a university-affiliated program. MEASUREMENTS: Observational checklists based on American Heart Association (AHA) guidelines with interrater and internal consistency reliability estimates; deliberate practice time needed for residents to achieve minimum competency standards; demographics; United States Medical Licensing Examination Step 1 and Step 2 scores; and resident ratings of program quality and utility. RESULTS: Performance improved significantly after simulator training. All residents met or exceeded the mastery competency standard. The amount of practice time needed to reach the MPS was a powerful (negative) predictor of posttest performance. The education program was rated highly. CONCLUSIONS: A curriculum featuring deliberate practice dramatically increased the skills of residents in ACLS scenarios. Residents needed different amounts of training time to achieve minimum competency standards. Residents enjoy training, evaluation, and feedback in a simulated clinical environment. This mastery learning program and other competency-based efforts illustrate outcome-based medical education that is now prominent in accreditation reform of residency education.

Simulation-Based Training of Internal Medicine Residents in Advanced Cardiac Life Support Protocols: A Randomized Trial

Background: Internal medicine residents must be competent in Advanced Cardiac Life Support (ACLS) for board certification. Purpose: The purpose was to use a medical simulator to assess baseline proficiency in ACLS and determine the impact of an intervention on skill development. Method: This was a randomized trial with wait-list controls. After baseline evaluation in all residents, the intervention group received 4 education sessions using a medical simulator. All residents were then retested. After crossover, the wait-list group received the intervention, and residents were tested again. Performance was assessed by comparison to American Heart Association guidelines for treatment of ACLS conditions with interrater and internal consistency reliability estimates. Results: Performance improved significantly after simulator training. No improvement was detected as a function of clinical experience alone. The educational program was rated highly.

A longitudinal study of internal medicine residents' retention of advanced cardiac life support skills

Background Internal medicine residents must be competent in Advanced Cardiac Life Support (ACLS) for board certification. Traditional ACLS courses have limited ability to enable residents to achieve and maintain skills. Educational programs featuring reliable measurements and improved retention of skills would be useful for residency education. Method We developed a training program using a medical simulator, small-group teaching and deliberate practice. Residents received traditional ACLS education and subsequently participated in four two-hour educational sessions using the simulator. Resident performance in six simulated ACLS scenarios was assessed using a standardized checklist. Results After the program, resident ACLS skill improved significantly. The cohort was followed prospectively for 14 months and the skills did not decay. Conclusions Use of a simulation-based educational program enabled us to achieve and maintain high levels of resident performance in simulated ACLS events. Given the limitations of traditional methods to train, assess and maintain competence, simulation technology can be a useful adjunct in high-quality ACLS education.

Does ultrasound training boost Year 1 medical student competence and confidence when learning abdominal examination

Objectives  Learning to perform physical examination of the abdomen is a challenge for medical students. Medical educators need to find engaging, effective tools to help students acquire competence and confidence in abdominal examination techniques. This study evaluates the added value of ultrasound training when Year 1 medical students learn abdominal examination.

Comparison of two standard-setting methods for advanced cardiac life support training.

Background This study used the Angoff and Hofstee standard-setting methods to derive minimum passing scores for six advanced cardiac life support (ACLS) procedures. Method An expert panel provided item-based (Angoff) and group-based (Hofstee) judgments about six ACLS performance checklists on two occasions separated by ten weeks. Interrater reliabilities and test-retest reliability (stability) of the judgments were calculated. Derived ACLS passing standards are compared to historical ACLS performance data from two groups of ACLS-trained internal medicine residents. Results Both the Angoff and Hofstee standard-setting methods produced reliable and stable data. Hofstee minimum passing scores (MPSs) were uniformly more stringent than Angoff MPSs. Interpretation of historical ACLS performance data from medical residents shows the MPSs derived in this study would yield higher-than-expected failure rates. Conclusion Systematic standard setting for ACLS procedures is a necessary step toward the creation of mastery learning educational programs.

Graduating internal medicine residents' self-assessment and performance of advanced cardiac life support skills.

Internal medicine residents in the US must be competent to perform procedures including Advanced Cardiac Life Support (ACLS) to become board-eligible. Our aim was to determine if residents near graduation could assess their skills in ACLS procedures accurately. Participants were 40 residents in a university-based training program. Self-assessments of confidence in managing six ACLS scenarios were measured on a 0 (very low) to 100 (very high) scale. These were compared to reliable observational ratings of residents’ performance on a high-fidelity simulator using published treatment protocols. Residents expressed strong self-confidence about managing the scenarios. Residents’ simulator performance varied widely (range from 45% to 94%). Self-confidence assessments correlated poorly with performance (median r = 0.075). Self-assessment of performance by graduating internal medicine residents was not accurate in this study. The use of self-assessment to document resident competence in procedures such as ACLS is not a proxy for objective evaluation.

Simulation-based Mastery Learning Improves Cardiac Auscultation Skills in Medical Students

BackgroundCardiac auscultation is a core clinical skill. However, prior studies show that trainee skills are often deficient and that clinical experience is not a proxy for competence.ObjectiveTo describe a mastery model of cardiac auscultation education and evaluate its effectiveness in improving bedside cardiac auscultation skills.DesignUntreated control group design with pretest and posttest.ParticipantsThird-year students who received a cardiac auscultation curriculum and fourth year students who did not.InterventionA cardiac auscultation curriculum consisting of a computer tutorial and a cardiac patient simulator. All third-year students were required to meet or exceed a minimum passing score (MPS) set by an expert panel at posttest.MeasurementsDiagnostic accuracy with simulated heart sounds and actual patients.ResultsTrained third-year students (n = 77) demonstrated significantly higher cardiac auscultation accuracy compared to untrained fourth year students (n = 31) in assessment of simulated heart sounds (93.8% vs. 73.9%, p < 0.001) and with real patients (81.8% vs. 75.1%, p = 0.003). USMLE scores correlated modestly with a computer-based multiple choice assessment using simulated heart sounds but not with bedside skills on real patients.ConclusionsA cardiac auscultation curriculum consisting of deliberate practice with a computer-based tutorial and a cardiac patient simulator resulted in improved assessment of simulated heart sounds and more accurate examination of actual patients.

Hiv-associated nephropathy occurring before HIV antibody seroconversion.

It currently is thought that human immunodeficiency virus-associated nephropathy (HIVAN) occurs late in the course of HIV infection. Although HIVAN may be the presenting manifestation of acquired immunodeficiency syndrome (AIDS), it usually occurs after a prolonged period of viral infection often associated with high levels of HIV viremia. The patient described here developed HIVAN as a manifestation of acute retroviral syndrome. A 41-year-old black man presented with nephrotic range proteinuria, renal insufficiency, and acute gastrointestinal and pulmonary symptoms. He recently had been treated for primary syphilis. Two HIV serologic tests, performed 3 months apart, were negative. Renal biopsy was consistent with HIVAN. After the biopsy, the patient was discovered to have more than 700,000 viral copies per mL in his blood. CD4(+) count was greater than 500/mm(3). Six weeks later, enzyme-linked immunosorbent assay and Western blot analyses for HIV antibody became positive. HIVAN can occur early in the course of HIV infection, even during acute infection before seroconversion, and prolonged exposure to virus is not necessary for this renal involvement to occur in the susceptible host.

Progress toward improving medical school graduates' skills via a "boot camp" curriculum.

Introduction Medical school graduates are expected to possess a broad array of clinical skills. However, concerns have been raised regarding the preparation of medical students to enter graduate medical education. We designed a simulation-based “boot camp” experience for students entering internal medicine residency and compared medical student performance with the performance of historical controls who did not complete boot camp. Methods This was a cohort study of a simulation-based boot camp educational intervention. Twenty medical students completed 2 days (16 hours) of small group simulation-based education and individualized feedback and skills assessment. Skills included (a) physical examination techniques (cardiac auscultation); technical procedures including (b) paracentesis and (c) lumbar puncture; (d) recognition and management of patients with life-threatening conditions (intensive care unit clinical skills/mechanical ventilation); and (e) communication with patients and families (code status discussion). Student posttest scores were compared with baseline scores of postgraduate year 1 (PGY-1) historical controls to assess the effectiveness of the intervention. Results Boot camp–trained medical students performed significantly better than PGY-1 historical controls on each simulated skill (P < 0.01). Results remained significant after controlling for age, sex, and US Medical Licensing Examination step 1 and 2 scores (P < 0.001). Conclusions A 2-day simulation-based boot camp for graduating medical students boosted a variety of clinical skills to levels significantly higher than PGY-1 historical controls. Simulation-based education shows promise to help ensure that medical school graduates are prepared to begin postgraduate training.

E-learning and deliberate practice for oral case presentation skills: A randomized trial

Background: Oral case presentations are critical for patient care and student assessment. The best method to prepare early medical students for oral presentations is unknown. Aim: We aimed to develop and evaluate a curriculum of on-line learning and deliberate practice to improve pre-clinical students’ case presentation skills. Methods: We developed a web-based, interactive curriculum emphasizing conciseness and clinical reasoning. Using a waitlist control design, we randomly assigned groups of second-year students to receive the curriculum in December 2010 or in April 2011. We evaluated their presentations at three time points. We also examined the performance of an untrained class of students as a historical comparison. Results: We evaluated 132 second-year medical students at three time points. After the curriculum, mean scores of the intervention students improved from 60.2% to 70.1%, while scores of the waitlist control students improved less, from 61.8% to 64.5% (p < 0.01 for between-group difference in improvement). Once all students had received the curriculum, mean scores for the intervention and waitlist control students rose to 77.8% and 78.4%, respectively, compared to 68.1% for the untrained comparison students (p < 0.0001 compared to all curriculum students). Conclusion: An on-line curriculum followed by deliberate practice improved students’ oral presentation skills.