Nicholas P. W. Coe

Boot Camp: Educational Outcomes After 4 Successive Years of Preparatory Simulation- Based Training at Onset of Internship

PURPOSE Preparatory training for new trainees beginning residency has been used by a variety of programs across the country. To improve the clinical orientation process for our new postgraduate year (PGY)-1 residents, we developed an intensive preparatory training curriculum inclusive of cognitive and procedural skills, training activities considered essential for early PGY-1 clinical management. We define our surgical PGY-1 Boot Camp as preparatory simulation-based training implemented at the onset of internship for introduction of skills necessary for basic surgical patient problem assessment and management. This orientation process includes exposure to simulated patient care encounters and technical skills training essential to new resident education. We report educational results of 4 successive years of Boot Camp training. Results were analyzed to determine if performance evidenced at onset of training was predictive of later educational outcomes. METHODS Learners were PGY-1 residents, in both categorical and preliminary positions, at our medium-sized surgical residency program. Over a 4-year period, from July 2007 to July 2010, all 30 PGY-1 residents starting surgical residency at our institution underwent specific preparatory didactic and skills training over a 9-week period. This consisted of mandatory weekly 1-hour and 3-hour sessions in the Simulation Center, representing a 4-fold increase in time in simulation laboratory training compared with the remainder of the year. Training occurred in 8 procedural skills areas (instrument use, knot-tying, suturing, laparoscopic skills, airway management, cardiopulmonary resuscitation, central venous catheter, and chest tube insertion) and in simulated patient care (shock, surgical emergencies, and respiratory, cardiac, and trauma management) using a variety of high- and low-tech simulation platforms. Faculty and senior residents served as instructors. All educational activities were structured to include preparatory materials, pretraining briefing sessions, and immediate in-training or post-training review and debriefing. Baseline cognitive skills were assessed with written tests on basic patient management. Post-Boot Camp tests similarly evaluated cognitive skills. Technical skills were assessed using a variety of task-specific instruments, and expressed as a mean score for all activities for each resident. All measurements were expressed as percent (%) best possible score. Cognitive and technical performance in Boot Camp was compared with subsequent clinical and core curriculum evaluations including weekly quiz scores, annual American Board of Surgery In-Training Examination (ABSITE) scores, program in-training evaluations (New Innovations, Uniontown, Ohio), and operative assessment instrument scores (OP-Rate, Baystate Medical Center, Springfield, Massachusetts) for the remainder of the PGY-1 year. RESULTS Performance data were available for 30 PGY-1 residents over 4 years. Baseline cognitive skills were lower for the first year of Boot Camp as compared with subsequent years (71 ± 13, 83 ± 9, 84 ± 11, and 86 ± 6, respectively; p = 0.028, analysis of variance; ANOVA). Performance improved between pretests and final testing (81 ± 11 vs 89 ± 7; p < 0.001 paired t test). There was statistically significant correlation between Boot Camp final cognitive test results and American Board of Surgery In-Training Examination scores (p = 0.01; n = 22), but not quite significant for weekly curriculum quiz scores (p = 0.055; n = 22) and New Innovations cognitive assessments (p = 0.09; n = 25). Statistically significant correlation was also noted between Boot Camp mean overall skills and New Innovations technical skills assessments (p = 0.002; n = 25) and OP-Rate assessments (p = 0.01; n = 12). CONCLUSIONS Individual simulation-based Boot Camp performance scores for cognitive and procedural skills assessments in PGY-1 residents correlate with subjective and objective clinical performance evaluations. This concurrent correlation with multiple traditional evaluation methods used to express competency in our residency program supports the use of Boot Camp performance measures as needs assessment tools as well as adjuncts to cumulative resident evaluation data.

A Surgery Oral Examination: Interrater Agreement and the Influence of Rater Characteristics.

BACKGROUND. Poor interrater reliability is a common objection to the use of oral examinations. METHOD. In 1990 the authors measured the agreement of 140 U.S. and Canadian surgical raters and the influences, if any, of age, years in practice, and experience as an examiner on individual oral examination scores. Eight actor examinees memorized transcripts of actual oral examinations and were videotaped using a single examiner. Examinee verbal style, dress, content of answers, and gender were purposefully adjusted. A repeated-measures analysis of variance was used for data analysis. RESULTS. Three aspects of examinee performance influenced scores (verbal style, dress, and content of answers). No rater characteristic significantly affected scores. Raters showed high agreement (86%) when rating “good” performances but less agreement (67%) when rating “poor” performances. CONCLUSION. The oral examination scores were not influenced by rater selection. The raters ranked good performances more consistently than poor performances. Therefore, more than one examiner appears necessary to confirm a poor performance during an examination.

The effect of improving communication competency on the certifying examination of the American board of surgery

BACKGROUND Since 1991 the authors have offered a course that identifies content deficits, but only provides instruction directed at improving verbal and nonverbal behaviors. We report the outcome of this 10-year effort as success on the certifying examination of the American Board of Surgery between 1991 and 2001. METHODS Sixteen 5-day courses were scheduled over 10 years. Participants included those who had not taken the oral examination or had failed at least once and invited senior faculty (n = 26). Sites were chosen to replicate the actual examination setting. RESULTS There were 122 participants, with follow-up data available on 88. Success in the certifying examination after completing the course is 96 percent. CONCLUSIONS Evaluation of communication deficits and training to improve them is strongly associated with success. Clearly, this course is effective at identifying communication behaviors that are interfering with success on the certifying examination of the American Board of Surgery.

Evaluation of gastrointestinal hemorrhage in patients with neurofibromatosis.

Over a 3-year period, two patients with neurofibromatosis were referred to our medical center for evaluation of repeated episodes of melena. Upper endoscopy was unrevealing in each case, as was colonoscopy. Arteriography during active hemorrhage was helpful in localizing the source of bleeding in one patient but not in the other. The source of bleeding in each patient was obvious at surgical exploration. Large neurofibromas protruded from the seorsal surface of a short region of jejunum in both cases and an additional segment of ileum in one case. Hemorrhage had occurred as a result of erosion of mucosa stretched over these tumors. Local resection of the involved segments produced long-term control of the hemorrhage. Since these tumors were grossly visible on the serosal surface of the involved intestinal segments, laparoscopic evaluation could have been used to hasten diagnosis in each case.

A time to listen

As methods of health care delivery, and advances in technology change our lives and practices, an essential element of personal and professional relationships has become neglected. We have stopped listening to each other, to our patients, and to ourselves; we have lost the art of communication. The essential aspects of optimal communication and the power of nonverbal signals are reviewed. Only by recognizing the importance of communication in surgical education, practice, and in fact in all aspects of daily life, will this encroaching societal deafness be rebuffed.

To drain or not to drain; still a valid question?

S & COMMENTARY