Sydney McQueen

Reflections on Competency-Based Education and Training for Surgical Residents

Although a number of surgical training institutions have started to adopt competency-based education (CBE) frameworks for training, the debate about the value of this model continues. Some proponents regard CBE as a method of guaranteeing residents competence, whereas others consider CBE to be reductive and lacking the richness in experiences that the traditional model offers. In this article, we reflect on CBE and review some salient attempts to implement CBE in surgical education. We identify challenges facing postgraduate surgical education, some of which are motivating educators to consider incorporating CBE into their curricula. We look at some purported advantages and disadvantages of CBE and describe initial reports from CBE programs currently being developed.

Simulation for Teaching Orthopaedic Residents in a Competency-based Curriculum: Do the Benefits Justify the Increased Costs?

BackgroundAlthough simulation-based training is becoming widespread in surgical education and research supports its use, one major limitation is cost. Until now, little has been published on the costs of simulation in residency training. At the University of Toronto, a novel competency-based curriculum in orthopaedic surgery has been implemented for training selected residents, which makes extensive use of simulation. Despite the benefits of this intensive approach to simulation, there is a need to consider its financial implications and demands on faculty time.Questions/purposesThis study presents a cost and faculty work-hours analysis of implementing simulation as a teaching and evaluation tool in the University of Toronto’s novel competency-based curriculum program compared with the historic costs of using simulation in the residency training program.MethodsAll invoices for simulation training were reviewed to determine the financial costs before and after implementation of the competency-based curriculum. Invoice items included costs for cadavers, artificial models, skills laboratory labor, associated materials, and standardized patients. Costs related to the surgical skills laboratory rental fees and orthopaedic implants were waived as a result of special arrangements with the skills laboratory and implant vendors. Although faculty time was not reimbursed, faculty hours dedicated to simulation were also evaluated. The academic year of 2008 to 2009 was chosen to represent an academic year that preceded the introduction of the competency-based curriculum. During this year, 12 residents used simulation for teaching. The academic year of 2010 to 2011 was chosen to represent an academic year when the competency-based curriculum training program was functioning parallel but separate from the regular stream of training. In this year, six residents used simulation for teaching and assessment. The academic year of 2012 to 2013 was chosen to represent an academic year when simulation was used equally among the competency-based curriculum and regular stream residents for teaching (60 residents) and among 14 competency-based curriculum residents and 21 regular stream residents for assessment.ResultsThe total costs of using simulation to teach and assess all residents in the competency-based curriculum and regular stream programs (academic year 2012–2013) (CDN 155,750, USD 158,050) were approximately 15 times higher than the cost of using simulation to teach residents before the implementation of the competency-based curriculum (academic year 2008–2009) (CDN 10,090, USD 11,140). The number of hours spent teaching and assessing trainees increased from 96 to 317 hours during this period, representing a threefold increase.ConclusionsAlthough the financial costs and time demands on faculty in running the simulation program in the new competency-based curriculum at the University of Toronto have been substantial, augmented learner and trainer satisfaction has been accompanied by direct evidence of improved and more efficient learning outcomes.Clinical RelevanceThe higher costs and demands on faculty time associated with implementing simulation for teaching and assessment must be considered when it is used to enhance surgical training.

Student-led learning: a new teaching paradigm for surgical skills

BACKGROUND Competency-based education and simulation are being used more frequently in surgical skills curricula. We explored a novel student-led learning paradigm, which allows trainees to become more active participants in the learning process while maintaining expert guidance and supervision. METHODS Twelve first-year orthopedic residents were randomized to either a student-led (SL) or a traditional instructor-led group during an intensive, month-long, laboratory-based technical skills training course. A rigorous qualitative-description approach was used for analysis. RESULTS Four prominent themes emerged: instructional style, feedback, peer and instructor collaboration, and self-efficacy. Compared with the instructor-led group, there was more peer assistance, feedback, collaboration, and hands-on and active learning observed in the SL group. CONCLUSIONS The flexible and socially rich nature of the SL learning environment may aid in development of both technical and nontechnical skills early in residency and ultimately privilege later clinical learning.

Examining the barriers to meaningful assessment and feedback in medical training

BACKGROUND Recent reports from both accreditation bodies in North America highlight problems with current assessment practices in postgraduate medical training. Previous work has shown that educators might be reluctant to report poor performance or fail underperforming trainees. This study explores the barriers perceived by medical educators to providing more meaningful assessment and feedback to trainees. METHODS Semistructured interviews were conducted with 22 physician educators. Interviews were audiotaped and transcribed verbatim. Three researchers analyzed the transcripts using a grounded theory approach. RESULTS Participants expressed a reluctance to provide poor assessments or feedback to trainees. Fifty-five percent of the participants reported passing trainees who could have benefited from additional training. Our data revealed a number of barriers which may account for these findings. Implementing more frequent formative assessments could help educators more effectively evaluate trainees and provide feedback, although a shift in the culture of medicine may be required. CONCLUSION It is imperative that the barriers to effective assessment and feedback identified in this study be addressed to improve postgraduate medical training and enhance patient care.

Bridging the Gap: Theoretical Principles Behind Surgical Boot Camps

The transition from medical school to residency is often considered the most difficult year for both teachers and learners. Previous work suggests incorporating a boot camp at the onset of residency can be a highly effective mechanism for easing the transition between medical school and residency by improving trainees’ medical knowledge, confidence, and procedural and technical skills before they start to care for patients. As other programs look to adopt this training method, medical educators must understand how novice trainees acquire new skills and thus why boot camps are effective. This chapter reviews theoretical principles from Motor Learning, Cognitive Psychology, and Education Science to explain the attentional demands of novice skill acquisition, what practice strategies promote novice learning, and how these can be applied in a boot camp setting. The goal of this chapter is to provide educators with information on how to be successful in the development and implementation of their own boot camps.

Using art for the development of teamwork and communication skills among health professionals: a literature review

Abstract Background Interpersonal skills such as teamwork and communication are essential to good health care. Methods: Four reference databases were searched using relevant keyword combinations in order to examine how the visual and performing arts have been used to develop teamwork and communication skills among health professionals and what the outcomes were. Reference list checking was also conducted in order to identify other relevant articles not captured in the initial search strategy. Results: Many of the studies that we reviewed revealed that after participation in the arts, participants felt positively about the experience, had a greater awareness of the importance of teamwork and communication skills in their professional practice, and reported general improvements in these skills. Conclusions: While the initial results of this review seem promising, a concerted effort to conduct more methodologically sound studies on various art forms and their effect on the development of health professionals’ interpersonal skills is required.

Board 214 - Program Innovations Abstract Title: Surgical Preparatory Camp

Introduction/Background Recent developments within the medical education system are having a stifling effect on surgical training. Reduced work hours1-3 for surgical trainees, coupled with an increased focus on patient safety inevitably Results in fewer educational opportunities and decreased quality of learning experiences for new trainees. Today’s surgical educators are actively searching for learning opportunities that extend beyond the operating room. Simulation-based programs allow practice at an individual pace in a risk-free environment and present an important supplement to traditional clinical teaching.4-7 One such program is the Surgical Prep Camp (SPC), a new simulation-based course that aims to prepare all incoming surgical trainees at the University of Toronto for their residency. The Surgical Prep Camp was developed by experts in surgical education and skill acquisition and has evolved from the award-winning Toronto Orthopaedic Boot Camp program.8-9 Methods All 54 first year surgical trainees participated in SPC, which took place at the onset of their residency. The program was delivered at the University of Toronto Surgical Skills Centre at Mount Sinai Hospital. The Surgical Prep Camp program focused on core skills that Program Directors agreed were essential for all surgical residents and created a unique learning environment to accelerate the development of these skills. For two weeks, residents from all specialities practiced suturing, central line insertion, surgical airway and other fundamental skills. The program also included didactic sessions encompassing key concepts in surgery. These two weeks were followed by an additional training period (the length of which was determined by each surgical division) which focused on developing speciality-specific skills. Trainees’ progress was closely monitored. Throughout the course, the residents and instructors were encouraged to complete interim skills assessments using electronic progress logs. Upon completion of the SPC program, trainees completed a skills examination. A retention examination will be conducted in eight months time to determine how well the skills persist. Additionally, extensive feedback was collected to help improve future iterations of SPC. We present preliminary data from our early experiences with SPC. Skills examination performance Results were impressive with a normalized mean total checklist score of 0.85 (0.15) (out of a possible 1). Nonetheless, two tasks (chest tube insertion and tracheostomy) proved to be particularly challenging. We also present detailed feedback from our residents and staff which will offer insight into the program’s development and implementation. Results: Conclusion The ultimate goal of SPC is to enhance patient safety and produce more competent surgeons who are better prepared for clinical practice. This program provides new trainees with a sound foundation upon which they can build their technical and clinical skills. Beyond providing an advantage at the beginning of training, the program seeks to instill effective learning habits that allow residents to engage in meaningful deliberate practice. Focusing on technical skills at the beginning of residency aims to allow trainees to advance to more complex tasks earlier within residency, which provides a much richer educational experience than has previously been possible.10 This is achieved in a manner which does not produce significant additional load on faculty since the teaching is shared between staff surgeons, fellows, senior residents and members of the allied healthcare team. Both this program and its predecessor have generated much interest in the surgical education community. Early evidence suggests that simulation-based programs can have a profound positive impact on residency training across all specialities. We believe that more widespread adoption of such programs will follow, helping to ensure that our future surgeons are better prepared to face the challenges that lie ahead. References 1. Calman KC, Temple JG, Naysmith R, Cairncross RG and Bennett SJ: Reforming higher specialist training in the United Kingdom - a step along the continuum of medical education. Med Educ 1999; 33: 28–33. 2. Pickersgill T: The European working time directive for doctors in training. BMJ 2001; 323(7324):1266. 3. Irani JL et al.: Surgical residents’ perceptions of the effects of the ACGME duty hour requirements 1 year after implementation. Surgery 2005; 138(2):246-253. 4. Carter BN: The fruition of Halsted’s concept of surgical training. Surgery 1952; 32(3): 518–527. 5. Reznick RK, MacRae H: Medical education - Teaching surgical skills - Changes in the wind. N Engl J Med 2006; 355(25): 2664–2669. 6. Ericsson KA, Krampe RT, Tesch-romer C: The role of deliberate practice in the acquisition of expert performance. Psychological Review 1993; 100(3): 363–406. 7. Ahlberg G, Enochsson L, Gallagher AG, et al.: Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007; 193(6):797–804. 8. Sonnadara RR, Van Vliet A, Safir O, et al.: Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course. Surgery 2011; 149(6): 745–749. 9. Sonnadara RR, Garbedian S, Safir O, et al.: Orthopaedic Boot Camp II: examining the retention rates of an intensive surgical skills course. Surgery 2012; 151(6): 803–7. 10. Sonnadara RR, Garbedian S, Safir O, Mui C, Mironova P, Nousiainen M, Ferguson P, Kraemer W, Alman B and Reznick R: Orthopaedic Boot Camp III: Examining the efficacy of self-regulated learning during an intensive laboratory-based surgical skills course. Surgery (In Press). Disclosures None.

Board 383 - Research Abstract Examining the Effects of a Student-Led Learning Paradigm in a Simulation-Based Surgical Skills Course (Submission #564)

Introduction/Background Recent changes in healthcare training, such as increasing demands on faculty time, reduced opportunities for teaching in the clinical setting and increasing awareness for patient safety, have led medical educators to rely more on simulation-based programs to supplement traditional clinical teaching.1–4 One such program is the Toronto Orthopaedic Boot Camp (TOBC), an intensive course designed to teach core surgical skills to incoming orthopaedic residents at the University of Toronto.5 Data from this program have revealed that it is possible to persistently advance the technical skills of first year residents to the level of senior residents for targeted tasks after just one month.6,7 However, technical skill development is only one aspect of surgical training and there is a recognized need to emphasize non-technical skills and promote professional identity formation.8,9 In the present study, we investigate the impact of implementing a student-led learning (SLL) paradigm on the early acquisition of non-technical skills. Methods Twelve incoming orthopaedic residents participated in this research, which was embedded in the month long TOBC skills course. Residents were randomly divided into two groups: six were taught using a new paradigm focused on supervised, student-led exploration and practice (SL group), while the other six were taught using a traditional, instructor-led paradigm (IL group).6,7 A typical day consisted of a didactic teaching session followed by an extended practical session in a simulation laboratory, with the instructional methodology governed by the assigned paradigm. Trained observers systematically documented the interactions that took place over the course of the month. Upon completion of the program, residents completed self-efficacy and exit questionnaires. Confidential interviews were conducted with the residents and primary instructors and senior faculty were polled for informal observations. Field observation notes were analyzed thematically. Interview responses were transcribed, checked for accuracy and analyzed for common themes by two independent raters. We then used a qualitative outcomes analysis approach to interpret our findings. Results Residents from both SL and IL groups were able to perform the targeted technical skills to an acceptable standard by the end of the TOBC program. We observed differences in the non-technical skills acquired by the two groups. SL participants exhibited more peer-to-peer interactions and both asked and were asked more interpretative questions. SL residents also reported feeling a greater sense of control over their learning than their IL counterparts. Interviews revealed that the IL group regarded their instructors as resources who could teach them the task at hand, whereas the SL group described their instructors as facilitators, offering guidance when asked but allowing them to explore skills and techniques for themselves. The instructors reported more cooperation and collegiality amongst the SL group. Conclusion Student-led learning was found to effectively promote and enhance interactions both between peers and also between trainees and their instructors. The SL paradigm also afforded residents pedagogic space to direct their own learning, which enabled them to feel more in control of the learning process. All of the traits promoted by the SL paradigm are known to contribute to effective learning,10–15 as well as the development of professional identity.16–19 The flexible and socially rich nature of the SL paradigm also appears to promote effective decision making, teamwork and leadership skills.9,20,21 Anecdotal reports from senior faculty suggest that this approach may lead to a better understanding of several core CanMEDS roles early in residency. Our data shows that well constructed simulation-based programs can privilege both technical and non-technical skills early in residency.7 Preliminary evidence suggests that such programs can better prepare new trainees for independent clinical practice, though further study is needed. References 1. Carter BN: The fruition of Halsted’s concept of surgical training. Surgery 1952; 32(3): 518–527. 2. Reznick RK, and MacRae H: Medical education - Teaching surgical skills - Changes in the wind. N Engl J Med 2006; 355(25): 2664–2669. 3. Ericsson KA, Krampe RT, and Tesch-romer C: The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993; 100(3): 363–406. 4. Ahlberg G, Enochsson L, Gallagher AG, et al: Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007; 193(6): 797–804. 5. Sonnadara RR, Van Vliet A, Safir O, et al: Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course. Surgery 2011; 149(6): 745–749. 6. Sonnadara RR, Garbedian S, Safir O, et al: Orthopaedic Boot Camp II: examining the retention rates of an intensive surgical skills course. Surgery 2012; 151(6): 803–807. 7. Sonnadara R, Garbedian S, Safir O, et al: Orthopaedic Boot Camp III: Examining the efficacy of student-regulated learning during an intensive laboratory-based surgical skills course. Surgery 2013; 154(1): 29-33. 8. Jarvis-Selinger S, Pratt DD, and Regehr G: Competency is not enough: integrating identity formation into the medical education discourse. Acad Med 2012; 87(9): 1185–1190. 9. Sharma B, Mishra A, Aggarwal R, and Grantcharov TP: Non-technical skills assessment in surgery. Surg Oncol 2010; 20(3): 169–177. 10. Knowles MS: Self-Directed Learning: A Guide for Learners and Teachers. New York, Association Press, 1975, pp 1-140. 11. Zimmerman BJ: A social cognitive view of self-regulated academic learning. J Educ Psychol 1989; 81(3): 329–339. 12. Bruner JS: Acts of Meaning. Cambridge, Harvard University Press, 1990, pp 71-102. 13. Zimmerman BJ: Self-Regulated Learning and Academic Achievement: An Overview. Educ Psychol 1990; 25: 3–17. 14. Brydges R, Carnahan H, Safir O, and Dubrowski A: How effective is self†guided learning of clinical technical skills? It’s all about process. Med Educ 2009; 43(6): 507–515. 15. Keetch KM, and Lee TD: The Effect of Self-Regulated and Experimenter-Imposed Practice Schedules on Motor Learning for Tasks of Varying Difficulty. Res Q Exercise Sport 2007; 78(5): 476–486. 16. Sandars J, Homer M, Pell G, and Croker T: Web 2.0 and social software: the medical student way of e-learning. Med Teach 2008; 30(3): 308–312. 17. Irby DM, Cooke M, and OʼBrien BC: Calls for Reform of Medical Education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med 2010; 85(2): 220–227. 18. Varga-Atkins T, Dangerfield P, and Brigden D. Developing professionalism through the use of wikis: A study with first-year undergraduate medical students. Med Teach 2010; 32(10): 824–829. 19. Cruess RL, and Cruess SR: Teaching professionalism: general principles. Med teach 2006; 28(3): 205–208. 20. Topping K: Peer Assessment Between Students in Colleges and Universities. Rev Educ Res 1998; 68(3): 249–276. 21. Topping KJ: Trends in Peer Learning. J Educ Psychol 2005; 25(6): 631–645. Disclosures Smith and Nephew, Inc. Zimmer, Inc.