Amy Nakajima

Competency-Based Medical Education: The Wave of the Future*

Competency-based medical education (CBME) is a new educational paradigm that will enable the medical education community to meet societal, patient, and learner needs of the 21st century. CBME offers a renewed commitment to both clinical and educational outcomes, a new focus on assessment and developmental milestones, a mechanism to promote a true continuum of medical education, and a method to promote learner-centred curricula in the context of accountability. Accountability is central to CBME, ensuring that graduating practitioners are well-rounded and competent to provide safe and effective patient care. The structure of CBME in obstetrics and gynaecology must be rooted in, and reflect, Canadian practice. Its development and implementation require an understanding of the principles that are the foundation of CBME, along with the involvement of the entire community of obstetricians and gynaecologists and other maternity care providers. We provide here an overview of the basic principles of teaching and learning and the theories underpinning CBME.

Competency-Based Medical Education: Developing a Framework for Obstetrics and Gynaecology

The development of a Canadian competency-based medical education (CBME) curriculum in obstetrics and gynaecology, slated to begin in 2017, must be rooted in, and aligned with, the principles of CanMEDS 2015 and Competence by Design. It must also reflect the unique realities of the practice of the specialty. The Dutch Society of Obstetrics and Gynaecology has been at the forefront of the movement to design and implement competency-based training for obstetrics and gynaecology. The Dutch curriculum represents a practical example of how such a program could be developed. Several CBME curricular initiatives have now also begun across Canada.

Global Women's Health Education in Canadian Obstetrics and Gynaecology Residency Programs: A Survey of Program Directors and Senior Residents

OBJECTIVE To become culturally competent practitioners with the ability to care and advocate for vulnerable populations, residents must be educated in global health priorities. In the field of obstetrics and gynaecology, there is minimal information about global womens health (GWH) education and interest within residency programs. We wished to determine within obstetrics and gynaecology residency programs across Canada: (1) current GWH teaching and support, (2) the importance of GWH to residents and program directors, and (3) the level of interest in a national postgraduate GWH curriculum. METHODS We conducted an online survey across Canada of obstetrics and gynaecology residency program directors and senior obstetrics and gynaecology residents. RESULTS Of 297 residents, 101 (34.0%) responded to the survey and 76 (26%) completed the full survey. Eleven of 16 program directors (68.8%) responded and 10/16 (62.5%) provided complete responses. Four of 11 programs (36.4%) had a GWH curriculum, 2/11 (18.2%) had a GWH budget, and 4/11 (36.4%) had a GWH chairperson. Nine of 10 program directors (90%) and 68/79 residents (86.1%) felt that an understanding of GWH issues is important for all Canadian obstetrics and gynaecology trainees. Only 1/10 program directors (10%) and 11/79 residents (13.9%) felt that their program offered sufficient education in these issues. Of residents in programs with a GWH curriculum, 12/19 (63.2%) felt that residents in their program who did not undertake an international elective would still learn about GWH, versus only 9/50 residents (18.0%) in programs without a curriculum (P < 0.001). CONCLUSION Obstetrics and gynaecology residents and program directors feel that GWH education is important for all trainees and is currently insufficient. There is a high level of interest in a national postgraduate GWH educational module.

Development of an undergraduate curriculum in obstetrical simulation

algorithm that customises the spacing and content of subsequent questions in order to maximise knowledge acquisition and retention for each learner. As a stand-alone educational intervention, spaced education has been shown in randomised trials to increase knowledge acquisition, boost learning retention and improve clinical behaviour. Why the idea was necessary A recent report by the US Department of Education found that online education which is blended with live courses is more effective than online educational interventions alone. It is not clear how spaced education can best be ‘blended’ with traditional forms of medical education to improve its efficacy and acceptability. We conducted a pilot study to determine whether spaced education can be an effective supplement to a live lecture-based continuing medical education (CME) course. What was done Attendees of the Harvard Medical School Update in Internal Medicine course, held in Boston, MA in December 2008, were recruited via e-mail. The spaced education programme began at the close of the programme in December 2008 and ran to May 2009. The programme consisted of 12 questions on hepatitis C, which was the topic of a featured lecture during the course. The 12-item adaptive spaced education course was structured so that learners were sent two questions every 2 days. If a question was answered incorrectly, it was repeated 12 days later. If a question was answered correctly, it was repeated 24 days later. Once a question had been answered correctly twice in a row, it was retired and not repeated again. Participants completed the programme by retiring 80% of the questions and, upon doing so, were asked to complete an endof-programme evaluation using 5-point Likert scales. Evaluation of results and impact A total of 86 attendees enrolled in the trial, 15 of whom (17%) did not answer any questions. Of the remaining 71 participants, 47 (66%) completed the programme and 42 of these (89%) submitted the endof-programme evaluation. Respondents reported that: (i) the programme enhanced the impact of the live CME course (90% strongly agree ⁄ agree); (ii) they would recommend the spaced education programme to a colleague (85%); (iii) the programme reinforced their knowledge of hepatitis C (91%), and (iv) the programme increased their confidence in managing patients with hepatitis C (81%). Forty-one of 42 respondents (98%) requested to participate in further spaced education programmes offered as supplements to live CME courses. In summary, our study demonstrates that online spaced education is an effective and well-accepted supplement to a live CME course. Further research is needed to determine if spaced education can boost the ability of live CME courses to improve clinical practice patterns.

Board 390 - Research Abstract Assessment of the Challenging CanMEDS Competencies: Professional, Health Advocate and Scholar During Crisis Simulation. (Submission #1149)

Introduction/Background The Royal College of Physicians and Surgeons of Canada developed the Canadian Medical Directives for Specialist (CanMEDS) with seven core competencies: Medical Expert (ME) and six Intrinsic competencies Communicator, Collaborator, Manager, Health Advocate, Scholar and Professional. Competency of Professional, Health Advocate and Scholar (PHAS) CanMEDS competencies are difficult to define and assess during clinical practice and in simulations, in contrast to Medical Expert (ME) and other Intrinsic competencies. Our objective was to collect evidence to support construct validity of revised Generic Integrated Objective Structured Assessment Tool (GIOSAT) including content, response process, internal structure, relation to other variables and consequences using simulated scenarios targeting PHAS competencies. Research Question: Can we collect evidence to support construct validity for Professional, Health Advocate and Scholar CanMEDS competencies assessment Results for anesthesia residents performing two simulation scenarios using the Generic Integrated Objective Structured Assessment Tool and four trained blinded raters? Methods REB approval and informed consent was obtained for a prospective single blind correlation study. Twenty one anesthesia residents rotating at the University of Ottawa volunteered in this study where each of them performed both scenarios as the primary physician to manage the situation. Content: Two simulation scenarios: Do-not resuscitate (DNR) and Morphine overdose (MOD) with disclosure, were developed by a panel of experts highlighting PHAS competencies.1-3 GIOSAT is divided in two sections ME with eight items and Intrinsic with six items. Each item has abbreviated anchors and is scored with a Likert rating scale (1=very poor to 6=very good). Response process: Pilot scenarios performed by actors at optimal and sub-optimal level of performance were used to train four the raters from different institution blinded from residents identity. Raters rules were created to define borderline performances. Twenty one anesthesia residents volunteered to participate as primary physicians to manage the simulation scenarios. Internal structure was analyzed with inter-rater intra-class correlations (ICCs) and generalizability studies for ME and intrinsic and also for ME and PHAS. Relation with other variables: Comparison between scenario scores was performed with Student’s t -test. Our primary outcome was the correlations between post-graduate year of residency (PGY) and average PHAS, Intrinsic, Medical Expert and Total scores. The secondary outcome was the correlation between PHAS scores with Intrinsic, ME and total GIOSAT scores. Results ICCs for PHAS, Intrinsic, Medical Expert (ME) and total scores single measures were moderate in both scenarios (.42-.68, p<.000), and for average scores (were substantial to almost perfect (.76-.88. p<.000). Participant (p) accounted for 23% of variance and 20% for PHAS. Scenario (s) and raters (r) did not account for important variation component (VC) but the interaction between ps and psr accounted for 14 and 19 %VC for ME and Intrinsic respectively. G-study for PHAS had similar Results with ps accounting for 7% VC and psr accounting for 17% VC. (Table 1) G-coefficient for the Intrinsic was .64 and .66 for PHAS. Two raters and eight scenarios using ME and Intrinsic are required to obtain a G-coefficient >.8. Two raters and eleven scenarios using ME and PHAS are required to obtain a G-coefficient >.8.(Table 2) PGY correlated with PHAS (r=.59, p=.004), Intrinsic (r= .65, p=.002) and total scores (r=.46, p=.034) but not with ME (r= .26, p=.25). PHAS scores significantly correlated with and Intrinsic (r=.98, p<.000), ME (r= .7, p<.001) and Total (r=.89, p<.000). Conclusion Our study demonstrates construct validity evidence for assessing PHAS and Intrinsic competencies using clinical simulation with a G-coefficient of .64. Future studies with similar methodology may support construct validity at high stakes level using two raters and eight or more scenarios. References 1. Frank, J. (2005). The CanMEDS 2005 Physician Competency Framework Edited by Royal College of Physicians and Surgeons od Canada. Available from URL: http://www.rcpsc.medical.org. 2. Neira, V. M., Bould, M. D., Nakajima, A., Boet, S., Barrowman, N., Mossdorf, P., … Hamstra, S. J. (2013). “GIOSAT”: a tool to assess CanMEDS competencies during simulated crises. Canadian journal of anaesthesia. 3. Lynch, D. C., Surdyk, P. M., & Eiser, A. R. (2004). Assessing professionalism: a review of the literature. Medical teacher, 26(4), 366–73. 4. Ponton-Carss, A., Hutchison, C., & Violato, C. (2011a). Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study. American journal of surgery, 202(4), 433–40. 5. Morrison, L. J., Kierzek, G., Diekema, D. S., Sayre, M. R., Silvers, S. M., Idris, A. H., & Mancini, M. E. (2010). Part 3: ethics: 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation, 122(18 Suppl 3), S665–75. 6. Syed, S., Paul, J. E., Hueftlein, M., Kampf, M., & McLean, R. F. (2006). Morphine overdose from error propagation on an acute pain service. Canadian journal of anaesthesia. 7. The Canadian Medical Protective Association. (2008). Communicating with your patient about harm DISCLOSURE ROAD MAP. Retrieved from www.cmpa-acpm.ca. Disclosures None.