Kelly J. Caverzagie

The Internal Medicine Reporting Milestones and the Next Accreditation System

The Accreditation Council for Graduate Medical Education (ACGME) developed the Milestones Project to facilitate more synthetic and narrative-based assessments of educational outcomes. This commenta...

The development of entrustable professional activities for internal medicine residency training: A report from the Education Redesign Committee of the Alliance for Academic Internal Medicine

Purpose The Alliance for Academic Internal Medicine charged its Education Redesign Committee with the task of assisting internal medicine residency program directors in meeting the challenges of competency-based assessment that were part of the Accreditation Council for Graduate Medical Education’s (ACGME’s) Next Accreditation System. Method Recognizing the limitations of the ACGME general competencies as an organizing framework for assessment and the inability of the milestones to provide the needed context for faculty to assess residents’ competence, the Education Redesign Committee in 2011 adopted the work-based assessment framework of entrustable professional activities (EPAs). The committee selected the EPA framework after reviewing the literature on competency-based education and EPAs and consulting with experts in evaluation and assessment. The committee used an iterative approach with broad-based feedback from multiple sources, including program directors, training institutions, medical organizations, and specialty societies, to develop a set of EPAs that together define the core of the internal medicine profession. Results The resulting 16 EPAs are those activities expected of a resident who is ready to enter unsupervised practice, and they provide a starting point from which training programs could develop assessments and curricula. The committee also provided a strategy for the use of these EPAs in competency-based evaluation. Conclusions These EPAs are intended to serve as a starting point or guide for program directors to begin developing meaningful, work-based assessments that inform the evaluation of residents’ competence.

Early feedback on the use of the internal medicine reporting milestones in assessment of resident performance.

BACKGROUND The educational milestones were designed as a criterion-based framework for assessing resident progression on the 6 Accreditation Council for Graduate Medical Education competencies. OBJECTIVE We obtained feedback on, and assessed the construct validity and perceived feasibility and utility of, draft Internal Medicine Milestones for Patient Care and Systems-Based Practice. METHODS All participants in our mixed-methods study were members of competency committees in internal medicine residency programs. An initial survey assessed participant and program demographics; focus groups obtained feedback on the draft milestones and explored their perceived utility in resident assessment, and an exit survey elicited input on the value of the draft milestones in resident assessment. Surveys were tabulated using descriptive statistics. Conventional content analysis method was used to assess the focus group data. RESULTS Thirty-four participants from 17 programs completed surveys and participated in 1 of 6 focus groups. Overall, the milestones were perceived as useful in formative and summative assessment of residents. Participants raised concerns about the length and complexity of some draft milestones and suggested specific changes. The focus groups also identified a need for faculty development. In the exit survey, most participants agreed that the Patient Care and Systems-Based Practice Milestones would help competency committees assess trainee progress toward independent practice. CONCLUSIONS Draft reporting milestones for 2 competencies demonstrated significant construct validity in both the content and response process and the perceived utility for the assessment of resident performance. To ensure success, additional feedback from the internal medicine community and faculty development will be necessary.

The role of physician engagement on the impact of the hospital-based practice improvement module (PIM)†

BACKGROUND Physicians play an important role in hospital quality improvement (QI) activities. The Hospital-Based Practice Improvement Module (Hospital PIM) is a web-based assessment tool designed by the American Board of Internal Medicine (ABIM) to facilitate physician involvement in QI as a part of maintaining certification. OBJECTIVE The primary objective of this study is to explore the impact of the Hospital PIM on physicians participating in hospital-based QI. DESIGN Qualitative design consisting of semistructured telephone interviews. PARTICIPANTS A purposeful sample of 21 early-completers of the Hospital PIM. MEASUREMENTS Grounded-theory analysis was used to analyze transcripts of the semistructured telephone interviews. RESULTS Physician completers of the Hospital PIM describe the impact in a variety of ways, including new learning about QI principles and activities, added value to their practice, and enhanced QI experience. An emerging theme was the mediating role of physician engagement in relation to the overall impact of the Hospital PIM. Four case studies illustrate these findings. Facilitators and barriers that influence the overall experience of the PIM are described. CONCLUSIONS The impact of completing the Hospital PIM is mediated by the degree of physician engagement with the QI process. Physicians who become engaged with the Hospital PIM and QI process may be more likely to report successful experiences in implementing QI activities in hospital settings than those who do not become engaged.

Internal medicine milestones.

William Iobst, MD, is Vice President of Academic Affairs, American Board of Internal Medicine; Eve Aagaard, MD, is Associate Professor of Medicine, University of Colorado School of Medicine; Hasan Bazari, MD, is Program Director, Internal Medicine Residency Program, Massachusetts General Hospital, and Associate Professor of Medicine, Harvard Medical School; Timothy Brigham, MDiv, PhD, is Chief of Staff and Senior Vice President, Department of Education, Accreditation Council for Graduate Medical Education; Roger W. Bush, MD, is Attending Physician, Virginia Mason Medical Center; Kelly Caverzagie, MD, is Assistant Professor of Medicine and Associate Vice Chair for Quality and Physician Competence, Department of Internal Medicine, University of Nebraska Medical Center; Davoren Chick, MD, is Clinical Assistant Professor of Medicine, Department of Internal Medicine, University of Michigan Medical School; Michael Green, MD, is Professor of Medicine, Yale University School of Medicine; Kevin Hinchey, MD, is Associate Professor, Tufts University School of Medicine, and Chief Academic Officer, Baystate Medical Center; Eric Holmboe, MD, is Chief Medical Officer, American Board of Internal Medicine; Sarah Hood, MS, is Director of Academic Affairs, American Board of Internal Medicine; Gregory Kane, MD, is Professor of Medicine, Interim Chairman of the Department of Medicine, Jefferson Medical College; Lynne Kirk, MD, is Professor of Internal Medicine, University of Texas Southwestern Medical Center; Lauren Meade, MD, is Assistant Professor of Medicine, Tufts University School of Medicine, and Associate Program Director for Internal Medicine, Baystate Medical Center, and Chair of Educational Research Outcomes Collaborative–Internal Medicine; Cynthia Smith, MD, is Senior Medical Associate for Content Development, American College of Physicians, and Adjunct Associate Professor, Perelman School of Medicine; and Susan Swing, PhD, is Vice President, Outcome Assessment, Accreditation Council for Graduate Medical Education.

Resident identification of learning objectives after performing self-assessment based upon the ACGME core competencies.

BackgroundSelf-assessment is increasingly being incorporated into competency evaluation in residency training. Little research has investigated the characteristics of residents’ learning objectives and action plans after self-assessment.ObjectiveTo explore the frequency and specificity of residents’ learning objectives and action plans after completing either a highly or minimally structured self-assessment.DesignInternal Medicine residents (N = 90) were randomized to complete a highly or minimally structured self-assessment instrument based on the Accreditation Council for Graduate Medical Education Core Competencies. All residents then identified learning objectives and action plans.MeasurementsLearning objectives and action plans were analyzed for content. Differences in specificity and content related to form, gender, and training level were assessed.ResultsSeventy-six residents (84% response rate) identified 178 learning objectives. Objectives were general (79%), most often focused on medical knowledge (40%), and were not related to the type of form completed (p > 0.01). “Reading more” was the most common action plan.ConclusionsResidents commonly identify general learning objectives focusing on medical knowledge regardless of the structure of the self-assessment form. Tools and processes that further facilitate self-assessment should be identified.

Playing with curricular milestones in the educational sandbox: Q-sort results from an internal medicine educational collaborative

Purpose In competency-based medical education, the focus of assessment is on learner demonstration of predefined outcomes or competencies. One strategy being used in internal medicine (IM) is applying curricular milestones to assessment and reporting milestones to competence determination. The authors report a practical method for identifying sets of curricular milestones for assessment of a landmark, or a point where a resident can be entrusted with increased responsibility. Method Thirteen IM residency programs joined in an educational collaborative to apply curricular milestones to training. The authors developed a game using Q-sort methodology to identify high-priority milestones for the landmark “Ready for indirect supervision in essential ambulatory care” (EsAMB). During May to December 2010, the programs’ambulatory faculty participated in the Q-sort game to prioritize 22 milestones for EsAMB. The authors analyzed the data to identify the top 8 milestones. Results In total, 149 faculty units (1–4 faculty each) participated. There was strong agreement on the top eight milestones; six had more than 92% agreement across programs, and five had 75% agreement across all faculty units. During the Q-sort game, faculty engaged in dynamic discussion about milestones and expressed interest in applying the game to other milestones and educational settings. Conclusions The Q-sort game enabled diverse programs to prioritize curricular milestones with interprogram and interparticipant consistency. A Q-sort exercise is an engaging and playful way to address milestones in medical education and may provide a practical first step toward using milestones in the real-world educational setting.

Implementing competency-based medical education: What changes in curricular structure and processes are needed?

Abstract Medical educators must prepare for a number of challenges when they decide to implement a competency-based curriculum. Many of these challenges will pertain to three key aspects of implementation: organizing the structural changes that will be necessary to deliver new curricula and methods of assessment; modifying the processes of teaching and evaluation; and helping to change the culture of education so that the CBME paradigm gains acceptance. This paper focuses on nine key considerations that will support positive change in first two of these areas. Key considerations include: ensuring that educational continuity exists amongst all levels of medical education, altering how time is used in medical education, involving CBME in human health resources planning, ensuring that competent doctors work in competent health care systems, ensuring that information technology supports CBME, ensuring that faculty development is supported, ensuring that the rights and responsibilities of the learner are appropriately balanced in the workplace, preparing for the costs of change, and having appropriate leadership in order to achieve success in implementation.

Overarching challenges to the implementation of competency-based medical education

Abstract Medical education is under increasing pressure to more effectively prepare physicians to meet the needs of patients and populations. With its emphasis on individual, programmatic, and institutional outcomes, competency-based medical education (CBME) has the potential to realign medical education with this societal expectation. Implementing CBME, however, comes with significant challenges. This manuscript describes four overarching challenges that must be confronted by medical educators worldwide in the implementation of CBME: (1) the need to align all regulatory stakeholders in order to facilitate the optimization of training programs and learning environments so that they support competency-based progression; (2) the purposeful integration of efforts to redesign both medical education and the delivery of clinical care; (3) the need to establish expected outcomes for individuals, programs, training institutions, and health care systems so that performance can be measured; and (4) the need to establish a culture of mutual accountability for the achievement of these defined outcomes. In overcoming these challenges, medical educators, leaders, and policy-makers will need to seek collaborative approaches to common problems and to learn from innovators who have already successfully made the transition to CBME.

Changing the culture of medical training: An important step toward the implementation of competency-based medical education

Abstract Objective: The current medical education system is steeped in tradition and has been shaped by many long-held beliefs and convictions about the essential components of training. The objective of this article is to propose initiatives to overcome biases against competency-based medical education (CBME) in the culture of medical education. Materials and methods: At a retreat of the International Competency Based Medical Education (ICBME) Collaborators group, an intensive brainstorming session was held to determine potential barriers to adoption of CBME in the culture of medical education. This was supplemented with a review of the literature on the topic. Results: There continues to exist significant key barriers to the widespread adoption of CBME. Change in educational culture must be embraced by all components of the medical education hierarchy. Research is essential to provide convincing evidence of the benefit of CBME. Conclusions: The widespread adoption of CBME will require a change in the professional, institutional, and organizational culture surrounding the training of medical professionals.