Jennifer O'Toole

Development, implementation, and dissemination of the I-PASS handoff curriculum: A multisite educational intervention to improve patient handoffs

Patient handoffs are a key source of communication failures and adverse events in hospitals. Despite Accreditation Council for Graduate Medical Education requirements for residency training programs to provide formal handoff skills training and to monitor handoffs, well-established curricula and validated skills assessment tools are lacking. Developing a handoff curriculum is challenging because of the need for standardized processes and faculty development, cultural resistance to change, and diverse institution- and unit-level factors. In this article, the authors apply a logic model to describe the process they used from June 2010 to February 2014 to develop, implement, and disseminate an innovative, comprehensive handoff curriculum in pediatric residency training programs as a fundamental component of the multicenter Initiative for Innovation in Pediatric Education–Pediatric Research in Inpatient Settings Accelerating Safe Sign-outs (I-PASS) Study. They describe resources, activities, and outputs, and report preliminary learner outcomes using data from resident and faculty evaluations of the I-PASS Handoff Curriculum: 96% of residents and 97% of faculty agreed or strongly agreed that the curriculum promoted acquisition of relevant skills for patient care activities. They also share lessons learned that could be of value to others seeking to adopt a structured handoff curriculum or to develop large-scale curricular innovations that involve redesigning firmly established processes. These lessons include the importance of approaching curricular implementation as a transformational change effort, assembling a diverse team of junior and senior faculty to provide opportunities for mentoring and professional development, and linking the educational intervention with the direct measurement of patient outcomes.

Training Tomorrow's Medical Education Leaders: Creating a General Pediatric Master Educator Fellowship

From the University of Cincinnati College of Medicine/Cincinnati Children’s Hospital Medical Center, Cincinnati, OH uring the past decade, the need for proficient and inspiring clinician educators in pediatric graduate medical education has become increasingly critical with the evolving work hour changes and increasing content requirements in medical student, resident, and fellow training. General pediatrics divisions are mainly responsible for resident continuity clinics and provide a large percentage of the leadership and teaching for both residency programs andmedical student clerkships, making training in education for these individuals essential. Traditionally, training in academic pediatric fellowships intensively focuses on research methodology and outcomes, and training in education was secondary. We believe medical educator training requires an equally rigorous approach to learn to create innovative curricula while effectively evaluating their impact on physicians’ skills, behaviors, and, ultimately, associated patient health care outcomes. Additionally, there is an increasing recognition that leadership training for academic pediatricians is required to prepare physicians to respond to new roles and responsibilities, such as leading faculty development programs, quality improvement projects, and strategic planning, and promoting evidence-based medicine and health care policy changes. To address these needs, an innovative General Pediatrics Master Educator Fellowship (GPMEF) was launched at Cincinnati Children’s Hospital Medical Center in fall 2011 to formally train general pediatricians in the science of education via a unique curriculum focusing on educational scholarship and research, structured learning experiences in clinical teaching, and fostering of leadership skills. This GPMEF is uniquely designed to create a cadre of medical educators trained in educational innovation, evaluation, research methodology, and academic leadership.

Resident Experiences With Implementation of the I-PASS Handoff Bundle

BACKGROUND The I-PASS Handoff Study found that introduction of a handoff bundle (handoff and teamwork training for residents, a mnemonic, a handoff tool, a faculty development program, and a sustainability campaign) at 9 pediatrics residency programs was associated with improved communication and patient safety. OBJECTIVE This parallel qualitative study aimed to understand resident experiences with I-PASS and to inform future implementation and sustainability strategies. METHODS Resident experiences with I-PASS were explored in focus groups (N = 50 residents) at 8 hospitals throughout 2012-2013. A content analysis of transcripts was conducted following the principles of grounded theory. RESULTS Residents generally accepted I-PASS as an ideal format for handoffs, and valued learning a structured approach. Across all sites, residents reported full adherence to I-PASS when observed, but selective adherence in usual practice. Residents adhered more closely when patients were complex, teams were unfamiliar, and during evening handoff. Residents reported using elements of the I-PASS mnemonic variably, with Illness Severity and Action Items most consistently used, but Synthesis by Receiver least used, except when observed. Most residents were receptive to the electronic handoff tool, but perceptions about usability varied across sites. Experiences with observation and feedback were mixed. Concern about efficiency commonly influenced attitudes about I-PASS. CONCLUSIONS Residents generally supported I-PASS implementation, but adherence was influenced by patient type, context, and individual and team factors. Our findings could inform future implementation, particularly around the areas of resident engagement in change, sensitivity to resident level, perceived efficiency, and faculty observation.

Multidisciplinary handoffs improve perceptions of communication.

BACKGROUND Communication errors during handoffs are a leading cause of sentinel events. The Accreditation Council for Graduate Medical Education 2011 duty hour standards (DHS) increase the frequency of handoffs. OBJECTIVE The goal of this study was to determine if a multidisciplinary group handoff bundle improves communication while working within the 2011 DHS. METHODS During 1-month pilot programs of the 2011 DHS, 2 groups were observed. Group A adopted a multidisciplinary group handoff bundle, including presence of residents and charge nurses, a standardized mnemonic in verbal and written form, and resident training. Group B received only a mnemonic pocket card. Residents completed preintervention and postintervention Likert scale surveys to assess handoff perceptions. Within-group preintervention to postintervention changes were analyzed by using the signed rank test. Measuring communication errors, an institutional tool was used to track unanticipated patient occurrences (UPOs) postintervention for both groups. RESULTS Significant improvements for the preintervention to postintervention surveys regarding the perceptions of quality of handoffs received, effective and efficient delivery of handoffs, comfort in giving handoffs, and handoff practices focusing on safety (all, P ≤ .05) were observed in group A. There were no significant changes in group B. Overall, 17% of collected group A UPO forms and 11% of group B UPO forms had at least 1 UPO recorded. The most common reason for a UPO was unaddressed nursing concerns. CONCLUSIONS A multidisciplinary group of residents and charge nurses and a handoff bundle was associated with improved resident perceptions of handoffs and communication within the 2011 DHS.

The practice patterns of recently graduated internal medicine-pediatric hospitalists.

OBJECTIVES To identify the current practice patterns and professional activities of internal medicine-pediatrics (med-peds) hospitalists who have graduated in the past 5 years (June 2009-June 2013). METHODS The national Medicine-Pediatrics Program Directors Association (MPPDA) conducted a cross-sectional survey study of the 79 residency program directors who are members of the MPPDA regarding the practice patterns of recent graduates (from 2009-2013) currently practicing as hospitalists. The survey was distributed in the spring of 2014 on the MPPDA listserv. The survey inquired about time spent caring for hospitalized adults and children, medical school appointments, practice in freestanding childrens hospitals, and completion of hospital medicine (HM) fellowships. RESULTS Forty-nine program directors (62%) completed the survey and provided data on 1042 graduates from 46 programs. Of those graduates, 26.4% (n=275) practice as hospitalists, and none had completed an HM fellowship. Approximately two-thirds (65%) of med-peds hospitalists provide care to hospitalized children and adults, with one-third providing care solely to hospitalized adults. Approximately one-half (53.5%) have an appointment with a medical school and roughly one-quarter (28%) practice in a freestanding childrens hospital. CONCLUSIONS An increasing percentage of recent med-peds graduates are pursuing careers in HM, and two-thirds are providing care to hospitalized children. As consideration for an accredited pediatric HM fellowship continues, certifying and accrediting bodies should consider how this will impact the med-peds workforce and allow med-peds graduates flexibility in their training requirements that will permit them to acquire the necessary skills to care for hospitalized children and adults.

Paracentesis Simulation: A Comprehensive Approach to Procedural Education

Introduction Structured procedural education and assessment of competency are growing needs for residency and fellowship programs. Simulation is a useful way to learn, experience, and practice procedural skills with competence. Paracentesis is a common procedure encountered in internal medicine. This educational resource for paracentesis education includes didactics, cases, and assessments to address cognitive skills, a simulation experience to address psychomotor procedural skills, and an entrustment-based assessment tool. Methods Prior to the simulation, learners completed preprocedural didactics and self-assessments. Utilizing a paracentesis trainer, ultrasound, and paracentesis kit, the case of a 46-year-old male with ascites in need of a paracentesis was presented. During the simulation, learners initially performed a paracentesis step by step, with assistance and feedback from the case instructor. This was immediately followed by paracentesis without assistance, where the instructor evaluated the learners with an assessment tool encompassing a procedural checklist, global skill assessment scale, and entrustment scale. Afterwards, learners completed case-based reviews and returned to the simulation lab several months later to repeat an unassisted paracentesis. Results The curriculum was used with internal medicine and medicine-pediatric residents of all training levels. To date, over 120 residents have completed the curriculum. Residents reported an increase in self-confidence and competence using ultrasound to identify ascites and performing a paracentesis. Learners provided positive feedback. Discussion This curriculum offers the opportunity for both cognitive and psychomotor paracentesis education in a low-risk simulation environment. The comprehensive strategy with didactics, cases, and multiple simulations is designed to promote knowledge and skill retention.

I-PASS Mentored Implementation Handoff Curriculum: Implementation Guide and Resources

Introduction Communication failures during shift-to-shift handoffs of patient care have been identified as a leading cause of adverse events in health care institutions. The I-PASS Handoff Program is a comprehensive handoff program that has been shown to decrease rates of medical errors and adverse events. As part of the spread and adaptation of this program, a comprehensive implementation guide was created to assist individuals in the implementation process. Methods The I-PASS Mentored Implementation Guide grew out of materials created for the original I-PASS Study, Society of Hospital Medicine (SHM) mentored implementation programs, and the experience of members of the I-PASS Study Group. The guide provides a comprehensive framework of all elements required to implement the large-scale I-PASS Handoff Program and contains detailed information on generating institutional support, training activities, a campaign, measuring impact, and sustaining the program. Results Thirty-two sites across North America utilized the guide as part of the SHM program. The guide served as a main reference for 477 hours of mentoring phone calls between site leads and their mentors. Postprogram surveys from wave 2 sites revealed that 85% (N = 34) of respondents felt the quality of the guide was very good/excellent. Site leads noted that they referenced the guide most often during the early part of the program and that they referenced the sections on the curriculum and handoff observations most often. Discussion The I-PASS Mentored Implementation Guide is an essential resource for those looking to implement the large-scale I-PASS Handoff Program at their institution.

Critical Deficiency Ratings in Milestone Assessment: A Review and Case Study.

Purpose The Accreditation Council for Graduate Medical Education (ACGME) requires programs to report learner progress using specialty-specific milestones. It is unclear how milestones can best identify critical deficiencies (CDs) in trainee performance. Specialties developed milestones independently of one another; not every specialty included CDs within milestones ratings. This study examined the proportion of ACGME milestone sets that include CD ratings, and describes one residency program’s experiences using CD ratings in assessment. Method The authors reviewed ACGME milestones for all 99 specialties in November 2015, determining which rating scales contained CDs. The authors also reviewed three years of data (July 2012–June 2015) from the University of Cincinnati Medical Center (UCMC) internal medicine residency assessment system based on observable practice activities mapped to ACGME milestones. Data were analyzed by postgraduate year, assessor type, rotation, academic year, and core competency. The Mantel–Haenszel chi-square test was used to test for changes over time. Results Specialties demonstrated heterogeneity in accounting for CDs in ACGME milestones, with 22% (22/99) of specialties having no language describing CDs in milestones assessment. Thirty-three percent (63/189) of UCMC internal medicine residents received at least one CD rating, with CDs accounting for 0.18% (668/364,728) of all assessment ratings. The authors identified CDs across multiple core competencies and rotations. Conclusions Despite some specialties not accounting for CDs in milestone assessment, UCMC’s experience demonstrates that a significant proportion of residents may be rated as having a CD during training. Identification of CDs may allow programs to develop remediation and improvement plans.

It's Complicated: Overcoming Statistical Challenges of Group Learning in Medical Education Research

Medical education interventions often have methodological challenges due to repeated measures and learning that often occur in groups. These challenges can be addressed appropriately by applying the relevant statistical techniques. Without the proper evaluation methods, an educator might conclude that a new curriculum or intervention is effective when it is not, or vice versa. Early consultation with a statistician is ideal, as many of these issues affect the sample size needed to detect meaningful differences and should be incorporated into the statistical analysis plan. This primer, along with examples, may be a useful place to start the discussion with the statistician.​statistician.

A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool

BACKGROUND The importance of effective clinical teaching skills is well established in the literature. However, reliable tools with validity evidence that are able to measure the development of these skills and can effectively be used by nonphysician raters do not exist. OBJECTIVE Our initiative had 2 aims: (1) to develop a teaching development assessment tool (TDAT) that allows skill assessment along a continuum, and (2) to determine if trained nonphysicians can assess clinical teachers with this tool. METHODS We describe the development of the TDAT, including identification of 6 global teaching domains and observable teaching behaviors along a 3-level continuum (novice/beginner, competent/proficient, expert) and an iterative revision process involving local and national content experts. The TDAT was studied with attending physicians during inpatient rounds with trained physician and nonphysician observers over 6 months. RESULTS The TDAT showed emerging evidence of content, construct, and viable validity (the degree to which an assessment tool is practical, affordable, suitable, evaluable, and helpful in the real world) for the evaluation of attending physicians on inpatient rounds. Moderate to near perfect interrater reliability was seen between physician and nonphysician raters for the domains of promotion of clinical reasoning, control of the learning environment, ability to teach to multiple levels of learners, and provision of feedback. CONCLUSIONS The TDAT holds potential as a valid and reliable assessment tool for clinical teachers to track the development of each individuals teaching skills along the continuum from early development to mastery.