Jennifer Hepps

Placing Faculty Development Front and Center in a Multisite Educational Initiative: Lessons From the I-PASS Handoff Study

From the University of Cincinnati College of Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio (Dr O’Toole); University of California San Francisco School of Medicine, Benioff Children’s Hospital, San Francisco, Calif (DrsWest andRosenbluth); HarvardMedical School, Boston Children’s Hospital (Drs Starmer, Landrigan, and Sectish); Uniformed Health Services University of the Health Sciences, Walter Reed National Military Medical Center, Bethesda, Md (Drs Yu, Hepps, and Lopreiato); Drexel University College of Medicine, St. Christopher’s Hospital for Children, Philadephia, PA (Drs Calaman and Spector); and Brigham and Women’s Hospital, Boston, Mass (Dr Landrigan) Dr Landrigan is supported by the Children’s Hospital Association for his work as an Executive Council member of the PRIS Network. He has also received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for delivering lectures on sleep deprivation, physician performance, handoffs, and safety. The other authors declare no conflicts of interest. Address correspondence to Jennifer K. O’Toole, MD, MEd, Cincinnati Children’s Hospital Medical Center/Division of Hospital Medicine, 3333 Burnet Ave, MLC 5018, Cincinnati, OH 45229-3039 (e-mail: Jennifer.otoole@cchmc.org).

The Creation of Standard-Setting Videos to Support Faculty Observations of Learner Performance and Entrustment Decisions.

Entrustable professional activities (EPAs) provide a framework to standardize medical education outcomes and advance competency-based assessment. Direct observation of performance plays a central role in entrustment decisions; however, data obtained from these observations are often insufficient to draw valid high-stakes conclusions. One approach to enhancing the reliability and validity of these assessments is to create videos that establish performance standards to train faculty observers. Little is known about how to create videos that can serve as standards for assessment of EPAs. The authors report their experience developing videos that represent five levels of performance for an EPA for patient handoffs. The authors describe a process that begins with mapping the EPA to the critical competencies needed to make an entrustment decision. Each competency is then defined by five milestones (behavioral descriptors of performance at five advancing levels). Integration of the milestones at each level across competencies enabled the creation of clinical vignettes that were converted into video scripts and ultimately videos. Each video represented a performance standard from novice to expert. The process included multiple assessments by experts to guide iterative improvements, provide evidence of content validity, and ensure that the authors successfully translated behavioral descriptions and vignettes into videos that represented the intended performance level for a learner. The steps outlined are generalizable to other EPAs, serving as a guide for others to develop videos to train faculty. This process provides the level of content validity evidence necessary to support using videos as standards for high-stakes entrustment decisions.

Families as Partners in Hospital Error and Adverse Event Surveillance

Importance Medical errors and adverse events (AEs) are common among hospitalized children. While clinician reports are the foundation of operational hospital safety surveillance and a key component of multifaceted research surveillance, patient and family reports are not routinely gathered. We hypothesized that a novel family-reporting mechanism would improve incident detection. Objective To compare error and AE rates (1) gathered systematically with vs without family reporting, (2) reported by families vs clinicians, and (3) reported by families vs hospital incident reports. Design, Setting, and Participants We conducted a prospective cohort study including the parents/caregivers of 989 hospitalized patients 17 years and younger (total 3902 patient-days) and their clinicians from December 2014 to July 2015 in 4 US pediatric centers. Clinician abstractors identified potential errors and AEs by reviewing medical records, hospital incident reports, and clinician reports as well as weekly and discharge Family Safety Interviews (FSIs). Two physicians reviewed and independently categorized all incidents, rating severity and preventability (agreement, 68%-90%; &kgr;, 0.50-0.68). Discordant categorizations were reconciled. Rates were generated using Poisson regression estimated via generalized estimating equations to account for repeated measures on the same patient. Main Outcomes and Measures Error and AE rates. Results Overall, 746 parents/caregivers consented for the study. Of these, 717 completed FSIs. Their median (interquartile range) age was 32.5 (26-40) years; 380 (53.0%) were nonwhite, 566 (78.9%) were female, 603 (84.1%) were English speaking, and 380 (53.0%) had attended college. Of 717 parents/caregivers completing FSIs, 185 (25.8%) reported a total of 255 incidents, which were classified as 132 safety concerns (51.8%), 102 nonsafety-related quality concerns (40.0%), and 21 other concerns (8.2%). These included 22 preventable AEs (8.6%), 17 nonharmful medical errors (6.7%), and 11 nonpreventable AEs (4.3%) on the study unit. In total, 179 errors and 113 AEs were identified from all sources. Family reports included 8 otherwise unidentified AEs, including 7 preventable AEs. Error rates with family reporting (45.9 per 1000 patient-days) were 1.2-fold (95% CI, 1.1-1.2) higher than rates without family reporting (39.7 per 1000 patient-days). Adverse event rates with family reporting (28.7 per 1000 patient-days) were 1.1-fold (95% CI, 1.0-1.2; P = .006) higher than rates without (26.1 per 1000 patient-days). Families and clinicians reported similar rates of errors (10.0 vs 12.8 per 1000 patient-days; relative rate, 0.8; 95% CI, .5-1.2) and AEs (8.5 vs 6.2 per 1000 patient-days; relative rate, 1.4; 95% CI, 0.8-2.2). Family-reported error rates were 5.0-fold (95% CI, 1.9-13.0) higher and AE rates 2.9-fold (95% CI, 1.2-6.7) higher than hospital incident report rates. Conclusions and Relevance Families provide unique information about hospital safety and should be included in hospital safety surveillance in order to facilitate better design and assessment of interventions to improve safety.

Resident Experiences With Implementation of the I-PASS Handoff Bundle

BACKGROUND The I-PASS Handoff Study found that introduction of a handoff bundle (handoff and teamwork training for residents, a mnemonic, a handoff tool, a faculty development program, and a sustainability campaign) at 9 pediatrics residency programs was associated with improved communication and patient safety. OBJECTIVE This parallel qualitative study aimed to understand resident experiences with I-PASS and to inform future implementation and sustainability strategies. METHODS Resident experiences with I-PASS were explored in focus groups (N = 50 residents) at 8 hospitals throughout 2012-2013. A content analysis of transcripts was conducted following the principles of grounded theory. RESULTS Residents generally accepted I-PASS as an ideal format for handoffs, and valued learning a structured approach. Across all sites, residents reported full adherence to I-PASS when observed, but selective adherence in usual practice. Residents adhered more closely when patients were complex, teams were unfamiliar, and during evening handoff. Residents reported using elements of the I-PASS mnemonic variably, with Illness Severity and Action Items most consistently used, but Synthesis by Receiver least used, except when observed. Most residents were receptive to the electronic handoff tool, but perceptions about usability varied across sites. Experiences with observation and feedback were mixed. Concern about efficiency commonly influenced attitudes about I-PASS. CONCLUSIONS Residents generally supported I-PASS implementation, but adherence was influenced by patient type, context, and individual and team factors. Our findings could inform future implementation, particularly around the areas of resident engagement in change, sensitivity to resident level, perceived efficiency, and faculty observation.

I-PASS Mentored Implementation Handoff Curriculum: Implementation Guide and Resources

Introduction Communication failures during shift-to-shift handoffs of patient care have been identified as a leading cause of adverse events in health care institutions. The I-PASS Handoff Program is a comprehensive handoff program that has been shown to decrease rates of medical errors and adverse events. As part of the spread and adaptation of this program, a comprehensive implementation guide was created to assist individuals in the implementation process. Methods The I-PASS Mentored Implementation Guide grew out of materials created for the original I-PASS Study, Society of Hospital Medicine (SHM) mentored implementation programs, and the experience of members of the I-PASS Study Group. The guide provides a comprehensive framework of all elements required to implement the large-scale I-PASS Handoff Program and contains detailed information on generating institutional support, training activities, a campaign, measuring impact, and sustaining the program. Results Thirty-two sites across North America utilized the guide as part of the SHM program. The guide served as a main reference for 477 hours of mentoring phone calls between site leads and their mentors. Postprogram surveys from wave 2 sites revealed that 85% (N = 34) of respondents felt the quality of the guide was very good/excellent. Site leads noted that they referenced the guide most often during the early part of the program and that they referenced the sections on the curriculum and handoff observations most often. Discussion The I-PASS Mentored Implementation Guide is an essential resource for those looking to implement the large-scale I-PASS Handoff Program at their institution.

Using I-PASS in Psychiatry Residency Transitions of Care

The term “patient care handoff” is used to describe the process of transferring patient-specific clinical information between health care providers. This process represents a critical juncture in which communication failures may lead to a decrease in the quality of care, including direct harm to patients. In order to improve the handoff process, many graduate medical education (GME) programs have begun using standardized handoff protocols; unfortunately, there are many psychiatry programs without any established protocol nor there are much literature about the use of handoff tools in psychiatry [1]. In fact, on a search of the CINAHL, EMBASE, PsycINFO, and PubMed databases, only one article discussed a specific handoff tool used in a psychiatry program. Increased attention has also been placed on the handoff process, due to the implementation of reduced resident work hours as mandated by the Accreditation Council for Graduate Medical Education (ACGME) [2]. One consequence of the reduction in work hours is an increase in the number of patient care handoffs needed to ensure continuity of care, leading to an increased opportunity for miscommunication [2]. Serious communication breakdowns often involve ambiguity about responsibilities or missing details during the transition. In order to resolve this problem, a standardized handoff protocol should be used to ensure that incoming providers are equipped with the proper information [3]. Handoffs are often overlooked in resident education, but they merit as much focus and attention as all other patient safety aspects of the GME curriculum Table 1.

Comparing Peer and Staff Evaluations of Pediatric Residents Using the Acgme Milestones

BACKGROUND: The IOM’s report To Err Is Human identified that in the United States medical errors are a significant cause of deaths annually. The ACGME’s CLER program states that residents should participate in patient safety education and experiential learning. From 2011-14, our institution, required interns to present a self identified patient safety event at a monthly conference entitled the Systems Improvement Conference(SIC). This retrospective study aimed to classify the types and severity of events interns most often self identified as part of a required systems based practice experience. METHODS: In May 2014, we performed a retrospective analysis of resident cases presented from 2011-2014 by interns during the SIC. We reviewed each presentation for the following: 1.) Type of safety event presented, 2.)Classification and severity of safety events brought forward by residents, 3.)Proximate causes developed, 4.) Action items developed and implemented. This was an IRB approved study. RESULTS: Our retrospective data revealed that 27 different patient safety events were brought forward by pediatric residents. The majority of these events were delay in patient care (12/27), followed by adverse drug events (7/27), failed communication, and readmission. 10 of the events were classified as potential safety events, 8 were classified as serious safety events, 4 were classified as near miss events, and 5 were unclassified. Of the 12 documented action items, 3 have been implemented including central line protocol and diabetes care pathway and order set. Other action items were difficult to implement or unsustainable. Resident feedback included frustration with implementation, lack of interprofessional buy-in, and repeated errors presented. CONCLUSIONS: Pediatric residents are able to identify and investigate patient safety events. Our residents recognized the value of investigating near miss and serious safety events. However, completion of action items and implementation that affect patient outcomes was difficult due to lack of interprofessional teams.