Maria Ahmed

Objective structured assessment of debriefing: bringing science to the art of debriefing in surgery

Objective:To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings. Introduction:Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing—yet there is a paucity of research to guide best practice. Methods:Phase 1—Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2—Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations. Results:Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test–retest ICC = 0.90], and face and content valid (content validity index = 0.94). Conclusions:OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.

The use of simulation in neurosurgical education and training: A systematic review

OBJECT There is increasing evidence that simulation provides high-quality, time-effective training in an era of resident duty-hour restrictions. Simulation may also permit trainees to acquire key skills in a safe environment, important in a specialty such as neurosurgery, where technical error can result in devastating consequences. The authors systematically reviewed the application of simulation within neurosurgical training and explored the state of the art in simulation within this specialty. To their knowledge this is the first systematic review published on this topic to date. METHODS The authors searched the Ovid MEDLINE, Embase, and PsycINFO databases and identified 4101 articles; 195 abstracts were screened by 2 authors for inclusion. The authors reviewed data on study population, study design and setting, outcome measures, key findings, and limitations. RESULTS Twenty-eight articles formed the basis of this systematic review. Several different simulators are at the neurosurgeons disposal, including those for ventriculostomy, neuroendoscopic procedures, and spinal surgery, with evidence for improved performance in a range of procedures. Feedback from participants has generally been favorable. However, study quality was found to be poor overall, with many studies hampered by nonrandomized design, presenting normal rather than abnormal anatomy, lack of control groups and long-term follow-up, poor study reporting, lack of evidence of improved simulator performance translating into clinical benefit, and poor reliability and validity evidence. The mean Medical Education Research Study Quality Instrument score of included studies was 9.21 ± 1.95 (± SD) out of a possible score of 18. CONCLUSIONS The authors demonstrate qualitative and quantitative benefits of a range of neurosurgical simulators but find significant shortfalls in methodology and design. Future studies should seek to improve study design and reporting, and provide long-term follow-up data on simulated and ideally patient outcomes.

Identifying best practice guidelines for debriefing in surgery: a tri-continental study.

BACKGROUND Changes in surgical training have decreased opportunities for experiential learning in the operating room (OR). With this decrease, a commensurate increase in debriefing-dependent simulation-based activities has occurred. Effective debriefing could optimize learning from both simulated and real clinical encounters. METHODS Thirty-three semistructured interviews with surgeons, anesthesiologists, and OR nurses from the United Kingdom, United States, and Australia identified the goals of debriefing, core components of an effective debrief, and solutions to its effective implementation. Interviews were audiotaped, transcribed, and coded using emergent theme analysis. RESULTS Core components of an effective debrief include having the appropriate approach, establishing a learning environment, learner engagement, managing learner reaction, reflection, analysis, diagnosis, and application to real clinical practice. Solutions to enhance practice involve promotion of a debriefing culture within the surgical community with protected time to conduct a structured debriefing. CONCLUSIONS A need exists to enhance surgical training through regular structured debriefing. Identifying the key components of an effective debrief is a first step toward improving practice and embedding a debriefing culture within the OR.

Operation debrief: a SHARP improvement in performance feedback in the operating room.

Objectives: To explore the current status of performance feedback (debriefing) in the operating room and to develop and evaluate an evidence-based, user-informed intervention termed “SHARP” to improve debriefing in surgery. Background: Effective debriefing is a key educational technique for optimizing learning in surgical settings. However, there is a lack of a debriefing culture within surgery. Few studies have prospectively evaluated educational interventions to improve the quality and quantity of performance feedback in surgery. Methods: This was a prospective pre- and post-study of 100 cases involving 22 trainers (attendings) and 30 surgical residents (postgraduate years 3–8). A trained researcher assessed the quality of debriefings provided to the trainee using the validated Objective Structured Assessment of Debriefing (OSAD) tool alongside ethnographic observation. Following the first 50 cases, an educational intervention termed “SHARP” was introduced and measures repeated for a further 50 cases. User satisfaction with SHARP was assessed via questionnaire. Twenty percent of the cases were observed independently by a second researcher to test interrater reliability. Results: Interrater reliability for OSAD was excellent (ICC = 0.994). Objective scores of debriefing (OSAD) improved significantly after the SHARP intervention: median pre = 19 (range, 8–31); median post = 33 (range, 26–40), P < 0.001. Strong correlations between observer (OSAD) and trainee rating of debriefing were obtained (median &rgr; = 0.566, P < 0.01). Ethnographic observations also supported a significant improvement in both quality and style of debriefings. Users reported high levels of satisfaction in terms of usefulness, feasibility, and comprehensiveness of the SHARP tool. Conclusions: SHARP is an effective and efficient means of improving performance feedback in the operating room. Its routine use should be promoted to optimize workplace-based learning and foster a positive culture of debriefing and performance improvement within surgery.

Actual vs perceived performance debriefing in surgery: practice far from perfect

BACKGROUND Performance feedback or debriefing in surgery is increasingly recognized as an essential means to optimize learning in the operating room (OR). However, there is a lack of evidence regarding the current practice and barriers to debriefing in the OR. METHODS Phase 1 consisted of semistructured interviews with surgical trainers and trainees to identify features of an effective debriefing and perceived barriers to debriefing. Phase 2 consisted of ethnographic observations of surgical cases to identify current practice and observed barriers to debriefing. RESULTS Surgical trainers and trainees identified key features of effective debriefing with regard to the approach and content; however, these were not commonly identified in practice. Culture was recognized as a significant barrier to debriefing across both phases of the study. CONCLUSIONS There is a disparity between what the surgical community views as effective debriefing and actual debriefing practices in the OR. Improvements to the current debriefing culture and practice within the field of surgery should be considered to facilitate learning from clinical practice.

Building a safer foundation: the Lessons Learnt patient safety training programme

Objectives To develop, implement and evaluate a novel patient safety training programme for junior doctors across a Foundation School—‘Lessons Learnt: Building a Safer Foundation’. Design, setting and participants Prospective preintervention /postintervention study across 16 Foundation Programmes in North West England, UK. 1169 participants including all Foundation Programme Directors, Administrators, Foundation trainees and senior faculty. Interventions Half-day stakeholder engagement event and faculty development through recruitment and training of local senior doctors. Foundation trainee-led monthly 60-min sessions integrated into compulsory Foundation teaching from January to July 2011 comprising case-based discussion and analysis of patient safety incidents encountered in practice, facilitated by trained faculty. Main outcome measures Participants’ satisfaction and Foundation trainees’ patient safety knowledge, skills, attitudes and behavioural change. Results Participants reported high levels of satisfaction with ‘Lessons Learnt’. There was a significant improvement in trainees’ objective patient safety knowledge scores (Meanpreintervention=51.1%, SD=17.3%; Meanpostintervention=57.6%, SD=20.1%, p<0.001); subjective knowledge ratings and patient safety skills. Trainees’ perceived control and behavioural intentions regarding safety improved significantly postintervention. Feelings and personal beliefs about safety did not shift significantly. Trainees reported significantly more patient safety incidents in the 6 months following introduction of ‘Lessons Learnt’ (Meanpreintervention=0.67, SD=1.11; Meanpostintervention=1.18, SD=1.46, p<0.001). 32 quality improvement projects were initiated by trainees, spanning the development of novel clinical protocols; implementation of user-informed teaching and improved rota design Conclusions Patient safety training can be implemented and sustained to deliver significant improvements in patient safety knowledge, skills and behaviours of junior doctors—with potential for wider positive organisational impact. Medical education commissioners and providers could adopt and build upon the ‘Lessons Learnt’ approach as a springboard to promote medical engagement in quality and safety improvement.

Building capacity and capability for patient safety education: a train-the-trainers programme for senior doctors

Objectives To develop, implement and evaluate a training programme for senior doctors to become faculty leaders for patient safety training. Methods Senior doctors were recruited from across 20 hospitals in the North Western Deanery, England, UK. The intervention comprised a half-day course in patient safety theory, root cause analysis and small-group facilitation, following which participants were invited to sign up as faculty for a region-wide patient safety training programme for trainees ‘Lessons Learnt’. Course evaluation comprised a prospective longitudinal study conducted in 2010–2012. Patient safety knowledge, attitudes and skills were evaluated pre and post course and retention further evaluated 8 months post course. Results 216 senior doctors volunteered as faculty of whom 122 were appointed. Participants reported high levels of satisfaction with the course. Objective scores of patient safety knowledge significantly improved immediately post course (MedianPre course=70%, MedianPost course=80%, p<0.001) and were sustained at 8 months (Median8 month post course=90%). Similarly, measures of attitudes and self-reported safety skills also significantly improved post course and were sustained. Upon completion of the course, 88/122 (72%) participants facilitated 213 ‘Lessons Learnt’ sessions from January 2011 to July 2012 (mean 2, range 1–8 sessions per faculty member). Trainee satisfaction with faculty was high. Conclusions There is considerable appetite for senior doctors to engage with training in patient safety as teachers and learners. Training senior doctors in patient safety is feasible, acceptable and effective as a means of building capacity and capability for delivering training in this rapidly emerging field.

Junior doctors' reflections on patient safety

Aim To determine whether foundation year 1 (FY1) doctors reflect upon patient safety incidents (PSIs) within their portfolios and the potential value of such reflections for quality of care. Methods A cross-sectional retrospective review of every ‘reflective practice’ portfolio entry made by all FY1 doctors within an Acute Teaching Hospital Trust was conducted in February 2010. Entries were reviewed by two independent blinded researchers to determine whether they related to a PSI that is, any unintended or unexpected incident that could have or did lead to patient harm. For all entries rated positive by both reviewers, a content analysis approach was used to code PSI into incident type, contributing factors and patient outcome according to validated frameworks developed by the National Patient Safety Agency. Results 139 reflective practice entries were completed by 30 trainees (15 men, 15 women, mean age 24 years). Of the 139 entries, 49% reflected on a PSI. Of these, 22% were due to errors in clinical assessment; 22% were due to delayed access to care; 18% were due to infrastructure/staffing deficiencies; and 16% were due to medication errors. The most common contributing factors were team/social factors (23%), patient factors (22%), communication and task factors (both 17%). The majority of PSIs led to no harm. Six entries described PSIs resulting in patient death, the majority of which were attributable to diagnostic errors. Conclusions FY1 doctors commonly reflect on PSIs within their professional portfolios. Such critical reflection can encourage learning but may also promote patient safety and the quality of healthcare across all medical specialties.

Case-based learning for patient safety: the Lessons Learnt program for UK junior doctors.

Nordquist et al. [1] report their experience with implementing surgical case-based learning (CBL) at the Undergraduate Medical Programme at Karolinska Institutet. A series of CBL seminars were introduced in place of traditional lectures at one of four teaching hospital sites. The intention was to stimulate higher levels of cognitive learning among the students, to foster reflection and peer-learning, and to help integrate theoretical knowledge with the clinical context. Interviews held with a sample of teachers and students at the pilot site highlighted various deficiencies in the process of implementation of CBL rather than a fault in the ‘‘function’’ of CBL per se. Failure to engage faculty and students; insufficient training/preparation, and a lack of alignment with clinical rotations and assessment were some of the factors that led to unsatisfactory implementation of CBL. From these findings, the authors derived a checklist to assist in the implementation of educational interventions aimed at enhancing surgical teaching and learning. Case-based learning is an adult learning approach that is gaining popularity across healthcare specialties as a means of enhancing workplace-based learning [2]. Authentic cases stimulate the acquisition of knowledge, skills, and attitudes in a safe learner-centered environment. This team-based approach also promotes deeper understanding through interaction and dialogue between learners, which goes beyond passive attendance at a lecture, and ensures building and exchange of thoughts [3]. The role of the faculty is to facilitate learner dialogue and to support understanding. Our team has recently taken a very similar approach, aimed at junior doctors in their first two years of clinical practice (termed ‘‘Foundation trainees’’ in the UK)—a population not dissimilar from that of Nordquist et al. In collaboration with the training provider of North West England (the North Western Deanery) we have implemented an innovative case-based Patient Safety training program for these junior doctors. Called Lessons Learnt: Building a Safer Foundation, the program comprises monthly sessions built into the Foundation teaching program. Foundation trainees lead a structured peer-group discussion of a patient safety incident (adverse healthcare event) encountered in the workplace in order to analyze its contributing factors using a validated framework (London Protocol [4]) and pose potential solutions. Senior clinicians (Attendings) trained in incident analysis facilitate the sessions. As an emerging discipline within medical and surgical education, the implementation of patient safety training brings its own challenges. Various barriers to curricular integration have been identified, including poor learner engagement; lack of expert faculty; local institutional culture; and a lack of evidence for the impact of such training on healthcare [5]. Our approach to implementation was to address each of these barriers to ensure sustainable integration within the Foundation teaching programs across 16 hospital sites (hosting over 1,000 Foundation trainees) beginning in January 2011. The first step was to engage all stakeholders. Foundation training directors, administrators, trainees, and prospective facilitators were invited to a half-day ‘‘launch event’’ to M. Ahmed (&) S. Arora C. Vincent N. Sevdalis Department of Surgery and Cancer, Imperial College London, St. Mary’s Hospital, Wright Fleming Building, Norfolk Place, London W2 1PG, UK e-mail: maria.ahmed@imperial.ac.uk

Towards an open and learning safety culture

Editor – We read the study by Varjavand et al., entitled ‘Changes in intern attitudes toward medical error and disclosure’, and the associated commentary with great interest. It is indeed heartening to find that attitudes in relation to patient safety have improved somewhat amongst interns. We agree that reflection on personal errors rather than on hypothetical scenarios or on the mistakes of others is essential to ensure deep learning and to promote positive behavioural change.