Richard Reznick

Communication failures in the operating room: an observational classification of recurrent types and effects

Background: Ineffective team communication is frequently at the root of medical error. The objective of this study was to describe the characteristics of communication failures in the operating room (OR) and to classify their effects. This study was part of a larger project to develop a team checklist to improve communication in the OR. Methods: Trained observers recorded 90 hours of observation during 48 surgical procedures. Ninety four team members participated from anesthesia (16 staff, 6 fellows, 3 residents), surgery (14 staff, 8 fellows, 13 residents, 3 clerks), and nursing (31 staff). Field notes recording procedurally relevant communication events were analysed using a framework which considered the content, audience, purpose, and occasion of a communication exchange. A communication failure was defined as an event that was flawed in one or more of these dimensions. Results: 421 communication events were noted, of which 129 were categorized as communication failures. Failure types included “occasion” (45.7% of instances) where timing was poor; “content” (35.7%) where information was missing or inaccurate, “purpose” (24.0%) where issues were not resolved, and “audience” (20.9%) where key individuals were excluded. 36.4% of failures resulted in visible effects on system processes including inefficiency, team tension, resource waste, workaround, delay, patient inconvenience and procedural error. Conclusion: Communication failures in the OR exhibited a common set of problems. They occurred in approximately 30% of team exchanges and a third of these resulted in effects which jeopardized patient safety by increasing cognitive load, interrupting routine, and increasing tension in the OR.

Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination.

PURPOSE: To compare the psychometric properties of checklists, global rating scales preceded by a checklist, and global rating scales alone in assessing surgery residents performances on an OSCE-like technical skills examination. METHOD: In 1996, 53 general surgery residents with one to six years of postgraduate training participated in a performance-based examination of technical skills consisting of eight 15-minute stations (bench-model simulations of operative procedures in general surgery). Two qualified surgeons marked at each station, one using a task-specific checklist (C) and a subsequent global rating scale (Gc), the other using a global rating scale only (G). RESULTS: Interstation reliabilities measured by Cronbachs alpha were .79 for C, .89 for Gc, and .85 for G. A series of multiple regressions predicting level of training from test scores revealed an R2 of .584 for C alone, which increased to .711 when Gc was entered after (p < .001), and increased to .704 when G was entered after C (p < .001). However, R2 for Gc alone was .711, and for G alone was .704, neither of which changed when C was entered into the prediction (p > .10). The R2 for Gc and G predicting level of training (.725) was not significantly greater than that of either Gc or G alone. A very similar pattern of results was seen when C, Gc, and G were used to predict independent evaluations of the operative outcomes. CONCLUSIONS: Global rating scales scored by experts showed higher inter-station reliability, better construct validity, and better concurrent validity than did checklists. Further, the presence of the checklists did not improve the reliability or validity of the global rating scale over that of the global rating scale alone. These results suggest that global rating scales administered by experts are a more appropriate summative measure when assessing candidates on performance-based examinations.

Validation of an objective structured assessment of technical skill for surgical residents.

PURPOSE: This study examined the concurrent validity of the Objective Structured Assessment of Technical Skill (OSATS), a new test of technical skill for general surgery residents. METHOD: Twelve residents (six in their senior, or fifth, year and six in their junior, or third, year) at the University of Toronto in 1994-95 were ranked within level of training according to their OSATS marks and by surgical faculty. Correspondence between OSATS and faculty rankings was assessed using Spearman rank-order correlation coefficients. RESULTS: The correlations between test scores and faculty rankings were generally high for the senior residents but low for the junior residents. CONCLUSION: Scores on the OSATS accurately reflect the independent opinions of faculty regarding the technical skills of senior residents, suggesting that it is a valid measure of technical skill for these individuals. The scores did not, however, reproduce faculty rankings of the junior residents. Whether this was a failing of the OSATS or the faculty rankings requires further study.

Getting teams to talk: development and pilot implementation of a checklist to promote interprofessional communication in the OR

Background: Pilot studies of complex interventions such as a team checklist are an essential precursor to evaluating how these interventions affect quality and safety of care. We conducted a pilot implementation of a preoperative team communication checklist. The objectives of the study were to assess the feasibility of the checklist (that is, team members’ willingness and ability to incorporate it into their work processes); to describe how the checklist tool was used by operating room (OR) teams; and to describe perceived functions of the checklist discussions. Methods: A checklist prototype was developed and OR team members were asked to implement it before 18 surgical procedures. A research assistant was present to prompt the participants, if necessary, to initiate each checklist discussion. Trained observers recorded ethnographic field notes and 11 brief feedback interviews were conducted. Observation and interview data were analyzed for trends. Results: The checklist was implemented by the OR team in all 18 study cases. The rate of team participation was 100% (33 vascular surgery team members). The checklist discussions lasted 1–6 minutes (mean 3.5) and most commonly took place in the OR before the patient’s arrival. Perceived functions of the checklist discussions included provision of detailed case related information, confirmation of details, articulation of concerns or ambiguities, team building, education, and decision making. Participants consistently valued the checklist discussions. The most significant barrier to undertaking the team checklist was variability in team members’ preoperative workflow patterns, which sometimes presented a challenge to bringing the entire team together. Conclusions: The preoperative team checklist shows promise as a feasible and efficient tool that promotes information exchange and team cohesion. Further research is needed to determine the sustainability and generalizability of the checklist intervention, to fully integrate the checklist routine into workflow patterns, and to measure its impact on patient safety.

Forming professional identities on the health care team: discursive constructions of the 'other' in the operating room.

Backgroundu2002 Inter‐professional health care teams represent the nucleus of both patient care and the clinical education of novices. Both activities depend upon the‘talk’ that team members use to interact with one another. This study explored team members’ interpretations of tense team communications in the operating room (OR).

Resident self-assessment of operative performance

BACKGROUNDnIn medicine, the development of expertise requires the recognition of ones capabilities and limitations. This study aimed to verify the accuracy of self-assessment for the performance of a surgical task, and to determine whether self-assessment may be improved through self-observation or exposure to relevant standards of performance.nnnMETHODSnTwenty-six senior surgical residents were videotaped performing a laparoscopic Nissen fundoplication in a pig. Experts rated the videos using two scoring systems. Subjects evaluated their performances after performance of the Nissen, after self-observation of their videotaped performance, and after review of four videotaped benchmark performances.nnnRESULTSnExpert interrater reliability was 0.66 (intraclass correlation coefficient). The correlation between experts and residents self-evaluations was initially moderate (r = 0.50, P <0.01), increasing significantly after the residents reviewed their own videotaped performance to r = 0.63 (Deltar = 0.13, P <0.01), yet did not change after review of the benchmarks.nnnCONCLUSIONSnSelf-observation of videotaped performance improved the residents ability to self-evaluate.

Using operative outcome to assess technical skill

BACKGROUNDnThis study examined whether an operative product and time to completion could serve as measures of technical skill.nnnMETHODSnNine final-year (PGY5) and 11 penultimate-year (PGY4) general surgery residents participated in a 6-station bench model examination. Time to completion was recorded. Twelve faculty surgeons (2 per station) evaluated the quality of the final product using a 5-point scale.nnnRESULTSnThe mean interrater reliability was 0. 59 for product quality. Interstation reliability was 0.59 for analysis of the final product and 0.72 for time to completion. There was 63% and 78% agreement between attendings ratings and product quality and time scores respectively. PGY5s mean product quality score was 4.14 +/- 0.26, compared with 3.82 +/- 0.33 for PGY4s (P < 0.05). PGY5s mean time was 110 +/- 19 minutes compared with PGY4s 132 +/- 15 (P < 0.05).nnnCONCLUSIONSnAnalysis of the operative end product and time to completion offer efficient alternatives to on-line examiner scoring for bench model examinations of technical competence.

Who should rate candidates in an objective structured clinical examination

PURPOSE. To determine who is the better rater of history taking in an objective structured clinical examination (OSCE): a physician or a standardized patient (SP). METHOD. During the 1991 pilot administration of an OSCE for the Medical Council of Canadas qualifying examination, five history-taking stations were videotaped. Candidates at these stations were scored by three raters: a physician (MD), an SP observer (SPO), and an SP rating from recall (SPR). To determine the validity of each raters scores, these scores were compared with a “gold standard”, which was the average of videotape ratings by three physicians, each scoring independently. Analysis included both correlations with the standard and a repeated-measures analysis of variance (ANOVA) comparing raters mean scores on each station with mean scores of the gold standard. RESULTS. Ninety-one videotapes were scored by the “gold-standard” physicians. Correlations with the standard showed no clear preference for MD, SPO, or SPR raters. ANOVAs revealed significant differences from the standard on three stations for the SPR, two stations for the SPO, and one stations for the MD. CONCLUSIONS. An MD rater is less likely to differ from a standard established by a consensus of MD ratings than are SP raters rating from recall. If an MD cannot be used, an SP observer is preferable to an SP rating from recall.

Three-Year Experience with an Innovative, Modular Competency-Based Curriculum for Orthopaedic Training

In response to multiple stresses in current surgical education, we developed a new model of orthopaedic training that combines curricular reform with a competency-based framework. For the past three years, this pilot program has been run in parallel to our conventional curriculum for a select number of residents. In this article, we share our initial experience with this approach to training and describe its successes and challenges. We review the existing concerns with surgical training in a new era of work-hour restrictions and describe the pedagogical rationale for the model that we have developed. We then discuss the design of this curriculum, including the basic tenets and principles that guided our approach. Finally, we detail our preliminary results, which add evidence that a focused, modular-based program, with concentrated teaching of technical skills and frequent formative and summative evaluations, can result in rapid acceleration in surgical competency, knowledge acquisition, and comprehensive professional skills. This new model deserves further study and consideration for implementation on a broader scale in today’s challenging medical education environment.nnThere have been questions about our general approach to surgical education for quite some time. Charles Bosk catalogued some of the good and much of the bad that went along with residency education a quarter of a century ago1. Similarly, William Nolan2 described the rigors of surgical training in Bellevue Hospital in New York, NY. The reverberations from The Bristol Royal Infirmary Inquiry3 sparked a focus on the issue of patient safety. This issue was reinforced with the dissemination of the Institute of Medicine report, To Err Is Human: Building a Safer Health System 4. Preoccupation with the issue of patient safety has led to many positive outcomes in patient care, but an important side effect has been decreased opportunities for residents for …

Reflections on Competency-Based Education and Training for Surgical Residents

Although a number of surgical training institutions have started to adopt competency-based education (CBE) frameworks for training, the debate about the value of this model continues. Some proponents regard CBE as a method of guaranteeing residents competence, whereas others consider CBE to be reductive and lacking the richness in experiences that the traditional model offers. In this article, we reflect on CBE and review some salient attempts to implement CBE in surgical education. We identify challenges facing postgraduate surgical education, some of which are motivating educators to consider incorporating CBE into their curricula. We look at some purported advantages and disadvantages of CBE and describe initial reports from CBE programs currently being developed.