Gene Hobbs

Assessing teamwork in medical education and practice: relating behavioural teamwork ratings and clinical performance.

Background: Problems with communication and team coordination are frequently linked to adverse events in medicine. However, there is little experimental evidence to support a relationship between observer ratings of teamwork skills and objective measures of clinical performance. Aim: Our main objective was to test the hypothesis that observer ratings of team skill will correlate with objective measures of clinical performance. Methods: Nine teams of medical students were videotaped performing two types of teamwork tasks: (1) low fidelity classroom-based patient assessment and (2) high fidelity simulated emergent care. Observers used a behaviourally anchored rating scale to rate each individual on skills representative of assertiveness, decision-making, situation assessment, leadership, and communication. A checklist-based measure was used to assess clinical team performance. Results: Moderate to high inter-observer correlations and moderate correlations between cases established the validity of a behaviourally anchored team skill rating tool for simulated emergent care. There was moderate to high correlation between observer ratings of team skill and checklist-based measures of team performance for the simulated emergent care cases (r = 0.65, p = 0.06 and r = 0.97, p < 0.0001). Conclusions: These results provide prospective evidence of a positive relationship between observer ratings of team skills and clinical team performance in a simulated dynamic health care task.

Preliminary Report on the Use of High-Fidelity Simulation in the Training of Study Coordinators Conducting a Clinical Research Protocol

Training of health care research personnel is acritical component of quality assurance in clinical trials. Interactivity (such as simulation) is desirable compared with traditional methods of teaching. We hypothesized that the addition of an interactive simulation exercise to standard training methods would increase the confidence of study coordinators. A simulation exercise was developed to replicate a complex clinical trial. Eighteen study coordinators completed pre- and postexercise confidence questionnaires. Questions were targeted at key trial components using a 0–10 scale (not confident to confident) and were categorized using Bloom’s Taxonomy. The primary analysis compared overall mean pre-and postexercise responses. Secondary analyses assessed affective, psychomotor, and cognitive confidence. Significance was at P < 0.05. A significant increase in overall confidence (8.64 versus 5.77; P < 0.0001) was reproduced in the subcategory analyses (affective, 8.24 versus 4.89; P < 0.0001; cognitive, 8.75 versus 6.42; P = 0.0003; psychomotor, 8.63 versus 5.26; P < 0.0001). A high level of internal consistency and reliability in question responses within domains was observed, validating the questionnaire tool. In this preliminary report, we confirmed that addition of a simulation exercise to the training of study coordinators resulted in increased confidence. Simulation exercises should be considered when training study coordinators for clinical research trials.

Coding and Visualizing Eye Tracking Data in Simulated Anesthesia Care

Eye tracking can be a valuable tool for collecting data about perception and attention in task performance, but its use in human factors research has been limited. This may be due to the fact that the coding and visualization of eye tracking data can be difficult and time-consuming. In this paper we introduce a video-coding application for coding and analyzing eye tracking data. We discuss various methods for visualizing these data for the purposes of identifying patterns or trends that can then be more formally analyzed. We also present several visualization examples from the simulated anesthesia care environment.

Standardized assessment for evaluation of team skills: validity and feasibility.

Introduction The authors developed a Standardized Assessment for Evaluation of Team Skills (SAFE-TeamS) in which actors portray health care team members in simulated challenging teamwork scenarios. Participants are scored on scenario-specific ideal behaviors associated with assistance, conflict resolution, communication, assertion, and situation assessment. This research sought to provide evidence of the validity and feasibility of SAFE-TeamS as a tool to support the advancement of science related to team skills training. Methods Thirty-eight medical and nursing students were assessed using SAFE-TeamS before and after team skills training. The SAFE-TeamS pretraining and posttraining scores were compared, and participants were surveyed. Generalizability analysis was used to estimate the variance in scores associated with the following: examinee, scenario, rater, pretraining/posttraining, examinee type, rater type (actor-live vs. external rater–videotape), actor team, and scenario order. Results The SAFE-TeamS scores reflected improvement after training and were sensitive to individual differences. Score variance due to rater was low. Variance due to scenario was moderate. Estimates of relative reliability for 2 raters and 8 scenarios ranged from 0.6 to 0.7. With fixed scenarios and raters, 2 raters and 2 scenarios, reliability is greater than 0.8. Raters believed SAFE-TeamS assessed relevant team skills. Examinees’ responses were mixed. Conclusions The SAFE-TeamS was sensitive to individual differences and team skill training, providing evidence for validity. It is not clear whether different scenarios measure different skills and whether the scenarios cover the necessary breadth of skills. Use of multiple scenarios will support assessment across a broader range of skills. Future research is required to determine whether assessments using SAFE-TeamS will translate to performance in clinical practice.

Evaluation of the System O2 Inc portable nonpressurized oxygen delivery system

OBJECTIVE To evaluate the performance of the System O2 portable non-pressurized delivery system (SysO2). This device produces oxygen through chemical reaction and might have utility for emergency/field use. METHODS Performance was evaluated with 10 unmanned trials conducted under standard laboratory conditions. Measures included oxygen flow (mean and peak), total oxygen yield, and system weight-indexed yield. RESULTS Oxygen flow peaked at 5.74 +/- 0.28 L x min(-1) (mean +/- SD) at 16.9 +/- 1.5 minutes before rapidly falling to zero. Mean flow was 2.98 +/- 1.52 L x min(-1) with a total yield of 62.9 +/- 6.6 L. Mean oxygen fraction was 0.96 +/- 0.15. The weight per unit of oxygen is substantially higher than for commercially available pressurized cylinders; e.g., 47.7 vs. 10.2 g x L(-1) for the small 246 L M9 cylinder. CONCLUSIONS Given the limited flow rate and supply duration, we believe the SysO2 system does not offer significant advantage over the available pressurized oxygen systems as a source for emergency oxygen.

Patient Load Effects on Response Time to Critical Arrhythmias in Cardiac Telemetry: A Randomized Trial

Objectives: Remotely monitored patients may be at risk for a delayed response to critical arrhythmias if the telemetry watchers who monitor them are subject to an excessive patient load. There are no guidelines or studies regarding the appropriate number of patients that a single watcher may safely and effectively monitor. Our objective was to determine the impact of increasing the number of patients monitored on response time to simulated cardiac arrest. Design: Randomized trial. Setting: Laboratory-based experiment. Subjects: Forty-two remote telemetry technicians and nurses from cardiac units. Interventions: Number of patients monitored in a simulation of cardiac telemetry monitoring work. Measurements and Main Results: We carried out a study to compare response times to ventricular fibrillation across five patient loads: 16, 24, 32, 40, and 48 patients. The simulation replicated the work of telemetry watchers using a combination of real recorded patient electrocardiogram signals and a simulated patient experiencing ventricular fibrillation. Study participants were assigned to one of the five patient loads and completed a 4-hour monitoring session, during which they performed tasks—including event documentation and phone calls to report events—similar to real monitoring work. When the simulated patient sustained ventricular fibrillation, the time required to report this arrhythmia was recorded. As patient loads increased, there was a statistically significant increase in response times to the ventricular fibrillation. In addition, frequency of failure to meet a response time goal of less than 20 seconds was significantly higher in the 48-patient condition than in all other conditions. Task performance decreased as patient load increased. Conclusions: As participants monitored more patients in a laboratory setting, their performance with respect to recognizing critical and noncritical events declined. This study has implications for the design of remote telemetry work and other patient monitoring tasks in critical and intermediate care units.

Fire in the operating room.

CURRICULAR INFORMATION Educational Rationale Fire in the operating room (OR) is a rare but critical event. According to the Anesthesia Patient Safety Foundation, hundreds of fires occur in the United States yearly.1 The majority of fires occur during head and neck surgery due to the presence of oxygen and the extensive use of lasers.2 Surgical fires can be prevented by educating staff about risk and prevention strategies. Prevention depends on understanding how the elements of the fire triad interact, recognizing how standard operating room equipment can initiate a fire, and vigilance monitoring for the circumstances that increase the likelihood of fire.3 Education on fire prevention and mitigation should be a part of all undergraduate medical, nursing, and other allied health profession education.4 Using a case report from Barker and Polson5 as our inspiration, we designed this scenario to have learners reflect on preventing and effectively managing an OR fire.6 Learning Objectives Learners will be assessed on the following Accreditation Council for Graduate Medical Education general competencies:

The "Simulation Roulette" game.

Introduction: “Simulation Roulette” is a new method of “on-the-fly” simulation scenario creation that incorporates a game-like approach to critical scenarios and emphasizes prescenario preparation. We designed it to complement our traditional anesthesia simulation curriculum, in which residents are exposed to predefined “critical” scenarios. During typical scenarios, trainees are often given minimum preparatory information; they then start the scenario knowing only that “something bad” is going to happen. As a result, trainees often report anxiety, which can be a barrier to learning. To overcome this barrier and to augment traditional critical incident training, we developed the “Simulation Roulette” game. Methods: “Simulation Roulette” consists of premade cards that are randomly selected to create a patient, another set of premade cards to assist in selecting “complications,” worksheets to guide a thorough “prebrief” discussion before the scenario, and scoresheets to facilitate the “debrief” discussion at the end. Similar to traditional scenarios, it requires coordination by a facilitator to ensure plausible scenarios and evaluation of trainee performance. Results: Although we have not conducted formal testing, we believe that (1) incorporating an element of random chance to scenario selection, (2) using a game-like framework, and (3) emphasizing the “prebrief” portion of simulation all have the potential to decrease trainee anxiety. Conclusions: We present the rationale for designing such a game; examples of instructions, cards, and scoresheets; and our initial experience with implementing this game within our simulation curriculum.

Board 525 - Technology Innovations Abstract Development of a Multiplayer Virtual-Reality Obstetric Hemorrhage Simulation Program (Submission #747)

Introduction/Background Obstetric hemorrhage (OH) is a leading cause of maternal morbidity and mortality worldwide. Multidisciplinary initiatives are needed to improve outcomes in women experiencing OH. To this end, the Joint Commission recommends the use of clinical protocols for OH management.1 These protocols must be practiced by the team to ensure effectiveness, but manikin-based simulations can be expensive and challenging to coordinate with all team members. In partnership with the Virtual Heroes division of Applied Research Associates, Inc., we developed a web-based computer OH simulation for the purpose of training and evaluating participants in use of an OH protocol as well as reinforcing teamwork and communication. Methods The Obstetric Hemorrhage Scenario is an immersive multiplayer learning application built using the ILE@D platform and deployed via web plugins. Participant communication is accomplished via voice over IP (VoIP) and text chat windows. Participant move avatars through a virtual patient room and are able to interact with both the patient and objects in the room. Scenario content is constructed around an OH due to uterine atony in a primigravida. The instructor controls all patient physiology by either manual control or macro-based controls, and can adjust patient vitals and appearance based on participant therapeutic interventions. Participants are able to perform physical exam, administer fluids and medications, order labs, order and transfuse blood components, and view patient data in an electronic health record. Teamwork and communication behaviors of participants are observed and evaluated according to TeamSTEPPS principles. Knowledge learning objectives include: 1) early recognition of the signs of uterine atony; 2) appropriate use of uterotonics; 3) activation of the OH protocol; and 4) ordering laboratory studies and transfusing blood products according to the OH protocol and laboratory data. Key elements of transfusion practice observed include two-person bedside verification against the patient armband, ensuring ABO compatibility of blood products, and observation of blood product expiration times. Simulation build is complete and currently undergoing multidisciplinary beta testing in both the United States and Australia, with plans to incorporate this simulation into medical student training in 2013. Results: Conclusion Our construction of a web-based virtual reality simulation has created a training tool that emphasizes team communication, evidence-based obstetric interventions, safe transfusion practices, and appropriate blood management for OH. This customizable, cost-effective tool allows for institution-wide training of team members in the OH protocol and provides a widely available opportunity for research into individual- and team-decision making during OH management without the need for geographic co-localization of participants. References 1. Joint Commission on Accreditation of Healthcare Organizations, USA. Preventing Maternal Death. Sentinel Event Alert, January 26, 2010; 44:1-4. Disclosures CSL Behring, Octapharma, TEM Systems Inc. Abbott, Pfizer Guidepoint, Maven Applied Research Associates-proprietary interest in jointly developed projects.

Board 538 - Technology Innovations Abstract Simulating Remote Cardiac Telemetry Monitoring (Submission #1001)

Introduction/Background Over 370,000 cardiopulmonary resuscitations are attempted each year in US hospitals.1 For many, pulseless ventricular tachycardia or ventricular fibrillation (VT/VF) is the first monitored arrhythmia, which may be treated successfully with prompt defibrillation. Yet for 30% of patients, defibrillation is delayed more than 2 minutes, reducing their chance of survival to hospital discharge by half.2 To increase the potential for timely detection of cardiac events, many at-risk patients are monitored by telemetry technicians. However, decisions regarding the appropriate number of patients that a single technician may safely and effectively monitor are primarily based on technological capabilities and not on our understanding of human information processing limitations. Simulation provides an opportunity to measure responses to life-threatening cardio-respiratory events in a time frame and with a degree of accuracy not feasible through assessment of response to true events. Our objective is to determine the impact of increasing the number of patients monitored on response time to cardio-respiratory events. To achieve this objective, we designed simulation of cardiac telemetry monitoring Methods Currently, we are carrying out a randomized controlled trial to compare response times to VF across five number-of-patient conditions. The simulation replicates the work of cardiac telemetry technicians using a combination of real patient data and a simulated patient experiencing VF. We video and audio-recorded true patient data with a single simulated patient embedded in the patient set. The technical implementation involved connecting an ECG rhythm simulator into the hospital’s network that transmits physiological signals to remote telemetry monitors. The signal appears in exactly the same way as it would appear for a real patient. Study participants are randomly assigned to one of the 5 patient loads and complete a four-hour monitoring session, during which they perform tasks similar to real monitoring work, such as calling patients’ providers to report events and documenting changes. After about 3 hours, the simulated patient sustains VF and the time required for the participants to report this arrhythmia is recorded. Study participants are asked to complete a survey regarding the realism of the simulation. 28 technicians and nurses from cardiac units have completed the study to date. As expected, there is a trend of increasing response time to the lethal arrhythmia as the number of patients monitored increases. Survey Results are presented in the Table. Results: Conclusion We simulated the work of remote cardiac telemetry technicians to study the effect of increasing the number of patients monitored on response time to a simulated arrhythmia. Overall, study participants perceived the experience to be relatively realistic. The knowledge to be gained will inform efforts to study this problem in real-world cardiac telemetry. References 1. Ballew KA, Philbrick JT. Causes of variation in reported in-hospital CPR survival: A critical review. Resuscitation 1995;30:203-15. 2. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in-hospital cardiac arrest. N Engl J Med 2008;358:9-17. Disclosures Abbott, Pfizer Guidepoint, Maven Applied Research Associates-proprietary interest in jointly developed projects BMS, Pfizer, The Medicines Company, Merck, Novartis, Medtronic, Boehringer Ingelheim, Astra Zeneca BMS, Pfizer, The Medicines Company, Boehringer Ingelheim, Daiitchi, Astra Zeneca.