Heléne Nilsson

Evaluation of medical command and control using performance indicators in a full-scale, major aircraft accident exercise.

INTRODUCTION Large, functional, disaster exercises are expensive to plan and execute, and often are difficult to evaluate objectively. Command and control in disaster medicine organizations can benefit from objective results from disaster exercises to identify areas that must be improved. OBJECTIVE The objective of this pilot study was to examine if it is possible to use performance indicators for documentation and evaluation of medical command and control in a full-scale major incident exercise at two levels: (1) local level (scene of the incident and hospital); and (2) strategic level of command and control. Staff procedure skills also were evaluated. METHODS Trained observers were placed in each of the three command and control locations. These observers recorded and scored the performance of command and control using templates of performance indicators. The observers scored the level of performance by awarding 2, 1, or 0 points according to the template and evaluated content and timing of decisions. Results from 11 performance indicators were recorded at each template and scores greater than 11 were considered as acceptable. RESULTS Prehospital command and control had the lowest score. This also was expressed by problems at the scene of the incident. The scores in management and staff skills were at the strategic level 15 and 17, respectively; and at the hospital level, 17 and 21, respectively. CONCLUSIONS It is possible to use performance indicators in a full-scale, major incident exercise for evaluation of medical command and control. The results could be used to compare similar exercises and evaluate real incidents in the future.

Performance Indicators as Quality Control for Testing and Evaluating Hospital Management Groups: A Pilot Study

INTRODUCTION An important issue in disaster medicine is the establishment of standards that can be used as a template for evaluation. With the establishment of standards, the ability to compare results will improve, both within and between different organizations involved in disaster management. OBJECTIVE Performance indicators were developed for testing in simulations exercises with the purpose of evaluating the skills of hospital management groups. The objective of this study is to demonstrate how these indicators can be used to create numerically expressed results that can be compared. METHODS Three different management groups were tested in standardized simulation exercises. The testing took place according to the organizations own disaster plan and within their own facilities. Trained observers used a pre-designed protocol of performance indicators as a template for the evaluation. RESULTS The management group that scored lowest in management skills also scored lowest in staff skills. CONCLUSION The use of performance indicators for evaluating the management skills of hospital groups can provide comparable results in testing situations and could provide a new tool for quality improvement of evaluations of real incidents and disasters.

Management of resources at major incidents and disasters in relation to patient outcome : A pilot study of an educational model

Background and objectives Organizations involved in disaster response often have a defined operative level of management (command and control) that can take the overall decisions regarding the mobilization and distribution of resources and distribution of casualties. This level of management can be referred to as strategic management. The aim of this pilot study was to show the possibility, in simulation exercises, to relate decisions made regarding resources to patient outcome. Methods The simulation system used measures to determine if lifesaving interventions are performed in time or not in relation to patient outcome. Evaluation was made with sets of performance indicators as templates and all management groups were evaluated not only as to how the decisions were made (management skills), but also how staff work was performed (staff procedure skills). Results Owing to inadequate response and insufficient distribution of patients to hospitals, 11 ‘patients’ died in the simulated incident, a fire at a football stand with subsequent collapse. The strategic level of management received 16 points out of a possible 22 according to a predesigned template of performance indicators. Conclusion The pilot study demonstrated the possibility to, in simulation exercises, relate decisions made regarding resources to patient outcome. This training technique could possibly lead to increased knowledge in what decisions are crucial to make in an early phase to minimize mortality and morbidity.

Increased situation awareness in major incidents - radio frequency identification (RFID) technique : a promising tool

INTRODUCTION In mass-casualty situations, communications and information management to improve situational awareness is a major challenge for responders. In this study, the feasibility of a prototype system that utilizes commercially available, low-cost components, including Radio Frequency Identification (RFID) and mobile phone technology, was tested in two simulated mass-casualty incidents. METHODS The feasibility and the direct benefits of the system were evaluated in two simulated mass-casualty situations: one in Finland involving a passenger ship accident resulting in multiple drowning/hypothermia patients, and another at a major airport in Sweden using an aircraft crash scenario. Both simulations involved multiple agencies and functioned as test settings for comparing the disaster managements situational awareness with and without using the RFID-based system. Triage documentation was done using both an RFID-based system, which automatically sent the data to the Medical Command, and a traditional method using paper triage tags. The situational awareness was measured by comparing the availability of up-to date information at different points in the care chain using both systems. RESULTS Information regarding the numbers and status or triage classification of the casualties was available approximately one hour earlier using the RFID system compared to the data obtained using the traditional method. CONCLUSIONS The tested prototype system was quick, stable, and easy to use, and proved to work seamlessly even in harsh field conditions. It surpassed the paper-based system in all respects except simplicity of use. It also improved the general view of the mass-casualty situations, and enhanced medical emergency readiness in a multi-organizational medical setting. The tested technology is feasible in a mass-casualty incident; further development and testing should take place.

Performance indicators for initial regional medical response to major incidents: a possible quality control tool

BackgroundTimely decisions concerning mobilization and allocation of resources and distribution of casualties are crucial in medical management of major incidents. The aim of this study was to evaluate documented initial regional medical responses to major incidents by applying a set of 11 measurable performance indicators for regional medical command and control and test the feasibility of the indicators.MethodsRetrospective data were collected from documentation from regional medical command and control at major incidents that occurred in two Swedish County Councils. Each incident was assigned to one of nine different categories and 11 measurable performance indicators for initial regional medical command and control were systematically applied. Two-way analysis of variance with one observation per cell was used for statistical analysis and the post hoc Tukey test was used for pairwise comparisons.ResultsThe set of indicators for regional medical command and control could be applied in 102 of the130 major incidents (78%), but 36 incidents had to be excluded due to incomplete documentation. The indicators were not applicable as a set for 28 incidents (21.5%) due to different characteristics and time frames. Based on the indicators studied in 66 major incidents, the results demonstrate that the regional medical management performed according to the standard in the early phases (1–10 min after alert), but there were weaknesses in the secondary phase (10–30 min after alert). The significantly lowest scores were found for Indicator 8 (formulate general guidelines for response) and Indicator 10 (decide whether or not resources in own organization are adequate).ConclusionsMeasurable performance indicators for regional medical command and control can be applied to incidents that directly or indirectly involve casualties provided there is sufficient documentation available. Measurable performance indicators can enhance follow- up and be used as a structured quality control tool as well as constitute measurable parts of a nationally based follow-up system for major incidents. Additional indicators need to be developed for hospital-related incidents such as interference with hospital infrastructure.

Studying distributed cognition of simulation-based team training with DiCoT

Abstract Health care organizations employ simulation-based team training (SBTT) to improve skill, communication and coordination in a broad range of critical care contexts. Quantitative approaches, such as team performance measurements, are predominantly used to measure SBTTs effectiveness. However, a practical evaluation method that examines how this approach supports cognition and teamwork is missing. We have applied Distributed Cognition for Teamwork (DiCoT), a method for analysing cognition and collaboration aspects of work settings, with the purpose of assessing the methodology’s usefulness for evaluating SBTTs. In a case study, we observed and analysed four Emergo Train System® simulation exercises where medical professionals trained emergency response routines. The study suggests that DiCoT is an applicable and learnable tool for determining key distributed cognition attributes of SBTTs that are of importance for the simulation validity of training environments. Moreover, we discuss and exemplify how DiCoT supports design of SBTTs with a focus on transfer and validity characteristics. Practitioner Summary: In this study, we have evaluated a method to assess simulation-based team training environments from a cognitive ergonomics perspective. Using a case study, we analysed Distributed Cognition for Teamwork (DiCoT) by applying it to the Emergo Train System®. We conclude that DiCoT is useful for SBTT evaluation and simulator (re)design.

Combining performance and outcome indicators can be used in a standardized way: a pilot study of two multidisciplinary, full-scale major aircraft exercises.

BackgroundDisaster medicine is a fairly young scientific discipline and there is a need for the development of new methods for evaluation and research. This includes full-scale disaster exercisers. A standardized concept on how to evaluate these exercises, could lead to easier identification of pitfalls caused by system-errors in the organization. The aim of this study was to demonstrate the feasibility of using a combination of performance and outcome indicators so that results can be compared in standardized full-scale exercises.MethodsTwo multidisciplinary, full-scale exercises were studied in 2008 and 2010. The panorama had the same setup. Sets of performance indicators combined with indicators for unfavorable patient outcome were recorded in predesigned templates. Evaluators, all trained in a standardized way at a national disaster medicine centre, scored the results on predetermined locations; at the scene, at hospital and at the regional command and control.ResultsAll data regarding the performance indicators of the participants during the exercises were obtained as well as all data regarding indicators for patient outcome. Both exercises could therefore be compared regarding performance (processes) as well as outcome indicators. The data from the performance indicators during the exercises showed higher scores for the prehospital command in the second exercise 15 points and 3 points respectively. Results from the outcome indicators, patient survival and patient complications, demonstrated a higher number of preventable deaths and a lower number of preventable complications in the exercise 2010. In the exercise 2008 the number of preventable deaths was lower and the number of preventable complications was higher.ConclusionsStandardized multidisciplinary, full-scale exercises in different settings can be conducted and evaluated with performance indicators combined with outcome indicators enabling results from exercises to be compared. If exercises are performed in a standardized way, results may serve as a basis for lessons learned. Future use of the same concept using the combination of performance indicators and patient outcome indicators may demonstrate new and important evidence that could lead to new and better knowledge that also may be applied during real incidents.

Simulation-assisted burn disaster planning

The aim of the study was to evaluate the Swedish medical systems response to a mass casualty burn incident in a rural area with a focus on national coordination of burn care. Data were collected from two simulations of a mass casualty incident with burns in a rural area in the mid portion of Sweden close to the Norwegian border, based on a large inventory of emergency resources available in this area as well as regional hospitals, university hospitals and burn centres in Sweden and abroad. The simulation system Emergo Train System(®) (ETS) was used and risk for preventable death and complications were used as outcome measures: simulation I, 18.5% (n=13) preventable deaths and 15.5% (n=11) preventable complications; simulation II, 11.4% (n=8) preventable deaths and 11.4% (n=8) preventable complications. The last T1 patient was evacuated after 7h in simulation I, compared with 5h in simulation II. Better national coordination of burn care and more timely distribution based on the experience from the first simulation, and possibly a learning effect, led to a better patient outcome in simulation II. The experience using a system that combines both process and outcome indicators can create important results that may support disaster planning.

Designing a Digital Medical Management Training Simulator Using Distributed Cognition Theory

Background. Training of medical professionals is important to improve care during mass-causality events. Therefore, it is essential to extend knowledge on how to design valid and usable simulation-based training environments. Purpose. This article investigates how distributed cognition and simulation theory concepts can guide design of simulation-based training environments. We present the design and user evaluation of DigEmergo, a simulator for training and assessing emergency medicine management. Design approach. A prior Distributed Cognition in Teamwork (DiCoT) analysis of the Emergo Train System (ETS) guided the design process. The design objective of DigEmergo was to be useful, usable, retain distributed cognition features of ETS, and strengthen validity and output reliability. Evaluation. Eight expert ETS instructors participated in a formative system evaluation. The Technology Assessment Model (TAM) questionnaire was used to measure usefulness and ease of use. Observations and post-test interviews were conducted to contextualize the measures. Results. The results showed that DigEmergo was perceived as somewhat to quite useful and somewhat easy to use. Overall, expert users considered DigEmergo promising and successful in retaining core ETS features. Conclusions. The study indicates that a design methodology based on distributed cognition and simulation theory can be successfully combined to guide simulator (re)design and strengthen simulator validity.

(P1-79) Regional Medical Command and Control Management of Influenza A (H1N1) Mass-Vaccination in the County of Östergötland, Sweden

Introduction On 11 June 2009, an Influenza A (H1N1) pandemic was declared by the World Health Organization (WHO). The Major Medical Incident Regional Command and Control Protocol in the County Council of Ostergotland, Sweden was activated. After vaccinations were competed, it was decided that the operation should be evaluated in a retrospective study. This study aims to increase knowledge regarding regional management of a pandemic flu. Methods All protocols from regional command meetings were studied together with central data regarding, logistics, vaccination site reports, incident reports, and all written correspondence between involved departments. Information from results of a questionnaire that was distributed to all vaccination site managers were summarized and studied. In addition, an interview was performed with the chief of medical operations. Results Out of the approximately 426,000 inhabitants of the county, a total of 224,780 (53%) were vaccinated during a five and a half month period. The mean pace was 1,246 vaccinated per day (range 0–9643). Regional command had 41 recorded meetings resulting in a collected number of about 740 working hours. Three hundred sixty-six employees were involved in the vaccination, working 38,741 hours. Twenty-eight safety and 52 security incidents were reported. Uncertainty about vaccine delivery and keeping the publics interest were reported to be of concern for the management. Discussion Even with the large scale of the operation, there were only a few security and safety issues. Although the goal of vaccinating 75–80% of the inhabitants was not reached, it could be assumed that the pandemic was dampened. Given the publics high initial interest, it could be considered that vaccination should not start until a large number of doses have been delivered. Conclusion The medical incident command structure and protocol successfully can be adapted to a mass vaccination event. Information from the Ostergotland County Council operation yielded significant experience for future mass vaccinations.