Jeremy Daniels

Time to listen: a review of methods to solicit patient reports of adverse events

Background Patients have been shown to report accurate observations of medical errors and adverse events. Various methods of introducing patient reporting into patient safety systems have been published with little consensus among researchers on the most effective method. Terminology for use in patient safety reporting has yet to be standardised. Methods Two databases, PubMed and MEDLINE, were searched for literature on patient reporting of medical errors and adverse events. Comparisons were performed to identify the optimal method for eliciting patient initiated events. Results Seventeen journal publications were reviewed by patient population, type of healthcare setting, contact method, reporting method, duration, terminology and reported response rate. Conclusion Few patient reporting studies have been published, and those identified in this review covered a wide range of methods in diverse settings. Definitive comparisons and conclusions are not possible. Patient reporting has been shown to be reliable. Higher incident rates were observed when open-ended questions were used and when respondents were asked about personal experiences in hospital and primary care. Future patient reporting systems will need a balance of closed-ended questions for cause analysis and classification, and open-ended narratives to allow for patients limited understanding of terminology. Establishing the method of reporting that is most efficient in collecting reliable reports and standardising terminology for patient use should be the focus of future research.

A framework for evaluating usability of clinical monitoring technology

Technology design is a complex task, and acceptability is enhanced when usability is central to its design. Evaluating usability is a challenge for purchasers and developers of technology. We have developed a framework for testing the usability of clinical monitoring technology through literature review and experience designing clinical monitors. The framework can help designers meet key international usability norms. The framework includes these direct testing methods: thinking aloud, question asking, co-discovery, performance and psychophysiological measurement. Indirect testing methods include: questionnaires and interviews, observation and ethnographic studies, and self-reporting logs. Inspection, a third usability testing method, is also included. The use of these methods is described and practical examples of how they would be used in the development of an innovative monitor are given throughout. This framework is built on a range of methods to ensure harmony between users and new clinical monitoring technology, and have been selected to be practical to use.

Identification by families of pediatric adverse events and near misses overlooked by health care providers

Background: Identifying adverse events and near misses is essential to improving safety in the health care system. Patients are capable of reliably identifying and reporting adverse events. The effect of a patient safety reporting system used by families of pediatric inpatients on reporting of adverse events by health care providers has not previously been investigated. Methods: Between Nov. 1, 2008, and Nov. 30, 2009, families of children discharged from a single ward of British Columbia’s Children’s Hospital were asked to respond to a questionnaire about adverse events and near misses during the hospital stay. Rates of reporting by health care providers for this period were compared with rates for the previous year. Family reports for specific incidents were matched with reports by health care providers to determine overlap. Results: A total of 544 familes responded to the questionnaire. The estimated absolute increase in reports by health care providers per 100 admissions was 0.5% (95% confidence interval −1.8% to 2.7%). A total of 321 events were identified in 201 of the 544 family reports. Of these, 153 (48%) were determined to represent legitimate patient safety concerns. Only 8 (2.5%) of the adverse events reported by families were also reported by health care providers. Interpretation: The introduction of a family-based system for reporting adverse events involving pediatric inpatients, administered at the time of discharge, did not change rates of reporting of adverse events and near misses by health care providers. Most reports submitted by families were not duplicated in the reporting system for health care providers, which suggests that families and staff members view safety-related events differently. However, almost half of the family reports represented legitimate patient safety concerns. Families appeared capable of providing valuable information for improving the safety of pediatric inpatients.

A Novel Vibrotactile Display to Improve the Performance of Anesthesiologists in a Simulated Critical Incident

BACKGROUND: Current methods of information transfer in the operating room between monitor and anesthesiologist rely on visual and auditory modalities. These modalities can easily become overloaded in a high cognitive workload situation, such as in a critical incident. The use of vibrotactile communication has been shown to improve information transfer in other high cognitive workload environments such as aviation. We designed a novel waist-mounted vibrotactile display to be worn by the anesthesiologist to test if a vibrotactile display could improve the clinical response time to begin treating a simulated case of anaphylaxis when compared with a group using traditional information displays. In addition, we evaluated differences in situational awareness (SA) between the two groups. METHODS: Twenty-four volunteer anesthesiologists were randomized to diagnose and treat a simulated case of anaphylaxis using the vibrotactile display and standard monitoring (vibrotactile display group) or standard monitoring alone (control group). The time taken to administer epinephrine was measured, and objective post hoc analysis of participant SA was performed. RESULTS: Participants in the vibrotactile group took 4.08 min (95% CI = 1.22) to deliver definitive treatment compared with 7.21 min (95% CI = 2.07) for the control group (P < 0.05). Despite the reduced time to treatment, no improvement in SA was measured. CONCLUSION: Our study provides evidence that vibrotactile communication can reduce response time to critical incidents.

A Knowledge Authoring Tool for Clinical Decision Support

Anesthesiologists in the operating room are unable to constantly monitor all data generated by physiological monitors. They are further distracted by clinical and educational tasks. An expert system would ideally provide assistance to the anesthesiologist in this data-rich environment. Clinical monitoring expert systems have not been widely adopted, as traditional methods of knowledge encoding require both expert medical and programming skills, making knowledge acquisition difficult. A software application was developed for use as a knowledge authoring tool for physiological monitoring. This application enables clinicians to create knowledge rules without the need of a knowledge engineer or programmer. These rules are designed to provide clinical diagnosis, explanations and treatment advice for optimal patient care to the clinician in real time. By intelligently combining data from physiological monitors and demographical data sources the expert system can use these rules to assist in monitoring the patient. The knowledge authoring process is simplified by limiting connective relationships between rules. The application is designed to allow open collaboration between communities of clinicians to build a library of rules for clinical use. This design provides clinicians with a system for parameter surveillance and expert advice with a transparent pathway of reasoning. A usability evaluation demonstrated that anesthesiologists can rapidly develop useful rules for use in a predefined clinical scenario.

An evaluation of a novel software tool for detecting changes in physiological monitoring.

BACKGROUND: We have developed a software tool (iAssist) to assist clinicians as they monitor the physiological data that guide their actions during anesthesia. The system tracks the statistical properties of multiple dynamic physiological processes and identifies new trend patterns. We report our initial evaluation of this tool (in pseudo real-time) and compare the detection of trend changes to a post hoc visual review of the full trend. We suggest a combination of criteria by which to evaluate the performance of monitoring devices that aim to enhance trend detection. METHODS: Nineteen children and 28 adults consented to be included in the study, encompassing more than 68 h of anesthesia. In each surgical case, an anesthesiologist reported all perceived clinical changes in monitoring in real-time. A trained observer simultaneously documented the verbally reported changes and every anesthesiologist action. The same cases were subsequently evaluated offline (in pseudo real-time) by a novel software tool (iAssist). Heart rate, end-tidal carbon dioxide, exhaled minute ventilation, and respiratory rate were modeled using a dynamic linear growth model whose noise distribution was estimated by an adaptive Kalman filter based on a recursive expectation-maximization method. Changes were detected by adaptive local Cumulative Sum testing. Changes in the mean arterial noninvasive blood pressures and oxygen saturation were detected using adaptive Cumulative Sum testing on a filtered residual from an exponentially weighted moving averaging filter. In post hoc analysis, each change detected by iAssist was graded independently by two clinicians using a graphical display of the whole case. Missed changes were recorded. RESULTS: The iAssist software tool detected 869 true positive changes (at an average of 12.76/h) with a sensitivity of 0.91 and positive predictive value of 0.87. The post hoc review identified 91 missed changes (at an average of 1.34/h), resulting in an overall ratio of true positive rates to false-negative rates of 9.55. The clinicians in real-time reported 209 changes in trend (at an average of 3.07/h). CONCLUSION: The algorithms perform favorably compared with a visual inspection of the complete trend. Further research is needed to identify when and how to draw the clinician’s attention to these changes.

VISUAL CUEING WITH CONTEXT RELEVANT INFORMATION FOR REDUCING CHANGE BLINDNESS

ObjectivePhysiological monitoring is a requisite for optimal care to ensure that the condition of a patient is maintained within safe levels. Monitoring can be jeopardized by the inability of a clinician to recognize important changes in the visual display of data throughout the duration of the monitoring task. We hypothesized that the addition of a visual cue imparting contextual information to a physiological display would improve the detection ability and response time of a clinician to a change in a patient variable.MethodsContextual information based on trend information was added to a physiological display in the form of a visual cue. Following IRB approval, the resulting enhanced display was evaluated by 22 anesthesiologists in a simulated operating room, through the observation of six simulated scenarios using a standard anesthesia display and the enhanced display. Demographic information, response time, accuracy of detection, and usability data were collected.ResultsThe enhanced display reduced the detection time to a change in the simulated scenarios by 14.4 s (95% CI: −26.4 to −2.38), and reduced the expected number of missed events per scenario by 0.23 (95% CI: −0.439 to −0.0203), based on the repeated measures analysis (Poisson model).ConclusionsThe data collected and analyzed in this study supports the addition of a visual cue to future physiological monitors. The graphic representation and the context relevant information that it transmits appears to aid clinicians. While the results indicate that enhanced visualization of context relevant information can lead to a significant improvement in event recognition and identification, further evaluation in clinical settings is required.

Clinical evaluation of algorithms for context-sensitive physiological monitoring in children

BACKGROUND Subtle changes in monitored physiological signals might be used to guide clinical actions and give early warning of potential adverse events. Automated early warning systems could enhance the clinicians interpretation of data by instantaneously processing new information and presenting it within the context of previous observations. In this study, we tested algorithms for tracking the behaviour of dynamic physiological systems and automatically detecting key events over time. METHODS Algorithms were activated in real-time during anaesthesia to run context-sensitive monitoring of six variables (end-tidal PCO(2), heart rate, exhaled minute ventilation, non-invasive arterial pressure, respiratory rate, and oxygen saturation), alongside standard physiological monitors. The clinical evaluation included real-time feedback on each change point (change in the physiological trend) detected by the algorithms and the completion of a usability questionnaire. RESULTS Fifteen anaesthetists completed the evaluation during paediatric surgical cases. A total of 38 cases were evaluated, with a mean duration of 103 (102) min. The mean number of change points per case was 22.8 (23.4). Sixty-one per cent of all rated change points were considered clinically significant, and <7% were due to artifacts. CONCLUSIONS The algorithms were able to detect a range of clinically significant physiological changes during paediatric anaesthesia, and were considered useful by participating anaesthetists. These findings indicate that automated detection of context-sensitive changes is possible and could be used by early warning systems during physiological monitoring. Further investigations are required to assess how this information can best be communicated to the anaesthetist.

A human factors and survey methodology-based design of a web-based adverse event reporting system for families

PURPOSE Adverse event reporting systems allow healthcare institutions to detect and prevent recurrence of avoidable patient harm. It is known that standard reporting systems, which are initiated by clinicians, detect only a minority of chart-documented adverse events. The objective of the study was to develop a web-based system, the Family Reporting System (FRS), to elicit adverse event reports from families of children admitted to hospital through survey methodology and human factors engineering techniques. MEASUREMENTS Face validity and usability were measured via standardized survey instruments. Utility was measured via the rate, typology, degree of harm, likelihood of recurrence, quality of information, and inter-rater agreement analysis of the reported events. RESULTS The FRS has good face validity, excellent usability, and good clinical utility. CONCLUSION The application of survey and human factors methodologies to the design of an electronic system is an effective means of developing an electronic adverse event reporting system for the use of families of pediatric patients.

Introduction of new monitors into clinical anesthesia

Purpose of review There are an increasing number of monitors being developed to measure physiological parameters during the perioperative period. This review provides an overview of some of these new monitors developed for use in clinical anesthesia and outlines the potential advantages of each device. Pitfalls concerning the introduction of additional monitoring devices and the research gaps for introducing these monitors into clinical practice are discussed. Recent findings Many novel monitoring technologies have been developed, with invasive and noninvasive cardiac output monitoring devices and advanced display technologies being especially prominent. Most of the published literature focuses on monitor efficacy, whereas issues of technology integration and acceptability are given less emphasis. Research on novel display technologies is not integrated with the monitor development, nor is the display of new information considered during the development of new monitors. Summary More research is needed on how to integrate the newly developed monitors into the clinical context to assist information-overloaded anesthesiologists. This is essential to achieve the potential benefit of new monitoring devices.