Raj M. Ratwani

Exploring methods for identifying related patient safety events using structured and unstructured data

Most healthcare systems have implemented patient safety event reporting systems to identify safety hazards. Searching the safety event data to find related patient safety reports and identify trends is challenging given the complexity and quantity of these reports. Structured data elements selected by the event reporter may be inaccurate and the free-text narrative descriptions are difficult to analyze. In this paper we present and explore methods for utilizing both the unstructured free-text and structured data elements in safety event reports to identify and rank similar events. We evaluate the results of three different free-text search methods, including a unique topic modeling adaptation, and structured element weights, using a patient fall use case. The various search techniques and weight combinations tended to prioritize different aspects of the event reports leading to different search and ranking results. These search and prioritization methods have the potential to greatly improve patient safety officers, and other healthcare workers, understanding of which safety event reports are related.

Identifying Interruption Clusters in the Emergency Department

Interruptions can adversely affect human performance, particularly in fast-paced and high-risk environments. Much of the research on interruptions has been laboratory based and the extension of these methods to real-world settings has been challenging and limited. This paper discusses the development and usage of a new tool, TaskTracker, to increase understanding of interruptions in the emergency department. With the data collected from this tool we identified several temporal groupings of interruptions, what we define as interruption clusters. We found significantly more clusters during self-initiated computer tasks. In this setting, we also observed the tendencies of assistants, technicians, students, and nurses to interrupt attending physicians in clusters. A deeper understanding of who engages in interruption clusters and why may provide insights for future systemic strategies that could facilitate better communication patterns.

Emergency Physician Use of Cognitive Strategies to Manage Interruptions

Study objective: The purpose of this study is to examine whether emergency physicians use strategies to manage interruptions during clinical work. Interruption management strategies include immediately engaging the interruption by discontinuing the current task and starting the interruption, continuing the current task while engaging the interruption, rejecting the interruption, or delaying the interruption. Methods: An observational time and motion study was conducted in 3 different urban, academic emergency departments with 18 attending emergency physicians. Each physician was observed for 2 hours, and the number of interruptions, source of interruptions, type of task being interrupted, and use of interruption management strategies were documented. Results: Participants were interrupted on average of 12.5 times per hour. The majority of interruptions were in person from other staff, including nurses, residents, and other attending physicians. When participants were interrupted, they were often working on their computer. Participants almost always immediately engaged the interruption task (75.4% of the time), followed by multitasking, in which the primary task was continued while the interrupting task was performed (22.2%). Physicians rejected or delayed interruptions less than 2% of the time. Conclusion: Our results suggest there is an opportunity to introduce emergency physicians to the use of interruption management strategies as a method of handling the frequent interruptions they are exposed to. Use of these strategies when high‐risk primary tasks are performed may reduce the disruptiveness of some interruptions and improve patient safety.

A framework for evaluating electronic health record vendor user-centered design and usability testing processes.

Objective: Currently, there are few resources for electronic health record (EHR) purchasers and end users to understand the usability processes employed by EHR vendors during product design and development. We developed a framework, based on human factors literature and industry standards, to systematically evaluate the user-centered design processes and usability testing methods used by EHR vendors. Materials and Methods: We reviewed current usability certification requirements and the human factors literature to develop a 15-point framework for evaluating EHR products. The framework is based on 3 dimensions: user-centered design process, summative testing methodology, and summative testing results. Two vendor usability reports were retrieved from the Office of the National Coordinator’s Certified Health IT Product List and were evaluated using the framework. Results: One vendor scored low on the framework (5 pts) while the other vendor scored high on the framework (15 pts). The 2 scored vendor reports demonstrate the framework’s ability to discriminate between the variabilities in vendor processes and to determine which vendors are meeting best practices. Discussion: The framework provides a method to more easily comprehend EHR vendors’ usability processes and serves to highlight where EHR vendors may be falling short in terms of best practices. The framework provides a greater level of transparency for both purchasers and end users of EHRs. Conclusion: The framework highlights the need for clearer certification requirements and suggests that the authorized certification bodies that examine vendor usability reports may need to be provided with clearer guidance.

Barriers to comparing the usability of electronic health records.

Despite the widespread adoption of electronic health records (EHRs), usability of many EHRs continues to be suboptimal, with some vendors failing to meet usability standards, resulting in clinician frustration and patient safety hazards. In an effort to increase EHR vendor competition on usability, recommendations have been made and legislation drafted to develop comparison tools that would allow purchasers to better understand the usability of EHR products prior to purchase. Usability comparison can be based on EHR vendor design and development processes, vendor usability testing as part of the Office of the National Coordinator for Health Information Technology certification program, and usability of implemented products. Barriers exist within the current certified health technology program that prevent effective comparison of usability during each of these stages. We describe the importance of providing purchasers with improved information about EHR usability, barriers to making usability comparisons, and solutions to overcome these barriers.

An Analysis of Patient Safety Incident Reports Associated with Electronic Health Record Interoperability

Background: With the widespread use of electronic health records (EHRs) for many clinical tasks, interoperability with other health information technology (health IT) is critical for the effective delivery of care. While it is generally recognized that poor interoperability negatively impacts patient care, little is known about the specific patient safety implications. Understanding the patient safety implications will help prioritize interoperability efforts around architectures and standards. Objectives: Our objectives were to (1) identify patient safety incident reports that reflect EHR interoperability challenges with other health IT, and (2) perform a detailed analysis of these reports to understand the health IT systems involved, the clinical care processes impacted, whether the incident occurred within or between provider organizations, and the reported severity of the patient safety events. Methods: From a database of 1.735 million patient safety event (PSE) reports spanning multiple provider organizations, 2625 reports that were indicated as being health IT related by the event reporter were reviewed to identify EHR interoperability related reports. Through a rigorous coding process 209 EHR interoperability related events were identified and coded. Results: The majority of EHR interoperability PSE reports involved interfacing with pharmacy systems (i.e. medication related), followed by laboratory, and radiology. Most of the interoperability challenges in these clinical areas were associated with the EHR receiving information from other health IT systems as opposed to the EHR sending information to other systems. The majority of EHR interoperability challenges were within a provider organization and while many of the safety events reached the patient, only a few resulted in patient harm. Conclusions: Interoperability efforts should prioritize systems in pharmacy, laboratory, and radiology. Providers should recognize the need to improve EHRs interfacing with other health IT systems within their own organization.

Making Sense of Mobile Eye-Tracking Data in the Real-World A Human-in-the-Loop Analysis Approach

Stationary eye-tracking technology has been used extensively in human-computer interaction to both understand how humans interact with computers and as an interaction mechanism. Mobile eye-tracking technology is becoming more prevalent, yet the analysis and annotation of mobile eye-tracking data remains challenging. We present a novel human-in-the-loop approach for mobile eye-tracking data analysis that dramatically reduces resource requirements. This method incorporates human insight in a semi-automatic decision making process, leveraging both computational power and human decision making abilities. We demonstrate the accuracy of this approach with eye movement data from two real-world use cases. Average accuracy across the two environments is 82.3%. Our approach holds tremendous promise and has the potential to open the door to more robust eye movement studies in the real-world.

Electronic Health Records and Interruptions: The Need for New Interruption Management Strategies:

Background: Interruptions have been widely studied in the health care domain, yet the nature of work in healthcare is changing with the pervasive use of electronic health records (EHRs). It is unclear how interruptions are changing with the introduction of new technology. Method: This study examines the nature of interruptions experienced by emergency medicine physicians before and after the transition from a home grown EHR to a commercially available system with electronic nursing documentation and computerized physician order entry (CPOE) capabilities. A minute-by-minute paper-based observation template was utilized to record the type of tasks interrupted (patient room, computer, verbal communication or paper charts/labs) as well as who initiated the interruption. Results: There was a significant difference in the type of tasks performed between the preimplementation and post-implementation (go-live) phase. Physicians spent less time on paper-related tasks (p < 0.01) and more time on computer-related tasks (p < 0.01) in the go-live phase. With the shift in task composition, physicians experienced a significantly higher rate of interruptions of computer tasks (p = 0.05), and a significantly lower rate of interruptions to of paper-based tasks (p < 0.01). There were no significant differences in the source of the interruption between the two phases. Discussion: Given the shifts in the types of tasks interrupted after the implementation of a commercially available system with CPOE, it is important to develop new interruption coping strategies. These strategies might include interface design principles to facilitate interruption recovery, less disruptive tactics for the person initiating the interruption, and providing the physicians with training on test systems with realistic data prior to using the new system to care for patients. Conclusion: It was demonstrated that EHRs change the nature of interruptions experienced during the immediate go-live period. Going forward, exploring methods for mitigating harm associated with this change will be critical in preserving patient safety.

Going Live Implementing an Electronic Health Record System in the Emergency Department

This study evaluated the immediate effects of implementing an electronic health record (EHR) system on physician workflow in the emergency department. Two sets of observations were conducted in one emergency department. The first set of observations, the baseline period, was completed in the 22 days prior to the implementation of a new EHR. The second set of observations, the go-live period, was completed during the seven-day period immediately after the EHR go-live date. A comparison across four task-based categories revealed that during the go-live period there was a decrease in the proportion of time physicians spent in patient rooms and engaged with paper-based tasks, and an increase in the proportion of time physicians spent at computer stations and working with other people. In addition, physicians engaged in more information seeking behaviors during the go-live period than during the baseline period. The impact of these shifts in tasks and behaviors is discussed with a focus on the importance of fully understanding the EHR go-live process.

Implications of electronic health record downtime: an analysis of patient safety event reports

Objective We sought to understand the types of clinical processes, such as image and medication ordering, that are disrupted during electronic health record (EHR) downtime periods by analyzing the narratives of patient safety event report data. Materials and Methods From a database of 80 381 event reports, 76 reports were identified as explicitly describing a safety event associated with an EHR downtime period. These reports were analyzed and categorized based on a developed code book to identify the clinical processes that were impacted by downtime. We also examined whether downtime procedures were in place and followed. Results The reports were coded into categories related to their reported clinical process: Laboratory, Medication, Imaging, Registration, Patient Handoff, Documentation, History Viewing, Delay of Procedure, and General. A majority of reports (48.7%, n = 37) were associated with lab orders and results, followed by medication ordering and administration (14.5%, n = 11). Incidents commonly involved patient identification and communication of clinical information. A majority of reports (46%, n = 35) indicated that downtime procedures either were not followed or were not in place. Only 27.6% of incidents (n = 21) indicated that downtime procedures were successfully executed. Discussion Patient safety report data offer a lens into EHR downtime-related safety hazards. Important areas of risk during EHR downtime periods were patient identification and communication of clinical information; these should be a focus of downtime procedure planning to reduce safety hazards. Conclusion EHR downtime events pose patient safety hazards, and we highlight critical areas for downtime procedure improvement.