Lei Hua

Usability Evaluation of a Voluntary Patient Safety Reporting System: Understanding the Difference between Predicted and Observed Time Values by Retrospective Think-Aloud Protocols

The study evaluated the usability of a voluntary patient safety reporting system using two established methods of cognitive task analysis and retrospective think-aloud protocols. Two usability experts and ten end users were employed in two separated experiments, and predicted and observed task execution times were obtained for comparison purpose. According to the results, mental operations contributed to the major effort in reporting. The significant time differences were identified that pointed out the difficulty in human cognition as users interacted with the system. At last, the data collected by retrospective think-aloud technique, e.g. the response consistency on structured questions and the user’s attitudes, revealed the frequent usability problems impeding completion of a quality report.

Developing a user-centered voluntary medical incident reporting system.

Medical errors are one of leading causes of death among adults in the United States. According to the Institute of Medicine, reporting of medical incidents could be a cornerstone to learn from errors and to improve patient safety, if incident data are collected in a properly structured format which is useful for the detection of patterns, discovery of underlying factors, and generation of solutions. Globally, a number of medical incident reporting systems were deployed for collecting observable incident data in care delivery organizations (CDO) over the past several years. However, few researches delved into design of user-centered reporting system for improving completeness and accuracy of medical incident collection, let alone design models created for other institutes to follow. In this paper, we introduce the problems identified in a current using voluntary reporting system and our effort is being made towards complete, accurate and useful user-centered new reporting system through a usability engineering process.

Text prediction on structured data entry in healthcare: A two-group randomized usability study measuring the prediction impact on user performance

BACKGROUND Structured data entry pervades computerized patient safety event reporting systems and serves as a key component in collecting patient-related information in electronic health records. Clinicians would spend more time being with patients and arrive at a high probability of proper diagnosis and treatment, if data entry can be completed efficiently and effectively. Historically it has been proven text prediction holds potential for human performance regarding data entry in a variety of research areas. OBJECTIVE This study aimed at examining a function of text prediction proposed for increasing efficiency and data quality in structured data entry. METHODS We employed a two-group randomized design with fifty-two nurses in this usability study. Each participant was assigned the task of reporting patient falls by answering multiple choice questions either with or without the text prediction function. t-test statistics and linear regression model were applied to analyzing the results of the two groups. RESULTS While both groups of participants exhibited a good capacity of accomplishing the assigned task, the results were an overall 13.0% time reduction and 3.9% increase of response accuracy for the group utilizing the prediction function. CONCLUSION As a primary attempt investigating the effectiveness of text prediction in healthcare, study findings validated the necessity of text prediction to structured date entry, and laid the ground for further research improving the effectiveness of text prediction in clinical settings.

Leveraging user's performance in reporting patient safety events by utilizing text prediction in narrative data entry

BACKGROUND Narrative data entry pervades computerized health information systems and serves as a key component in collecting patient-related information in electronic health records and patient safety event reporting systems. The quality and efficiency of clinical data entry are critical in arriving at an optimal diagnosis and treatment. The application of text prediction holds potential for enhancing human performance of data entry in reporting patient safety events. OBJECTIVE This study examined two functions of text prediction intended for increasing efficiency and data quality of text data entry reporting patient safety events. METHODS The study employed a two-group randomized design with 52 nurses. The nurses were randomly assigned into a treatment group or a control group with a task of reporting five patient fall cases in Chinese using a web-based test system, with or without the prediction functions. T-test, Chi-square and linear regression model were applied to evaluating the outcome differences in free-text data entry between the groups. RESULTS While both groups of participants exhibited a good capacity for accomplishing the assigned task of reporting patient falls, the results from the treatment group showed an overall increase of 70.5% in text generation rate, an increase of 34.1% in reporting comprehensiveness score and a reduction of 14.5% in the non-adherence of the comment fields. The treatment group also showed an increasing text generation rate over time, whereas no such an effect was observed in the control group. CONCLUSION As an attempt investigating the effectiveness of text prediction functions in reporting patient safety events, the study findings proved an effective strategy for assisting reporters in generating complementary free text when reporting a patient safety event. The application of the strategy may be effective in other clinical areas when free text entries are required.

Identifying barriers and benefits of patient safety event reporting toward user-centered design

BackgroundTo learn from errors, electronic patient safety event reporting systems (e-reporting systems) have been widely adopted to collect medical incidents from the frontline practitioners in US hospitals. However, two issues of underreporting and low-quality of reports pervade and thus the system effectiveness remains dubious.MethodsThis study employing semi-structured interviews of health professionals in the Texas Medical Center investigated the perceived benefits and barriers from users who have used e-reporting systems.ResultsAs a result, the perceived benefits include the enhanced convenience in data processing and the assistant functions leading to patient safety enhancement. The perceived barriers to the acceptance and quality use of the system include the lack of instructions, lack of reporter-friendly classifications, lack of time, and lack of feedback. The identified benefits and barriers help design a user-centered e-reporting system where learning and assistant features are discussed during the interviews.ConclusionsAs a response, the learning and assistant features aiming at enhancing benefits and removing barriers of e-reporting systems should be included for facilitating the acceptance and effective use of the systems.

Identifying and addressing effectiveness in a user-centered design of voluntary medical incident reporting system

Two major challenges to current voluntary medical incident reporting system (VMIR) are underreporting and low quality reports, which make it difficult for health providers to effective use and learn from mistakes for patient safety improvement. According to prior researches on a variety of voluntary system design and usage in different domains such as aviation, system usability and utility were considered as significant contributing factors that influence the system acceptance. However, the published studies of dealing with these factors to maximize system acceptance and effectiveness in medical incident reporting domain are few and fragmented. In this study, we elaborated the background and problems in target domain of medical incident reporting firstly, and then synthesized practical and academic instance by unobtrusive data analysis and literature review to propose a design framework to the future applications. Finally, a developing computer-based prototype as a partial solution of proposed design was paraphrased to illustrate our initial progress on user-centered design of VMIR. It is expected to increase reporting efficiency and data quality that will promote a learning culture and help remove shame-blame barriers in collecting patient safety events.

Enhancing patient safety event reporting: A systematic review of system design features

OBJECTIVES Electronic patient safety event reporting (e-reporting) is an effective mechanism to learn from errors and enhance patient safety. Unfortunately, the value of e-reporting system (a software or web server based platform) in patient safety research is greatly overshadowed by low quality reporting. This paper aims at revealing the current status of system features, detecting potential gaps in system design, and accordingly proposing suggestions for future design and implementation of the system. METHODS Three literature databases were searched for publications that contain informative descriptions of e-reporting systems. In addition, both online publicly accessible reporting forms and systems were investigated. RESULTS 48 systems were identified and reviewed. 11 system design features and their frequencies of occurrence (Top 5: widgets (41), anonymity or confidentiality (29), hierarchy (20), validator (17), review notification (15)) were identified and summarized into a system hierarchical model. CONCLUSIONS The model indicated the current e-reporting systems are at an immature stage in their development, and discussed their future development direction toward efficient and effective systems to improve patient safety.Citation: Gong Y, Kang H, Wu X, Hua L. Enhancing Patient Safety Event Reporting. Appl Clin Inform 2017; 8: 893-909 https://doi.org/10.4338/ACI-2016-02-R-0023.