Charlene R. Weir

Direct Text Entry in Electronic Progress Notes An Evaluation of Input Errors

OBJECTIVES It is not uncommon that the introduction of a new technology fixes old problems while introducing new ones. The Veterans Administration recently implemented a comprehensive electronic medical record system (CPRS) to support provider order entry. Progress notes are entered directly by clinicians, primarily through keyboard input. Due to concerns that there may be significant, invisible disruptions to information flow, this study was conducted to formally examine the incidence and characteristics of input errors in the electronic patient record. METHODS Sixty patient charts were randomly selected from all 2,301 inpatient admissions during a 5-month period. A panel of clinicians with informatics backgrounds developed the review criteria. After establishing inter-rater reliability, two raters independently reviewed 1,891 notes for copying, copying errors, inconsistent text, inappropriate object insertion and signature issues. RESULTS Overall, 60% of patients reviewed had one or more input-related errors averaging 7.8 errors per patient. About 20% of notes showed evidence of copying, with an average of 1.01 error per copied note. Copying another clinicians note and making changes had the highest risk of error. Templating resulted in large amounts of blank spaces. Overall, MDs make more errors than other clinicians even after controlling for the number of notes. CONCLUSIONS Moving towards a more progressive model for the electronic medical record, where actions are recorded only once, history and physical information is encoded for use later, and note generation is organized around problems, would greatly minimize the potential for error.

A Cognitive Task Analysis of Information Management Strategies in a Computerized Provider Order Entry Environment

OBJECTIVE Computerized Provider Order Entry (CPOE) with electronic documentation, and computerized decision support dramatically changes the information environment of the practicing clinician. Prior work patterns based on paper, verbal exchange, and manual methods are replaced with automated, computerized, and potentially less flexible systems. The objective of this study is to explore the information management strategies that clinicians use in the process of adapting to a CPOE system using cognitive task analysis techniques. DESIGN Observation and semi-structured interviews were conducted with 88 primary-care clinicians at 10 Veterans Administration Medical Centers. MEASUREMENTS Interviews were taped, transcribed, and extensively analyzed to identify key information management goals, strategies, and tasks. Tasks were aggregated into groups, common components across tasks were clarified, and underlying goals and strategies identified. RESULTS Nearly half of the identified tasks were not fully supported by the available technology. Six core components of tasks were identified. Four meta-cognitive information management goals emerged: 1) Relevance Screening; 2) Ensuring Accuracy; 3) Minimizing memory load; and 4) Negotiating Responsibility. Strategies used to support these goals are presented. CONCLUSION Users develop a wide array of information management strategies that allow them to successfully adapt to new technology. Supporting the ability of users to develop adaptive strategies to support meta-cognitive goals is a key component of a successful system.

Developing a Taxonomy for Research in Adverse Drug Events: Potholes and Signposts

Computerized decision support and order entry shows great promise for reducing adverse drug events (ADEs). The evaluation of these solutions depends on a framework of definitions and classifications that is clear and practical. Unfortunately the literature does not always provide a clear path to defining and classifying adverse drug events. While not a systematic review, this paper uses examples from the literature to illustrate problems that investigators will confront as they develop a conceptual framework for their research. It also proposes a targeted taxonomy that can facilitate a clear and consistent approach to the research of ADEs and aid in the comparison to results of past and future studies. The taxonomy addresses the definition of ADE, types, seriousness, error, and causality.

Report of the AMIA EHR 2020 task force on the status and future direction of EHRs

Over the last 5 years, stimulated by the changing healthcare environment and the Health Information Technology for Economic and Clinical Health (HITECH) Meaningful Use (MU) Electronic Health Record (EHR) Incentive program, EHR adoption has increased remarkably, and there is early evidence that such adoption has resulted in healthcare safety and quality benefits.1,2 However, with this broad adoption, many clinicians are voicing concerns that EHR use has had unintended clinical consequences, including reduced time for patient-clinician interaction,3 new and burdensome data entry tasks being transferred to front-line clinicians,4,5 and lengthened clinician workdays.6–8 Additionally, interoperability between different EHR systems has languished despite large efforts towards that goal.9,10 These challenges are contributing to physicians’ decreased satisfaction with their work lives.11–13 In professional journals,14 press reports,15–17 on wards, and in clinics, we have heard of the difficulties that the transition from paper records to EHRs has created.18 As a result, clinicians are seeking help to get through their work days, which often extend into evenings devoted to writing notes. Examples of comments we have received from clinicians and patients include: “Computers always make things faster and cheaper. Not this time,” and “My doctor pays more attention to the computer than to me.” Ultimately the healthcare systems goal is to create a robust, integrated, and interoperable healthcare system that includes patients, physician practices, public health, population management, and support for clinical and basic sciences research. This ecosystem has been referred to as the “learning health system.”19 EHRs are an important part of the learning health system, along with many other clinical systems, but future ways in which information is transformed into knowledge will likely require all parts of the system working together. Potentially every patient encounter could present an …

Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure

OBJECTIVES Left ventricular ejection fraction (EF) is a key component of heart failure quality measures used within the Department of Veteran Affairs (VA). Our goals were to build a natural language processing system to extract the EF from free-text echocardiogram reports to automate measurement reporting and to validate the accuracy of the system using a comparison reference standard developed through human review. This project was a Translational Use Case Project within the VA Consortium for Healthcare Informatics. MATERIALS AND METHODS We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements. RESULTS System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%). DISCUSSION An EF value of <40% can be accurately identified in VA echocardiogram reports. CONCLUSIONS An automated information extraction system can be used to accurately extract EF for quality measurement.

The state of the evidence for computerized provider order entry: A systematic review and analysis of the quality of the literature

OBJECTIVE This paper presents the results of a systematic literature review and a formal analysis of the scientific quality of empirical research on computerized provider order-entry (CPOE) applications. DESIGN Formal, systematic review techniques were used to search the literature, determine study relevance, and evaluate study quality. MEASUREMENT A search of multiple databases from 1976 through mid-2007 yielded a final set of 46 articles. Relevance criteria included: (1) a direct comparison of a CPOE system with a non-CPOE system; (2) implementation in a clinical setting; and (3) clinically relevant outcomes. RESULTS Study quality varied widely. Three major areas were identified for improvement in future studies: (1) internal validity, especially in terms of study designs, blinding, and instrumentation bias; (2) construct validity of the phenomenon of CPOE itself; and (3) measurement strategies, including reliability and validity assessments. CONCLUSIONS The evidence for the impact of CPOE needs to be improved to support scientific generalizability. Several common confounds are found in this literature. Future researchers will want to address them to improve the strength of the inference between CPOE and clinical outcomes. Discussion focuses on methods to improve future CPOE research.

Text summarization in the biomedical domain

OBJECTIVE The amount of information for clinicians and clinical researchers is growing exponentially. Text summarization reduces information as an attempt to enable users to find and understand relevant source texts more quickly and effortlessly. In recent years, substantial research has been conducted to develop and evaluate various summarization techniques in the biomedical domain. The goal of this study was to systematically review recent published research on summarization of textual documents in the biomedical domain. MATERIALS AND METHODS MEDLINE (2000 to October 2013), IEEE Digital Library, and the ACM digital library were searched. Investigators independently screened and abstracted studies that examined text summarization techniques in the biomedical domain. Information is derived from selected articles on five dimensions: input, purpose, output, method and evaluation. RESULTS Of 10,786 studies retrieved, 34 (0.3%) met the inclusion criteria. Natural language processing (17; 50%) and a hybrid technique comprising of statistical, Natural language processing and machine learning (15; 44%) were the most common summarization approaches. Most studies (28; 82%) conducted an intrinsic evaluation. DISCUSSION This is the first systematic review of text summarization in the biomedical domain. The study identified research gaps and provides recommendations for guiding future research on biomedical text summarization. CONCLUSION Recent research has focused on a hybrid technique comprising statistical, language processing and machine learning techniques. Further research is needed on the application and evaluation of text summarization in real research or patient care settings.

A lower extremity physical function computerized adaptive testing instrument for orthopaedic patients.

Background: The aim of this study was to develop a lower extremity (LE) physical function computerized adaptive testing (CAT) instrument based on Patient Reported Outcomes Measurement Information System (PROMIS) physical function items. Methods The PROMIS physical function item bank was administered to adult outpatient orthopaedic patients. Three hundred eighty-two patients presenting with LE disorders were analyzed using item response theory modeling. A LE physical function item bank was developed by distilling relevant and psychometrically sound items from the full PROMIS physical function bank. Real data CAT simulations were conducted to examine specifications for a live CAT. Results The LE physical function item bank was sufficiently unidimensional and free of item bias. It demonstrated high reliability along with content and construct validity. The flexible length LE CAT was highly correlated with the full LE instrument and showed uniformly high precision across the entire measurement continuum. The average CAT length was 6 to 7 items when standard error of measurement was 0.3 or less. Conclusion This LE physical function CAT is a valid, reliable and feasible physical function assessment tool for orthopaedic patients with LE problems that has the potential to reduce patient burden as well as administrative costs associated with data collection.

An exploration of the impact of computerized patient documentation on clinical collaboration

PURPOSE The purpose of this study was to explore the experience of experienced users of computerized patient documentation for the purpose of collaboration and coordination. A secondary analysis of qualitative data using Clarks theoretical framework of communication was conducted with the goal of bringing research findings into design. METHODS Physicians, nurses and administrative staff volunteered to participate in focus groups at 4 VA sites. Each focus group lasted 1.5h and targeted experience and issues with using computerized documentation. All focus groups were audio-taped and transcribed and submitted to extensive qualitative analysis using ATLAS, iterative identification of concepts and categories. The communication category was targeted for secondary theoretical analysis in order to deepen understanding of the findings. Clarks theory of communication, joint action and common ground heuristics was used to analyze concepts. RESULTS Key concepts included: (1) CPD has changed the way that narrative documentation is used in clinical settings to include more communication functions, strategies to establish joint action in both negative and positive ways; (2) functionality added to CPD to increase the efficiency of input may have increased the efficiency of CPD to support shared situation models, joint and action and the establishment of common ground; (3) new usage of CPD may increase tensions between clinical and administrative roles as the role of narrative is re-defined. CONCLUSIONS This study demonstrates how socio-technical systems co-evolve to support essential human function of coordination and collaboration. Users adapted the system in unique and useful ways that provide insight to future development.

Reviewing the impact of computerized provider order entry on clinical outcomes: The quality of systematic reviews

PURPOSE Computerized provider order entry (CPOE) is central to current efforts at improving clinical care. Understanding the quality of the evidence for CPOE is important to the practical decision of implementation, patient safety and future design efforts. This paper presents the results of a systematic analysis of the quality of systematic reviews of empirical CPOE research. METHODS The systematic search process included PubMed, CINAHL, Scopus, Cochrane, INSPEC, and PsychInfo databases from the years 1987-mid 2010 in English only. All reviews with a focus on CPOE, electronic ordering, Electronic Health Record, or Health Information Technology were included. Studies were excluded if they did not mention a systematic review in the title or text, report a formal search process, report results of the search, or specifically include a separate section on CPOE in the results. Quality was assessed using systematic criteria developed by Oxman and Guyatt, QUOROM, and PRISMA. All three authors conducted the reviews independently. Disagreements were resolved through discussion. Descriptive data was extracted. RESULTS The search process yielded 185 initial unique references with 13 final reviews meeting the inclusion criteria. The rating of overall quality in the Oxman and Guyatt scale averaged 4.9 out of a possible 7 and the average mean of the sum of the other questions was 5.69. The overall QUOROM/PRISMA ratings averaged 63% completion and ranging from 45% to 81%. CONCLUSIONS The quality of these reviews were moderate. Only one study conducted a full quantitative synthesis, and overall heterogeneity was reported as very high in the 3 studies that measured it. Recommendations emphasize clarifying the phenomenon of CPOE by avoiding reporting conclusions across sub-group analyses, increasing emphasis on the development of theoretical models, including more quantitative assessments, and increasing breadth of outcomes.