Peter J. Embi

Caveats for the use of operational electronic health record data in comparative effectiveness research.

The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.

Impacts of Computerized Physician Documentation in a Teaching Hospital: Perceptions of Faculty and Resident Physicians

OBJECTIVE Computerized physician documentation (CPD) has been implemented throughout the nations Veterans Affairs Medical Centers (VAMCs) and is likely to increasingly replace handwritten documentation in other institutions. The use of this technology may affect educational and clinical activities, yet little has been reported in this regard. The authors conducted a qualitative study to determine the perceived impacts of CPD among faculty and housestaff in a VAMC. DESIGN A cross-sectional study was conducted using semistructured interviews with faculty (n = 10) and a group interview with residents (n = 10) at a VAMC teaching hospital. MEASUREMENTS Content analysis of field notes and taped transcripts were done by two independent reviewers using a grounded theory approach. Findings were validated using member checking and peer debriefing. RESULTS Four major themes were identified: (1) improved availability of documentation; (2) changes in work processes and communication; (3) alterations in document structure and content; and (4) mistakes, concerns, and decreased confidence in the data. With a few exceptions, subjects felt documentation was more available, with benefits for education and patient care. Other impacts of CPD were largely seen as detrimental to aspects of clinical practice and education, including documentation quality, workflow, professional communication, and patient care. CONCLUSION CPD is perceived to have substantial positive and negative impacts on clinical and educational activities and environments. Care should be taken when designing, implementing, and using such systems to avoid or minimize any harmful impacts. More research is needed to assess the extent of the impacts identified and to determine the best strategies to effectively deal with them.

A review of approaches to identifying patient phenotype cohorts using electronic health records

Objective To summarize literature describing approaches aimed at automatically identifying patients with a common phenotype. Materials and methods We performed a review of studies describing systems or reporting techniques developed for identifying cohorts of patients with specific phenotypes. Every full text article published in (1) Journal of American Medical Informatics Association, (2) Journal of Biomedical Informatics, (3) Proceedings of the Annual American Medical Informatics Association Symposium, and (4) Proceedings of Clinical Research Informatics Conference within the past 3 years was assessed for inclusion in the review. Only articles using automated techniques were included. Results Ninety-seven articles met our inclusion criteria. Forty-six used natural language processing (NLP)-based techniques, 24 described rule-based systems, 41 used statistical analyses, data mining, or machine learning techniques, while 22 described hybrid systems. Nine articles described the architecture of large-scale systems developed for determining cohort eligibility of patients. Discussion We observe that there is a rise in the number of studies associated with cohort identification using electronic medical records. Statistical analyses or machine learning, followed by NLP techniques, are gaining popularity over the years in comparison with rule-based systems. Conclusions There are a variety of approaches for classifying patients into a particular phenotype. Different techniques and data sources are used, and good performance is reported on datasets at respective institutions. However, no system makes comprehensive use of electronic medical records addressing all of their known weaknesses.

Clinical Research Informatics: Challenges, Opportunities and Definition for an Emerging Domain

OBJECTIVES Clinical Research Informatics, an emerging sub-domain of Biomedical Informatics, is currently not well defined. A formal description of CRI including major challenges and opportunities is needed to direct progress in the field. DESIGN Given the early stage of CRI knowledge and activity, we engaged in a series of qualitative studies with key stakeholders and opinion leaders to determine the range of challenges and opportunities facing CRI. These phases employed complimentary methods to triangulate upon our findings. MEASUREMENTS Study phases included: 1) a group interview with key stakeholders, 2) an email follow-up survey with a larger group of self-identified CRI professionals, and 3) validation of our results via electronic peer-debriefing and member-checking with a group of CRI-related opinion leaders. Data were collected, transcribed, and organized for formal, independent content analyses by experienced qualitative investigators, followed by an iterative process to identify emergent categorizations and thematic descriptions of the data. RESULTS We identified a range of challenges and opportunities facing the CRI domain. These included 13 distinct themes spanning academic, practical, and organizational aspects of CRI. These findings also informed the development of a formal definition of CRI and supported further representations that illustrate areas of emphasis critical to advancing the domain. CONCLUSIONS CRI has emerged as a distinct discipline that faces multiple challenges and opportunities. The findings presented summarize those challenges and opportunities and provide a framework that should help inform next steps to advance this important new discipline.

Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study

Objective Inadequate participant recruitment is a major problem facing clinical research. Recent studies have demonstrated that electronic health record (EHR)-based, point-of-care, clinical trial alerts (CTA) can improve participant recruitment to certain clinical research studies. Despite their promise, much remains to be learned about the use of CTAs. Our objective was to study whether repeated exposure to such alerts leads to declining user responsiveness and to characterize its extent if present to better inform future CTA deployments. Methods During a 36-week study period, we systematically documented the response patterns of 178 physician users randomized to receive CTAs for an ongoing clinical trial. Data were collected on: (1) response rates to the CTA; and (2) referral rates per physician, per time unit. Variables of interest were offset by the log of the total number of alerts received by that physician during that time period, in a Poisson regression. Results Response rates demonstrated a significant downward trend across time, with response rates decreasing by 2.7% for each advancing time period, significantly different from zero (flat) (p<0.0001). Even after 36 weeks, response rates remained in the 30%–40% range. Subgroup analyses revealed differences between community-based versus university-based physicians (p=0.0489). Discussion CTA responsiveness declined gradually over prolonged exposure, although it remained reasonably high even after 36 weeks of exposure. There were also notable differences between community-based versus university-based users. Conclusions These findings add to the limited literature on this form of EHR-based alert fatigue and should help inform future tailoring, deployment, and further study of CTAs.

Toward Reuse of Clinical Data for Research and Quality Improvement: The End of the Beginning?

Todays health information technology (HIT) landscape provides an unprecedented convergence of comprehensive electronic health records, robust computational processing power, data-sharing capabilities, and emerging financial incentives that favor the widespread adoption and meaningful use of such systems. Even as the primary value proposition for HIT adoption by clinical practices remains a matter of debate (1), 2 articles in this issue (2, 3) help to illustrate the potential for secondary benefits by discussing the value of distributed health data networks for improving research and health care quality. The multiple goals of population health, pharmaceutical surveillance, comparative effectiveness research, and other major initiatives being advanced to address the needs of our health care and research enterprises have created a growing need for access to high-quality, patient-level health information. One approach to enabling such access is the creation of centralized repositories to which data can be transferred and then readily accessed to answer questions, an approach that certainly has its merits. Alternatively, the distributed health data networks described in this issue can be designed with appropriate methods, policies, and systems to enable access and use of data housed in their original, disparate locations. As Maro and colleagues point out (2), a distributed approach has potential advantagesgiven todays regulatory, economic, and cultural realitiesand the technological capabilities to enable such an approach exist today. Why, then, do we not see more widespread use of such systems? The history of HIT successes and failures indicates that the technological challenges involved often pale in comparison with the many socio-organizational issues that must be understood and addressed to enable HIT advances (4). Any visitor to a local technology superstore knows how easy and inexpensive it is to collect and store vast amounts of datathe equivalent of thousands of patient recordsin a device that fits as easily in a shirt pocket today as a 1.44-megabyte floppy disk did in 1990. However, the ease of information dissemination this capability enables appropriately raises personal privacy and intellectual property concerns. Adequate funding is another important factor to the success of HIT initiatives; however, funding alone does not guarantee success. One need only look at the many recent examples of failed regional health information organizationsand their successful counterpartsto appreciate the importance of effective governance structures, regulatory policies, and properly aligned organizational incentives in establishing and sustaining an effective distributed health data network (5). In addition to these prerequisites, developers and those who attempt to leverage HIT resources have recognized even more fundamental issues inherent to reusing clinical data. Although systems to enable clinical research from large health information collections have been around for some time (69), they have often been criticized because the quality and comprehensiveness of the clinical data were not up to research standards or the analytical methods used to overcome these limitations were inadequate to overcome systematic biases inherent to data collected primarily for clinical care (1012). Has data quality improved since the time of these earlier systems? In recent years, increased data capture in electronic systems, improvements in the speed and standardization of data transfers between systems, and the ability to leverage data from multiple clinical sources (such as objective test results or therapeutic information) have reduced the oft-criticized reliance on administrative and billing data sources. However, no purely technical solution can overcome the capture of inaccurate information by the user of a clinical information system. As such, nontechnical innovations that help improve the accuracy of recorded information and incentivize consistently accurate data collection are critical to the success of research initiatives that rely on the presence of such data. Is data capture more comprehensive now than in the earlier systems? Comprehensive data capture all relevant exposures and outcomes, features that are essential to meaningful comparative effectiveness assessments. Within a single health system, improved technology has facilitated the automated capture of more comprehensive data in electronic form, which minimizes the need for laborious manual data extraction from paper charts. However, many patients receive care across several health systems and alternative health care settings, such that a single, comprehensive, longitudinal record rarely exists for any given patient. Unfortunately, the goal of widespread information system interoperabilityto enable integrated health information access that spans different health systems and vendor productsremains elusive. Achieving interoperability and ensuring adherence to common standards will be critical to the success of secondary-use initiatives (13). Without it, distributed data access could have limited valueor lead to misinterpretation of the prevalence of and relationships between exposures and outcomes. Finally, have the methods for analyzing routinely collected clinical data improved? The process of clinical care introduces treatment bias, in which the statistical association between therapy and outcome is confounded by measured and unmeasured factors that influence both the choice of treatment and the likelihood of the outcome. Instrumental variable and propensity score analyses, which have been applied for years in the social sciences, and new methods, such as prior-event rate ratios (14), are increasingly being applied in the medical domain to overcome treatment biases, with variable degrees of success and acceptance. Understanding the clinical circumstances and types of research questions for which these methods may yield valid resultsperhaps even consistent with those of traditional clinical studiesrequires simulations, sensitivity analyses, and validation against the findings of randomized trials. The promises of our current HIT environment are clearly great and growing. The efforts of biomedical informaticians, health services researchers, biostatisticians, and others have significantly advanced our knowledge of how to collect, organize, retrieve, analyze, and apply health data to improve individual patient care, as well as for such additional purposes as population health and biomedical research. However, significant collaborative effort by many of the stakeholders involved, including health care institutions and clinicians, HIT vendors, researchers, informaticians, regulators and policymakers, payers, and patients, is required to realize the full promise of these resources. If history is any indication, fostering and nurturing this collaboration will be challenging and take some time. Nevertheless, as the 2 articles in this issue help illustrate, current technology and existing models of success have put us in a better position today than we have been in before to realize the promises of HIT to advance research and create a safer and more efficient health care enterprise. To quote Sir Winston Churchill, This is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning.

Clinical decision support systems in the pediatric intensive care unit

Objective: To review the use of clinical decision support systems (CDSS) available in the pediatric intensive care unit (PICU). Data Sources: Relevant English language publications indexed in Medline, as well as CDSS-related white papers and texts. Study Selection and Data Extraction: Studies related to CDSS were considered. Data Synthesis: CDSS are operationally defined as computer software programs that aid healthcare providers in their clinical decision making. Once used solely for diagnostic support, many CDSS now have the ability to transform clinical practice through interactive assistance with therapeutic best practices. The recent emphasis on improving quality and patient safety through the incorporation of electronic health records as supported by Leapfrog and other agencies has encouraged advancements in the use of CDSS tools that leverage the capabilities of stand-alone electronic health records. CDSS are of particular interest in the PICU where rapid decision-making benefits from tools that can improve patient safety. CDSS have been described in the PICU with varying effects on healthcare outcomes. A growing consensus indicates that the success of such interventions depends as much or more on how they are implemented and used in such complex environments as on their programming. In the current review, the types and features of various CDSS tools and the supporting evidence are discussed. Factors such as liability, human factors engineering, alert fatigue, and audit trails are also covered. Conclusion: CDSS have the potential to improve clinical practice in PICU settings. Care should be taken when selecting and implementing such systems to achieve the goal of improved clinical practice while avoiding potential adverse impacts sometimes associated with the implementation of new technologies in complex healthcare settings.

Health data use, stewardship, and governance: ongoing gaps and challenges: a report from AMIA's 2012 Health Policy Meeting

Large amounts of personal health data are being collected and made available through existing and emerging technological media and tools. While use of these data has significant potential to facilitate research, improve quality of care for individuals and populations, and reduce healthcare costs, many policy-related issues must be addressed before their full value can be realized. These include the need for widely agreed-on data stewardship principles and effective approaches to reduce or eliminate data silos and protect patient privacy. AMIAs 2012 Health Policy Meeting brought together healthcare academics, policy makers, and system stakeholders (including representatives of patient groups) to consider these topics and formulate recommendations. A review of a set of Proposed Principles of Health Data Use led to a set of findings and recommendations, including the assertions that the use of health data should be viewed as a public good and that achieving the broad benefits of this use will require understanding and support from patients.

Unintended errors with EHR-based result management: a case series

Test result management is an integral aspect of quality clinical care and a crucial part of the ambulatory medicine workflow. Correct and timely communication of results to a provider is the necessary first step in ambulatory result management and has been identified as a weakness in many paper-based systems. While electronic health records (EHRs) hold promise for improving the reliability of result management, the complexities involved make this a challenging task. Experience with test result management is reported, four new categories of result management errors identified are outlined, and solutions developed during a 2-year deployment of a commercial EHR are described. Recommendations for improving test result management with EHRs are then given.

An exploration of the impact of computerized patient documentation on clinical collaboration

PURPOSE The purpose of this study was to explore the experience of experienced users of computerized patient documentation for the purpose of collaboration and coordination. A secondary analysis of qualitative data using Clarks theoretical framework of communication was conducted with the goal of bringing research findings into design. METHODS Physicians, nurses and administrative staff volunteered to participate in focus groups at 4 VA sites. Each focus group lasted 1.5h and targeted experience and issues with using computerized documentation. All focus groups were audio-taped and transcribed and submitted to extensive qualitative analysis using ATLAS, iterative identification of concepts and categories. The communication category was targeted for secondary theoretical analysis in order to deepen understanding of the findings. Clarks theory of communication, joint action and common ground heuristics was used to analyze concepts. RESULTS Key concepts included: (1) CPD has changed the way that narrative documentation is used in clinical settings to include more communication functions, strategies to establish joint action in both negative and positive ways; (2) functionality added to CPD to increase the efficiency of input may have increased the efficiency of CPD to support shared situation models, joint and action and the establishment of common ground; (3) new usage of CPD may increase tensions between clinical and administrative roles as the role of narrative is re-defined. CONCLUSIONS This study demonstrates how socio-technical systems co-evolve to support essential human function of coordination and collaboration. Users adapted the system in unique and useful ways that provide insight to future development.