Judith R. Logan

Caveats for the use of operational electronic health record data in comparative effectiveness research.

The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.

Impacts of Computerized Physician Documentation in a Teaching Hospital: Perceptions of Faculty and Resident Physicians

OBJECTIVE Computerized physician documentation (CPD) has been implemented throughout the nations Veterans Affairs Medical Centers (VAMCs) and is likely to increasingly replace handwritten documentation in other institutions. The use of this technology may affect educational and clinical activities, yet little has been reported in this regard. The authors conducted a qualitative study to determine the perceived impacts of CPD among faculty and housestaff in a VAMC. DESIGN A cross-sectional study was conducted using semistructured interviews with faculty (n = 10) and a group interview with residents (n = 10) at a VAMC teaching hospital. MEASUREMENTS Content analysis of field notes and taped transcripts were done by two independent reviewers using a grounded theory approach. Findings were validated using member checking and peer debriefing. RESULTS Four major themes were identified: (1) improved availability of documentation; (2) changes in work processes and communication; (3) alterations in document structure and content; and (4) mistakes, concerns, and decreased confidence in the data. With a few exceptions, subjects felt documentation was more available, with benefits for education and patient care. Other impacts of CPD were largely seen as detrimental to aspects of clinical practice and education, including documentation quality, workflow, professional communication, and patient care. CONCLUSION CPD is perceived to have substantial positive and negative impacts on clinical and educational activities and environments. Care should be taken when designing, implementing, and using such systems to avoid or minimize any harmful impacts. More research is needed to assess the extent of the impacts identified and to determine the best strategies to effectively deal with them.

Race, Ethnicity, and Sex Affect Risk for Polyps >9 mm in Average-Risk Individuals

BACKGROUND & AIMS Colorectal cancer risk differs based on patient demographics. We aimed to measure the prevalence of significant colorectal polyps in average-risk individuals and to determine differences based on age, sex, race, or ethnicity. METHODS In a prospective study, colonoscopy data were collected, using an endoscopic report generator, from 327,785 average-risk adults who underwent colorectal cancer screening at 84 gastrointestinal practice sites from 2000 to 2011. Demographic characteristics included age, sex, race, and ethnicity. The primary outcome was the presence of suspected malignancy or large polyp(s) >9 mm. The benchmark risk for age to initiate screening was based on white men, 50-54 years old. RESULTS Risk of large polyps and tumors increased progressively in men and women with age. Women had lower risks than men in every age group, regardless of race. Blacks had higher risk than whites from ages 50 through 65 years and Hispanics had lower risk than whites from ages 50 through 80 years. The prevalence of large polyps was 6.2% in white men 50-54 years old. The risk was similar among the groups of white women 65-69 years old, black women 55-59 years old, black men 50-54 years old, Hispanic women 70-74 years old, and Hispanic men 55-59 years old. The risk of proximal large polyps increased with age, female sex, and black race. CONCLUSIONS There are differences in the prevalence and location of large polyps and tumors in average-risk individuals based on age, sex, race, and ethnicity. These findings could be used to select ages at which specific groups should begin colorectal cancer screening.

Boston Bowel Preparation Scale scores provide a standardized definition of “adequate” for describing bowel cleanliness

BACKGROUND Establishing a threshold of bowel cleanliness below which colonoscopies should be repeated at accelerated intervals is important, yet there are no standardized definitions for an adequate preparation. OBJECTIVE To determine whether Boston Bowel Preparation Scale (BBPS) scores could serve as a standard definition of adequacy. DESIGN Cross-sectional observational analysis of colonoscopy data from 36 adult GI endoscopy practices and prospective survey showing 4 standardized colonoscopy videos with varying degrees of bowel cleanliness. SETTING The Clinical Outcomes Research Initiative. PATIENTS Average-risk patients attending screening colonoscopy. INTERVENTIONS Colonoscopy. MAIN OUTCOME MEASUREMENTS Recommended follow-up intervals among average-risk, screening colonoscopies without polyps stratified by BBPS scores. RESULTS We evaluated 2516 negative screening colonoscopies performed by 74 endoscopists. If the BBPS score was ≥2 in all 3 segments (N = 2295), follow-up was recommended in 10 years in 90% of cases. Examinations with total BBPS scores of 3 to 5 (N = 167) had variable recommendations. Follow-up within 1 year was recommended for 96% of examinations with total BBPS scores of 0 to 2 (N = 26). Similar results were noted among 167 participants in a video survey with pre-established BBPS scores. LIMITATIONS Retrospective study. CONCLUSION BBPS scores correlate with endoscopist behavior regarding follow-up intervals for colonoscopy. A total BBPS score ≥6 and/or all segment scores ≥2 provides a standardized definition of adequate for 10-year follow-up, whereas total scores ≤2 indicate that a procedure should be repeated within 1 year. Future work should focus on finding consensus for management of examinations with total scores of 3 to 5.

Querying through a user interface

In contrast to a traditional setting where users express queries against the database schema, we assert that the semantics of data can often be understood by viewing the data in the context of the user interface (UI) of the software tool used to enter the data. That is, we believe that users will understand the data in a database by seeing the labels, drop-down menus, tool tips, or other help text that are built into the user interface. Our goal is to allow domain experts with little technical skill to understand and query data. In this paper, we present our GUi As View (Guava) framework and describe how we use forms-based UIs to generate a conceptual model that represents the information in the user interface. We then describe how we generate a query interface from the conceptual model. We characterize the resulting query language using a subset of the relational algebra. Since most application developers want to craft a physical database to meet desired performance needs, we present here a transformation channel that can be configured by instantiating one or more of our transformation operators. The channel, once configured, automatically transforms queries from our query interface into queries that address the underlying physical database and delivers query results that conform to our query interface. In this paper, we define and formalize our database transformation operators. The contributions of this paper are that first, we demonstrate the feasibility of creating a query interface based directly on the user interface and second, we introduce a general purpose database transformation channel that will likely shorten the application development process and increase the quality of the software by automatically generating software artifacts that are often made manually and are prone to errors.

Recommendations for the use of operational electronic health record data in comparative effectiveness research.

There is an increasing amount of clinical data in operational electronic health record (EHR) systems. Such data provide substantial opportunities for their re-use for many purposes, including comparative effectiveness research (CER). In a previous paper, we identified a number of caveats related to the use of such data, noting that they may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, or incompatible with research protocols. In this paper, we provide recommendations for overcoming these caveats with the goal of leveraging such data to benefit CER and other health care activities. These recommendations include adaptation of “best evidence” approaches to use of data; processes to evaluate availability, completeness, quality, and transformability of data; creation of tools to manage data and their attributes; determination of metrics for assessing whether data are “research grade”; development of methods for comparative validation of data; construction of a methodology database for methods involving use of clinical data; standardized reporting methods for data and their attributes; appropriate use of informatics expertise; and a research agenda to determine biases inherent in operational data and to assess informatics approaches to their improvement.

Evaluating the accuracy of electronic pediatric drug dosing rules

OBJECTIVE To determine the accuracy of vendor-supplied dosing eRules for pediatric medication orders. Inaccurate or absent dosing rules can lead to high numbers of false alerts or undetected prescribing errors and may potentially compromise safety in this already vulnerable population. MATERIALS AND METHODS 7 months of medication orders and alerts from a large pediatric hospital were analyzed. 30 medications were selected for study across 5 age ranges and 5 dosing parameters. The resulting 750 dosing rules from a commercial system formed the study corpus and were examined for accuracy against a gold standard created from traditional clinical resources. RESULTS Overall accuracy of the rules in the study corpus was 55.1% when the rules were transformed to fit a priori age ranges. Over a pediatric lifetime, the dosing rules were accurate an average of 57.6% of the days. Dosing rules pertaining to the newborn age range were as accurate as other age ranges on average, but exhibited more variability. Daily frequency dosing parameters showed more accuracy than total daily dose, single dose minimum, or single dose maximum. DISCUSSION The accuracy of a vendor-supplied set of dosing eRules is suboptimal when compared with traditional dosing sources, exposing a gap between dosing rules in commercial products and actual prescribing practices by pediatric care providers. More research on vendor-supplied eRules is warranted in order to understand the effects of these products on safe prescribing in children.

Validity of a web-based educational program to disseminate a standardized bowel preparation rating scale

Goals: Our goal was to assess the validity of a Web-based educational program on the Boston Bowel Preparation Scale (BBPS). Background: Data on Web-based education for improving the practice and quality of colonoscopy are limited. Study: Endoscopists worldwide participated in the BBPS Educational Program. We assessed program completion rates, satisfaction, short-term (0 to 90 d) and long-term (91 to 180 d) uptake of the BBPS, and the validity of the program by measuring the reliability of the BBPS among participants. Results: A total of 207 endoscopists completed the program. Overall, 93% found the content relevant, 89% felt confident in using the BBPS, and 97% thought the quality was good or excellent. Uptake of the BBPS into clinical practice was robust with 91% and 98% of colonoscopy reports containing the BBPS at short-term and long-term follow-up, respectively. The interobserver and test-retest reliability of BBPS segment and total scores were both substantial. Conclusions: A BBPS Web-based educational program facilitates adoption into clinical practice and teaches the BBPS to be used reliably by a diverse group of endoscopists worldwide.

The Use of Databases and Registries to Enhance Colonoscopy Quality

Administrative databases, registries, and clinical databases are designed for different purposes and therefore have different advantages and disadvantages in providing data for enhancing quality. Administrative databases provide the advantages of size, availability, and generalizability, but are subject to constraints inherent in the coding systems used and from data collection methods optimized for billing. Registries are designed for research and quality reporting but require significant investment from participants for secondary data collection and quality control. Electronic health records contain all of the data needed for quality research and measurement, but that data is too often locked in narrative text and unavailable for analysis. National mandates for electronic health record implementation and functionality will likely change this landscape in the near future.

Context-sensitive clinical data integration

Current methods for data integration are as difficult to use as they are powerful. Motivated by our work with clinical data and the people who analyze it, we present two components that allow non-technical users that are domain experts to create and reuse complex data integration processes. The GUAVA (GUI As View Apparatus) component enables data analysts to make informed data integration decisions based on detailed accounts of the user interface that was used to generate the data. The MultiClass component allows analysts to revisit decisions made for prior studies and reuse them or not each time the data is used. We describe these two components with examples where a warehouse of clinical data is used to support research studies. We describe the state of our implementation and why we believe the two components can be automatically translated into ETL workflows.