Tanja Perry

Health Information Technology: An Updated Systematic Review With a Focus on Meaningful Use

In the United States, adoption of health information technology (IT) has been accelerated by the meaningful use incentive program, which provides financial incentives to individual health care providers and organizations that demonstrate that they use certified health IT to meet a set of several use criteria specified by the Centers for Medicare & Medicaid Services (1, 2). This increase in use has been accompanied by a concomitant increase in the number of published evaluations of health IT. Because of the rapidly expanding evidence base, the Office of the National Coordinator requested a systematic update of the literature (3). The objective of this review is to update previous reviews (46) and examine recent evidence that relates health IT functionalities prescribed in meaningful use regulations to health care quality (including process, health, and patient and provider satisfaction outcomes), safety (including medication safety and other manifestations of patient safety), and efficiency (including costs, utilization, timeliness, and time burden of health care). Methods Although we did not develop a formal protocol for this update, we based it on the search strategy, inclusion and exclusion criteria, data collection, and synthesis methods from previous reviews on health IT (46). Data Sources and Search Strategy A 5-person technical expert panel, which included academic, health care delivery, and policy experts in health IT, guided the review process. Literature searches were based on the strategy initially used by Chaudhry and colleagues (4) and updated by Goldzweig (5) and Buntin (6) and their respective colleagues. This strategy uses broad-based search terms for the English-language literature indexed in PubMed. Our initial search covered the period of January 2010 to November 2011. We used a computer-aided screening method (7) to update that search to November 2012, and then updated searches again to August 2013 (Tables 1 and 2 of the Supplement). Our expert panel reviewed the search results and suggested additional articles that may have been missed. Supplement. Tables Study Selection Two expert reviewers used a Web-based system, DistillerSR (8), to independently select studies. Following the methods of our previous reviews, we considered hypothesis-testing studies of health IT effects and descriptive quantitative studies for inclusion. We classified articles as hypothesis-testing if the investigators compared data between groups or across periods and used statistical tests to assess differences. Hypothesis-testing studies were further classified by study design (such as randomized, controlled trials [RCTs]). To be included, a study needed to evaluate a health IT functionality encompassed by the meaningful use regulations. The meaningful use requirements specify 25 criteria for health IT functionality and use (such as Use CPOE for medication orders), of which providers must meet a portion to receive incentive payments (Tables 3 and 4 of the Supplement). Data Synthesis and Analysis Using a structured form, we abstracted information about the following: study design; clinical setting, health care conditions, and aspects of care assessed; research sites; health IT type (commercial or homegrown); meaningful use functionality evaluated; and context and implementation details. We adapted criteria developed to assess health IT applications in patient safety to classify articles according to reported context (4 domains) and implementation details (7 specific components) (9). We used a modified version of the outcome result classification framework (positive, mixed-positive, neutral, or negative), originally used by Buntin and colleagues (6), to assess outcomes. Our adaptations made the classification framework more conservative than the original framework, thus increasing the likelihood that an articles findings would be classified as mixed-positive, neutral, or negative (Table 5 of the Supplement). The functionality evaluated, context domains, implementation components, and article outcomes were classified by dual-review, and conflicts were resolved by consensus. Role of the Funding Source This project was done under contract to the Office of the National Coordinator. Representatives of the Office of the National Coordinator were briefed on study findings and reviewed a draft manuscript but were not involved in the analysis or decision to submit the manuscript for publication. Results We deemed 2482 of 12 678 titles identified in the searches potentially relevant. We excluded 2023 of these after abstract review and another 223 after full-text review. The 236 articles selected for review encompassed 278 outcomes because some articles addressed several aspects of care and outcomes (Figure). The full list of the included studies can be found in the reference list in the Supplement. Quality outcomes (n= 170) were evaluated more than safety (n= 46) and efficiency (n= 62) outcomes combined, and care processes (n= 103) outcomes were more than twice as common as health outcomes (n= 47). Studies with a simple prepost design were most common (31%), followed by RCTs (25%) and time-series studies (11%). More than one half (53%) evaluated commercial health IT products, although approximately one quarter did not report whether the products that were evaluated were commercial or homegrown. Figure. Evidence search and selection. Meaningful Use Functionalities Most studies addressed clinical decision support (CDS)(n= 85 [36%]); computerized provider order entry (CPOE) (n= 49 [21%]); or multifunctional health IT interventions (n= 47 [20%]), which evaluated broad IT interventions, such as electronic health records (EHRs), that encompassed many of the functionalities required under meaningful use. Twelve of the 25 functionalities in the meaningful use regulations, such as capacity to track vital signs or maintain medication allergy lists, were not specifically evaluated in any studies. These features, however, were likely critical to the functionality of IT interventions (such as CDS) that were evaluated in many studies. Nevertheless, for some aspects of meaningful use, such as implement systems to protect privacy and security, no eligible studies were found. Quality Outcomes Overall, 147 articles assessed the effect of health IT on 170 quality-related outcomes (Tables 6 and 7 of the Supplement). More than one half of these studies assessed CDS alerts and reminders; the most commonly studied outcomes were medication management, screening and preventive care, and process quality for diabetes and venous thromboembolism. More than three quarters of studies of alerts and reminders, including large quasi-experimental studies and RCTs, reported positive effects. Notable among these included a 1-year prospective, cluster randomized trial in 12 primary care pediatric practices that reported that EHR-embedded CDS substantially improved the use of asthma control medications, spirometry, and the maintenance of care plans (10); a study of nearly 20 000 surgical patients that found that CDS was associated with a 30% increase in adherence to infection prevention guidelines (11); a controlled before-and-after study compared 360 primary care physicians in New York, New York, with a matched set of control physicians and reported that EHR-sensitive measures of process quality (such as eye examinations and urine testing for patients with diabetes) improved substantially only in practices that received high levels of technical assistance in implementing their EHRs (12); and a before-and-after study showing that a CDS intervention in a commercial health IT system was associated with a significant increase in venous thromboembolism prophylaxis and a substantial decline in the rate of venous thromboembolism among nearly 40 000 patients admitted to a single academic medical center (13). Eighteen percent of studies reported no statistically significant improvements in quality or even negative effects. An illustrative example of these studies reported that on the basis of a nationally representative sample of physician office visits, EHRs were associated with poor depression care among patients diagnosed with multiple chronic conditions. The authors of this study hypothesized that EHR workflows gave precedence to the treatment of physical conditions, and therefore psychosocial problems were left unaddressed (14). Safety Outcomes Forty-six studies investigated the effects of health IT functionalities on patient safety outcomes, focusing exclusively on medication safety (Tables 8 and 9 of the Supplement). Approximately 78% of these studies reported at least some positive effects. Of note, benefits were found for a wide range of medication safety outcomes in various care settings. Automated dose calculation features within CPOE systems were found to have significant relative reductions in errors in medication dosage ranging from 37% to 80% (1517), and authors of 1 study reported that innovative features, such as incorporation of an order verification screen with a patient picture, was associated with complete elimination of incorrect patient orders (18). However, a few studies reported that, in some cases, health IT did not have the desired effect on medication safety, and alert fatigue and incongruent workflows were described as barriers to successful use of these systems. Efficiency Outcomes We identified 58 articles that assessed the effect of health IT on 62 efficiency-related outcomes (Tables 10 and 11 of the Supplement). Cost effects ranged from a 75% decrease to a 69% increase in the targeted costs. A few of the studies clustered in the range of 6% to 12% increases in the targeted costs. Understanding the relationship between health IT and health care utilization is complicated by external factors, such as the payment environment in which the health care providers operate. We identified some large studies that reported that patient and provider acces

Safety of Vaccines Used for Routine Immunization of US Children: A Systematic Review

BACKGROUND: Concerns about vaccine safety have led some parents to decline recommended vaccination of their children, leading to the resurgence of diseases. Reassurance of vaccine safety remains critical for population health. This study systematically reviewed the literature on the safety of routine vaccines recommended for children in the United States. METHODS: Data sources included PubMed, Advisory Committee on Immunization Practices statements, package inserts, existing reviews, manufacturer information packets, and the 2011 Institute of Medicine consensus report on vaccine safety. We augmented the Institute of Medicine report with more recent studies and increased the scope to include more vaccines. Only studies that used active surveillance and had a control mechanism were included. Formulations not used in the United States were excluded. Adverse events and patient and vaccine characteristics were abstracted. Adverse event collection and reporting was evaluated by using the McHarm scale. We were unable to pool results. Strength of evidence was rated as high, moderate, low, or insufficient. RESULTS: Of 20 478 titles identified, 67 were included. Strength of evidence was high for measles/mumps/rubella (MMR) vaccine and febrile seizures; the varicella vaccine was associated with complications in immunodeficient individuals. There is strong evidence that MMR vaccine is not associated with autism. There is moderate evidence that rotavirus vaccines are associated with intussusception. Limitations of the study include that the majority of studies did not investigate or identify risk factors for AEs; and the severity of AEs was inconsistently reported. CONCLUSIONS: We found evidence that some vaccines are associated with serious AEs; however, these events are extremely rare and must be weighed against the protective benefits that vaccines provide.

Reporting of context and implementation in studies of global health interventions: a pilot study

BackgroundThere is an increasing push for ‘evidence-based’ decision making in global health policy circles. However, at present there are no agreed upon standards or guidelines for how to evaluate evidence in global health. Recent evaluations of existing evidence frameworks that could serve such a purpose have identified details of program context and project implementation as missing components needed to inform policy. We performed a pilot study to assess the current state of reporting of context and implementation in studies of global health interventions.MethodsWe identified three existing criteria sets for implementation reporting and selected from them 10 criteria potentially relevant to the needs of policy makers in global health contexts. We applied these 10 criteria to 15 articles included in the evidence base for three global health interventions chosen to represent a diverse set of advocated global health programs or interventions: household water chlorination, prevention of mother-to-child transmission of HIV, and lay community health workers to reduce child mortality. We used a good-fair-poor/none scale for the ratings.ResultsThe proportion of criteria for which reporting was poor/none ranged from 11% to 54% with an average of 30%. Eight articles had ‘good’ or ‘fair’ documentation for greater than 75% of criteria, while five articles had ‘poor or none’ documentation for 50% of criteria or more. Examples of good reporting were identified.ConclusionsReporting of context and implementation information in studies of global health interventions is mostly fair or poor, and highly variable. The idiosyncratic variability in reporting indicates that global health investigators need more guidance about what aspects of context and implementation to measure and how to report them. This lack of context and implementation information is a major gap in the evidence needed by global health policy makers to reach decisions.