James H. Derzon

Effectiveness of Practices To Increase Timeliness of Providing Targeted Therapy for Inpatients with Bloodstream Infections: a Laboratory Medicine Best Practices Systematic Review and Meta-analysis

SUMMARY Background. Bloodstream infection (BSI) is a major cause of morbidity and mortality throughout the world. Rapid identification of bloodstream pathogens is a laboratory practice that supports strategies for rapid transition to direct targeted therapy by providing for timely and effective patient care. In fact, the more rapidly that appropriate antimicrobials are prescribed, the lower the mortality for patients with sepsis. Rapid identification methods may have multiple positive impacts on patient outcomes, including reductions in mortality, morbidity, hospital lengths of stay, and antibiotic use. In addition, the strategy can reduce the cost of care for patients with BSIs. Objectives. The purpose of this review is to evaluate the evidence for the effectiveness of three rapid diagnostic practices in decreasing the time to targeted therapy for hospitalized patients with BSIs. The review was performed by applying the Centers for Disease Control and Preventions (CDCs) Laboratory Medicine Best Practices Initiative (LMBP) systematic review methods for quality improvement (QI) practices and translating the results into evidence-based guidance (R. H. Christenson et al., Clin Chem 57:816–825, 2011, http://dx.doi.org/10.1373/clinchem.2010.157131). Search strategy. A comprehensive literature search was conducted to identify studies with measurable outcomes. A search of three electronic bibliographic databases (PubMed, Embase, and CINAHL), databases containing “gray” literature (unpublished academic, government, or industry evidence not governed by commercial publishing) (CIHI, NIHR, SIGN, and other databases), and the Cochrane database for English-language articles published between 1990 and 2011 was conducted in July 2011. Dates of search. The dates of our search were from 1990 to July 2011. Selection criteria. Animal studies and non-English publications were excluded. The search contained the following medical subject headings: bacteremia; bloodstream infection; time factors; health care costs; length of stay; morbidity; mortality; antimicrobial therapy; rapid molecular techniques, polymerase chain reaction (PCR); in situ hybridization, fluorescence; treatment outcome; drug therapy; patient care team; pharmacy service, hospital; hospital information systems; Gram stain; pharmacy service; and spectrometry, mass, matrix-assisted laser desorption-ionization. Phenotypic as well as the following key words were searched: targeted therapy; rapid identification; rapid; Gram positive; Gram negative; reduce(ed); cost(s); pneumoslide; PBP2; tube coagulase; matrix-assisted laser desorption/ionization time of flight; MALDI TOF; blood culture; EMR; electronic reporting; call to provider; collaboration; pharmacy; laboratory; bacteria; yeast; ICU; and others. In addition to the electronic search being performed, a request for unpublished quality improvement data was made to the clinical laboratory community. Main results. Rapid molecular testing with direct communication significantly improves timeliness compared to standard testing. Rapid phenotypic techniques with direct communication likely improve the timeliness of targeted therapy. Studies show a significant and homogeneous reduction in mortality associated with rapid molecular testing combined with direct communication. Authors conclusions. No recommendation is made for or against the use of the three assessed practices of this review due to insufficient evidence. The overall strength of evidence is suggestive; the data suggest that each of these three practices has the potential to improve the time required to initiate targeted therapy and possibly improve other patient outcomes, such as mortality. The meta-analysis results suggest that the implementation of any of the three practices may be more effective at increasing timeliness to targeted therapy than routine microbiology techniques for identification of the microorganisms causing BSIs. Based on the included studies, results for all three practices appear applicable across multiple microorganisms, including methicillin-resistant Staphylococcus aureus (MRSA), methicillin-sensitive S. aureus (MSSA), Candida species, and Enterococcus species.

Laboratory Medicine Best Practices: Systematic Evidence Review and Evaluation Methods for Quality Improvement

OBJECTIVE To develop methods for systematically reviewing evidence for identifying effective laboratory medicine (LM) practices associated with improved healthcare quality outcomes. RELEVANCE Although many evidence-evaluation systems have been developed, none are designed to include and rate healthcare quality improvement studies to identify evidence-based practices that improve patient safety and LM quality. METHODS Validated evidence-based medicine methods established by governmental agencies, the Guide to Community Preventive Services, and others were adapted for the LM field. Key methods modifications included (a) inclusion of quality improvement study designs; (b) mechanisms for inclusion of unpublished evidence, (c) combining of individual ratings of study quality, effect size, and relevance of outcome measures to evaluate consistency of practice evidence; and (d) deriving an overall strength rating to support evidence-based best practice recommendations. The methods follow the process steps of: ask; acquire; appraise; analyze; apply; and assess. Expert panels used the systematic evidence review results on practice effectiveness for improving healthcare quality outcomes consistent with the Institute of Medicines healthcare quality aims (safe, timely, effective, equitable, efficient, and patient-centered). CONCLUSIONS Adapting and developing methods from validated systems and applying them to systematically review and evaluate practices in LM by using published and unpublished studies is feasible. With these methods, evidence from quality improvement studies can be systematically synthesized and summarized to identify effective LM practices. Practical and scientifically validated demonstration of a positive impact on outcomes ensures that practitioners, policy makers, and decision makers at all levels have the evidence needed for improving healthcare quality and public health.

A Preliminary Study of the Population-Adjusted Effectiveness of Substance Abuse Prevention Programming: Towards Making IOM Program Types Comparable

The Institute of Medicine distinguishes between programs based on who is targeted: the entire population (universal), those at risk (selective), or persons exhibiting the early stages of use or related problem behavior (indicated). Evaluations suggest that although universal programs can be effective in reducing and preventing substance use, selective and indicated programs are both more effective and have greater cost-benefit ratios. This paper tests these assumptions by comparing the impact of these program types in reducing and preventing substance use at the individual level (i.e., those exposed to intervention services) and in the population (i.e., those exposed and not exposed to intervention services). A meta-analysis was performed on 43 studies of 25 programs to examine program comparability across IOM categories. When examining unadjusted effect sizes at the individual level, universal programs were modestly more successful in reducing tobacco use, but selective and indicated programs were modestly more successful in reducing alcohol and marijuana use. When adjusted to the population level, the average effect sizes for selective and indicated programs were reduced by approximately half. At the population level, universal programs were more successful in reducing tobacco and marijuana use and selective and indicated programs were more successful in reducing alcohol use. Editors’ Strategic Implications: The authors’ focus on the public health value of a prevention strategy is compelling and provides a model for analyses of other strategies and content areas.

Using Correlational Evidence to Select Youth for Prevention Programming

In a period of increased accountability and reduced prevention resources, the effective targeting of those limited resources is critical. One way in which limited resources are focused is to identify and provide services to those most at risk for later substance use. Risk status, or propensity, is typically estimated from correlational evidence. Using meta-analytic techniques this paper examines the evidence that 29 of the 35 constructs specified by the CTC risk and protective factor model are related to alcohol, tobacco, or marijuana use. While these factors are generally demonstrated to be predictive of substance use, the strength of relation is modest. Ten factors show a significantly different strength of relation with tobacco than with alcohol and marijuana. Given the correlations observed and the rate of substance use in the population, providing only selective intervention services likely ignores the majority of those who will later use substances. Although selection improves the percentage of those receiving services who are likely to benefit from services, the evidence summarized in this study suggests selective interventions will omit many of those who will likely use substances. Given typical base and selection rates, smaller program effects on universal populations may keep a greater number of youth from becoming alcohol, tobacco, or marijuana involved. Editors’ Strategic Implications: The data make a strong and provocative argument for primary prevention of youth substance abuse that should be heard by policymakers and service providers involved in strategic planning and appropriate deployment of resources.

A national evaluation of Safe Schools/Healthy Students: Outcomes and influences

The Safe Schools/Healthy Students (SS/HS) Initiative has awarded over

The use of one- versus two-tailed tests to evaluate prevention programs.

2 billion in grants to more than 350 school districts in partnership with local mental health, law enforcement, and juvenile justice agencies. To estimate the impact of grantee characteristics, grant operations, and near-term outcomes in reducing violence and substance use, promoting mental health, and enhancing school safety, logged odds ratios (LORs) were calculated contrasting Year 3 with Baseline performance from grantee-provided data on seven outcome measures. After comparing grantee performance across outcomes and outcomes across grantees, the LORs were entered as dependent variables in a series of meta-regressions in which grantee characteristics, grant operations, and near-term outcomes were tested after controlling for pre-grant characteristics. Findings indicate that the SS/HS Initiative significantly improved most outcomes, that within-grantee performance varied greatly by outcome, and that random-effects meta-regression appreciably decreased the variance available for modeling. The approach demonstrates that the SS/HS Initiative is effective and that locally collected performance data can be used to estimate grantee success in improving youth outcomes.

A synthetic comparator approach to local evaluation of school-based substance use prevention programming.

Investigators have used both one- and two-tailed tests to determine the significance of findings yielded by program evaluations. While the literature that addresses the appropriate use of each type of significance test should be used is historically inconsistent, almost all authorities now agree that one-tailed tests are rarely (if ever) appropriate. A review of 85 published evaluations of school-based drug prevention curricula specified on the National Registry of Effective Programs and Practices revealed that 20% employed one-tailed tests and, within this subgroup, an additional 4% also employed two-tailed tests. The majority of publications either did not specify the type of statistical test employed or used some other criterion such as effect sizes or confidence intervals. Evaluators reported that they used one-tailed tests either because they stipulated the direction of expected findings in advance, or because prior evaluations of similar programs had yielded no negative results. The authors conclude that one-tailed tests should never be used because they introduce greater potential for Type I errors and create an uneven playing field when outcomes are compared across programs. The authors also conclude that the traditional threshold of significance that places α at .05 is arbitrary and obsolete, and that evaluators should consistently report the exact p values they find.

The correspondence of family features with problem, aggressive, criminal, and violent behavior: a meta-analysis

We propose a method for creating groups against which outcomes of local pretest–posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness.