Dawid Pieper

Overviews of reviews often have limited rigor: a systematic review

OBJECTIVE To examine published overviews of systematic reviews in terms of descriptive and methodological characteristics. STUDY DESIGN AND SETTING MEDLINE, Database of Abstracts of Reviews of Effects, the Cochrane Database of Systematic Reviews, and several Health Technology Assessment databases were searched for overviews of reviews up to February 2012. We extracted data from the methods and results sections of the included overviews. These data were analyzed descriptively as frequencies or medians and interquartile ranges. RESULTS We included 126 overviews of reviews. According to our sample, publication rates for overviews have risen in the last decade. The quality of the included reviews was systematically appraised in 64% of the overviews. The most commonly used assessment tools were the Overview Quality Assessment Questionnaire (26%), Assessment of Multiple Systematic Reviews (11%), and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (5%). Only three of 18 overviews restricted to Cochrane reviews in our sample performed a quality assessment. Strategies to deal with discordant reviews were reported in 5% of the overviews. Searches for additional primary studies were conducted in 5% of the overviews. CONCLUSION Overviews of reviews often lack methodological rigor. Methodological standards and reporting guidelines for overviews are needed to improve the quality of this new publication type.

State of Evidence on the Relationship between High-Volume Hospitals and Outcomes in Surgery: A Systematic Review of Systematic Reviews

presented at the annual meeting of the German S Medicine and Prevention (DGSMP), Essen, Germany, S and the annual meeting of the German Network for Research (DNVF), Dresden, Germany, September 2012. Received October 29, 2012; Revised December 20, 2 December 20, 2012. From the Institute for Research in Operative Medicine, Fa School of Medicine, Witten/Herdecke University, Cologne Correspondence address: Dawid Pieper, MPH, Institute Operative Medicine, Faculty of Health, School of Me Herdecke University, Ostmerheimer Str 200, Building Cologne, Germany. email: dawid.pieper@uni-wh.de a 2013 by the American College of Surgeons Published by Elsevier Inc. COLLECTIVE REVIEW State of Evidence on the Relationship between High-Volume Hospitals and Outcomes in Surgery: A Systematic Review of Systematic Reviews Dawid Pieper, MPH, Tim Mathes, MHE, Edmund Neugebauer, PhD, Michaela Eikermann, MD In particular in surgical disciplines, many studies have been published on the volumeeoutcomes relationship since Luft and colleagues explained their theory of it. A lot of empirical work has shown differences in patient outcomes between high-volume hospitals (HVH) and low-volume hospitals (LVH) in favor of HVH. Mortality and survival have been explored the most in this debate. According to theory, superior outcomes in HVH can be explained by their structural characteristics (eg, teaching status or ownership) and a higher degree of experience. From this, it follows that the volumee outcomes relationship is likely to be most eminent in high-risk, low-volume procedures, such as pancreatectomy or esophagectomy. To improve patient outcomes, policy makers introduced minimum hospital volumes for surgical procedures. Among other quality criteria, accredited institutions performing bariatric surgery for Medicare patients must perform at least 125 operations per year. The Leapfrog Group has recommended annual hospital volumes for several surgical procedures. The European Society of Mastology claims for at least 150 newly diagnosed cases of primary breast cancer per year in specialized breast units. In Germany, minimum hospital volumes have been established for elected procedures. Although there seems to be a lot of evidence on the volumeeoutcomes relationship for several surgical procedures, there has also been a lot of criticism on the methodological quality of studies. Including methodologically flawed studies in systematic reviews can introduce bias in results when study quality is ignored. ociety of Social eptember 2012 Health Services

Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties.

OBJECTIVES To summarize all available evidence on measurement properties in terms of reliability, validity, and feasibility of the Assessment of Multiple Systematic Reviews (AMSTAR) tool, including R(evised)-AMSTAR. STUDY DESIGN AND SETTING MEDLINE, EMBASE, Psycinfo, and CINAHL were searched for studies containing information on measurement properties of the tools in October 2013. We extracted data on study characteristics and measurement properties. These data were analyzed following measurement criteria. RESULTS We included 13 studies, four of them were labeled as validation studies. Nine articles dealt with AMSTAR, two articles dealt with R-AMSTAR, and one article dealt with both instruments. In terms of interrater reliability, most items showed a substantial agreement (>0.6). The median intraclass correlation coefficient (ICC) for the overall score of AMSTAR was 0.83 (range 0.60-0.98), indicating a high agreement. In terms of validity, ICCs were very high with all but one ICC lower than 0.8 when the AMSTAR score was compared with scores from other tools. Scoring AMSTAR takes between 10 and 20 minutes. CONCLUSION AMSTAR seems to be reliable and valid. Further investigations for systematic reviews of other study designs than randomized controlled trials are needed. R-AMSTAR should be further investigated as evidence for its use is limited and its measurement properties have not been studied sufficiently. In general, test-retest reliability should be investigated in future studies.

Adherence-enhancing interventions for highly active antiretroviral therapy in HIV-infected patients – a systematic review

The objective of this systematic review was to evaluate the effectiveness of adherence‐enhancing interventions for highly active antiretroviral therapy (HAART) in HIV‐infected patients in developed countries.

Adherence enhancing interventions for oral anticancer agents: A systematic review

BACKGROUND The use of oral anticancer agents has increased in the last decades. Adherence is a crucial factor for the success of oral anticancer agent therapy. However, many patients are non-adherent. OBJECTIVE The objective was to evaluate the effectiveness of adherence interventions in patients taking oral anticancer agents. METHODS A systematic literature search was performed in Medline and Embase. Titles and abstracts and in case of potential relevance, full-texts were assessed for eligibility according to the predefined inclusion criteria. The study quality was evaluated. Both process steps were carried out independently by two reviewers. Relevant data on study design, patients, interventions and results were extracted in standardized tables by one reviewer and checked by a second reviewer. RESULTS Six controlled studies were included. Only one study was randomized. The study quality was moderate to low. One study showed statistically significant results in favor of the adherence intervention, two studies showed a tendency in favor of the intervention, one study showed an inconsistent result depending on the adherence definition and one study showed almost identical adherence rates in both groups. One study showed a tendency in favor of the control group. CONCLUSIONS Although most of the interventions are not very effective, it appears that certain adherence enhancing interventions could have a promising effect. One crucial point is the consideration of the baseline adherence when choosing patients to avoid ceiling effects. The evidence is limited due to lack of sufficient studies and partly inconsistent results. Further high quality studies are needed.

Methods of international health technology assessment agencies for economic evaluations--a comparative analysis.

BackgroundThe number of Health Technology Assessment (HTA) agencies increases. One component of HTAs are economic aspects. To incorporate economic aspects commonly economic evaluations are performed. A convergence of recommendations for methods of health economic evaluations between international HTA agencies would facilitate the adaption of results to different settings and avoid unnecessary expense. A first step in this direction is a detailed analysis of existing similarities and differences in recommendations to identify potential for harmonization. The objective is to provide an overview and comparison of the methodological recommendations of international HTA agencies for economic evaluations.MethodsThe webpages of 127 international HTA agencies were searched for guidelines containing recommendations on methods for the preparation of economic evaluations. Additionally, the HTA agencies were requested information on methods for economic evaluations. Recommendations of the included guidelines were extracted in standardized tables according to 13 methodological aspects. All process steps were performed independently by two reviewers.ResultsFinally 25 publications of 14 HTA agencies were included in the analysis. Methods for economic evaluations vary widely. The greatest accordance could be found for the type of analysis and comparator. Cost-utility-analyses or cost-effectiveness-analyses are recommended. The comparator should continuously be usual care. Again the greatest differences were shown in the recommendations on the measurement/sources of effects, discounting and in the analysis of sensitivity. The main difference regarding effects is the focus either on efficacy or effectiveness. Recommended discounting rates range from 1.5% - 5% for effects and 3% - 5% for costs whereby it is mostly recommended to use the same rate for costs and effects. With respect to the analysis of sensitivity the main difference is that oftentimes the probabilistic or deterministic approach is recommended exclusively. Methods for modeling are only described vaguely and mainly with the rational that the “appropriate model” depends on the decision problem. Considering all other aspects a comparison is challenging as recommendations vary regarding detailedness and addressed issues.ConclusionThere is a considerable unexplainable variance in recommendations. Further effort is needed to harmonize methods for preparing economic evaluations.

Adherence influencing factors in patients taking oral anticancer agents: a systematic review.

BACKGROUND The use of oral anticancer agents increased steadily in the last decades. Although oral anticancer agent adherence is important for a successful treatment, many patients are insufficiently adherent. PURPOSE To evaluate adherence influencing factors in patients taking oral anticancer agents. METHODS A systematic literature search was performed in Medline and Embase. Titles and abstracts and in case of relevance, full-texts were screened according to predefined inclusion criteria. The risk of bias was assessed. Both were carried out independently by two reviewers. Relevant data on study characteristics and results were extracted in standardized tables by one reviewer and checked by a second. A meta-analysis was not performed because of clinical and methodological heterogeneity between the studies to avoid misleading results. Data were synthesized in narrative way using a standardized procedure. RESULTS Twenty-two relevant studies were identified. The study quality was moderate. Especially the risk of bias regarding the measurement of influencing factors and adherence was mostly unclear. Social support, intake of aromatase inhibitors, and lower out-of-pocket costs for OACA seem to have a positive effect on adherence. Depression and the number of different medications seem to have a negative effect on adherence. Low age and very high age seem to be associated with lower adherence. The remaining factors showed either mostly no influence or were heterogeneous regarding the effect direction and statistical significance. CONCLUSIONS There are some factors that seem to have influence on adherence in patients taking OACA. However, due to the heterogeneity no general conclusions can be made also for these factors that can be applied to all indications, medications, settings, countries etc. The results should rather be considered as indications for factors that can have an influence on adherence to OACA.

Relationship between surgeon volume and outcomes: a systematic review of systematic reviews

BackgroundThe surgeon volume-outcome relationship has been discussed for many years and its existence or nonexistence is of importance for various reasons. A lot of empirical work has been published on it. We aimed to summarize systematic reviews in order to present current evidence.MethodsMedline, Embase, Cochrane database of systematic reviews (CDSR), and health technology assessment websites were searched up to October 2015 for systematic reviews on the surgeon volume-outcome relationship. Reviews were critically appraised, and results were extracted and synthesized by type of surgical procedure/condition.ResultsThirty-two reviews reporting on 15 surgical procedures/conditions were included. Methodological quality of included systematic reviews assessed with the assessment of multiple systematic reviews (AMSTAR) was generally moderate to high albeit included literature partly neglected considering methodological issues specific to volume-outcome relationship. Most reviews tend to support the presence of a surgeon volume-outcome relationship. This is most clear-cut in colorectal cancer, bariatric surgery, and breast cancer where reviews of high quality show large effects.ConclusionsWhen taking into account its limitations, this overview can serve as an informational basis for decision makers. Our results seem to support a positive volume-outcome relationship for most procedures/conditions. However, forthcoming reviews should pay more attention to methodology specific to volume-outcome relationship. Due to the lack of information, any numerical recommendations for minimum volume thresholds are not possible. Further research is needed for this issue.

Effects of computer-aided clinical decision support systems in improving antibiotic prescribing by primary care providers: a systematic review

OBJECTIVE To assess the effectiveness of computer-aided clinical decision support systems (CDSS) in improving antibiotic prescribing in primary care. METHODS A literature search utilizing Medline (via PubMed) and Embase (via Embase) was conducted up to November 2013. Randomized controlled trials (RCTs) and cluster randomized trials (CRTs) that evaluated the effects of CDSS aiming at improving antibiotic prescribing practice in an ambulatory primary care setting were included for review. Two investigators independently extracted data about study design and quality, participant characteristics, interventions, and outcomes. RESULTS Seven studies (4 CRTs, 3 RCTs) met our inclusion criteria. All studies were performed in the USA. Proportions of eligible patient visits that triggered CDSS use varied substantially between intervention arms of studies (range 2.8-62.8%). Five out of seven trials showed marginal to moderate statistically significant effects of CDSS in improving antibiotic prescribing behavior. CDSS that automatically provided decision support were more likely to improve prescribing practice in contrast to systems that had to be actively initiated by healthcare providers. CONCLUSIONS CDSS show promising effectiveness in improving antibiotic prescribing behavior in primary care. Magnitude of effects compared to no intervention, appeared to be similar to other moderately effective single interventions directed at primary care providers. Additional research is warranted to determine CDSS characteristics crucial to triggering high adoption by providers as a perquisite of clinically relevant improvement of antibiotic prescribing.

Can AMSTAR also be applied to systematic reviews of non-randomized studies?

BackgroundThere is a lack of an instrument to evaluate systematic reviews of non-randomized studies in epidemiological research. The Assessment of Multiple Systematic Reviews (AMSTAR) is widely used to evaluate the scientific quality of systematic reviews, but it has not been validated for SRs of non-randomized studies. The objective of this paper is to report our experience in applying AMSTAR to systematic reviews of non-randomized studies in terms of applicability, reliability and feasibility. Thus, we applied AMSTAR to a recently published review of 32 systematic reviews of non-randomized studies investigating the hospital volume-outcome relationship in surgery.ResultsThe inter-rater reliability was high (0.76), albeit items 8 (scientific quality used in formulating conclusions), 9 (appropriate method to combine studies), and 11 (conflicts of interest) scored moderate (≤0.58). However, there was a high heterogeneity between the two pairs of reviewers. In terms of feasibility, AMSTAR proved easy to apply to systematic reviews of non-randomized studies, each review taking 5–10 minutes to complete. We faced problems in applying three items, mainly related to scientific quality of the included studies.ConclusionsAMSTAR showed good psychometric properties, comparable to prior findings in systematic reviews of randomized controlled trials. AMSTAR can be applied to systematic reviews of non-randomized studies, although there are some item specific issues users should be aware of. Revisions and extensions of AMSTAR might be helpful.