Christina Kien

[GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes].

GRADE requires guideline developers to make an overall rating of confidence in estimates of effect (quality of evidence-high, moderate, low, or very low) for each important or critical outcome. GRADE suggests, for each outcome, the initial separate consideration of five domains of reasons for rating down the confidence in effect estimates, thereby allowing systematic review authors and guideline developers to arrive at an outcome-specific rating of confidence. Although this rating system represents discrete steps on an ordinal scale, it is helpful to view confidence in estimates as a continuum, and the final rating of confidence may differ from that suggested by separate consideration of each domain. An overall rating of confidence in estimates of effect is only relevant in settings when recommendations are being made. In general, it is based on the critical outcome that provides the lowest confidence.

Extended-release opioids in the management of cancer pain: a systematic review of efficacy and safety.

Despite the increased availability of strong analgesics and evidence‐based recommendations for pain management, under‐treatment of cancer‐related pain is still common. Extended‐release (ER) opioids, in contrast to immediate‐release opioids, provide prolonged analgesia. In this review, we aimed to compare the efficacy and safety of ER opioid analgesics in managing moderate‐to‐severe pain in patients with cancer. We identified randomized controlled trials (RCTs) and controlled observational studies that compared ER opioids in cancer pain by searching several databases, including MEDLINE, EMBASE and the Cochrane Library. Two independent reviewers screened and evaluated retrieved records to select relevant studies. We dually assessed the risk of bias for included studies and evaluated the overall strength of evidence for six critical outcomes using Grading of Recommendations Assessment, Development and Evaluation level of evidence. A total of three double‐blind RCTs (comparative efficacy and adverse events), two non‐blinded RCTs and four observational studies (comparative adverse events) were included in this review. All randomized trials and one observational study were of high risk of bias, and three observational studies of unclear risk of bias. The level of evidence for the selected efficacy and safety outcomes was low and very low. We synthesized the findings qualitatively because of the paucity of relevant studies as well as variable study design and quality. This systematic review indicates no substantial differences in efficacy and frequent adverse events among ER opioids for cancer pain. The body of evidence, however, is limited to few comparisons and fraught with methodological shortcomings.

Barriers to and facilitators of interventions to counter publication bias: thematic analysis of scholarly articles and stakeholder interviews

BackgroundWhen the nature and direction of research results affect their chances of publication, a distortion of the evidence base – termed publication bias – results. Despite considerable recent efforts to implement measures to reduce the non-publication of trials, publication bias is still a major problem in medical research. The objective of our study was to identify barriers to and facilitators of interventions to prevent or reduce publication bias.MethodsWe systematically reviewed the scholarly literature and extracted data from articles. Further, we performed semi-structured interviews with stakeholders. We performed an inductive thematic analysis to identify barriers to and facilitators of interventions to counter publication bias.ResultsThe systematic review identified 39 articles. Thirty-four of 89 invited interview partners agreed to be interviewed. We clustered interventions into four categories: prospective trial registration, incentives for reporting in peer-reviewed journals or research reports, public availability of individual patient-level data, and peer-review/editorial processes. Barriers we identified included economic and personal interests, lack of financial resources for a global comprehensive trial registry, and different legal systems. Facilitators identified included: raising awareness of the effects of publication bias, providing incentives to make data publically available, and implementing laws to enforce prospective registration and reporting of clinical trial results.ConclusionsPublication bias is a complex problem that reflects the complex system in which it occurs. The cooperation amongst stakeholders to increase public awareness of the problem, better tailoring of incentives to publish, and ultimately legislative regulations have the greatest potential for reducing publication bias.

The predictive validity of quality of evidence grades for the stability of effect estimates was low: a meta-epidemiological study

OBJECTIVE To determine the predictive validity of the U.S. Evidence-based Practice Center (EPC) approach to GRADE (Grading of Recommendations Assessment, Development and Evaluation). STUDY DESIGN AND SETTING Based on Cochrane reports with outcomes graded as high quality of evidence (QOE), we prepared 160 documents which represented different levels of QOE. Professional systematic reviewers dually graded the QOE. For each document, we determined whether estimates were concordant with high QOE estimates of the Cochrane reports. We compared the observed proportion of concordant estimates with the expected proportion from an international survey. To determine the predictive validity, we used the Hosmer-Lemeshow test to assess calibration and the C (concordance) index to assess discrimination. RESULTS The predictive validity of the EPC approach to GRADE was limited. Estimates graded as high QOE were less likely, estimates graded as low or insufficient QOE more likely to remain stable than expected. The EPC approach to GRADE could not reliably predict the likelihood that individual bodies of evidence remain stable as new evidence becomes available. C-indices ranged between 0.56 (95% CI, 0.47 to 0.66) and 0.58 (95% CI, 0.50 to 0.67) indicating a low discriminatory ability. CONCLUSION The limited predictive validity of the EPC approach to GRADE seems to reflect a mismatch between expected and observed changes in treatment effects as bodies of evidence advance from insufficient to high QOE.

Grade Leitlinien: 15. Von der Evidenz zur Empfehlung – Determinanten, die Richtung und Stärke einer Empfehlung bestimmen☆

In the GRADE approach, the strength of a recommendation reflects the extent to which we can be confident that the composite desirable effects of a management strategy outweigh the composite undesirable effects. This article addresses GRADEs approach to determining the direction and strength of a recommendation. The GRADE describes the balance of desirable and undesirable outcomes of interest among alternative management strategies depending on four domains, namely estimates of effect for desirable and undesirable outcomes of interest, confidence in the estimates of effect, estimates of values and preferences, and resource use. Ultimately, guideline panels must use judgment in integrating these factors to make a strong or weak recommendation for or against an intervention.

Identifying and prioritising systematic review topics with public health stakeholders: A protocol for a modified Delphi study in Switzerland to inform future research agendas

Introduction The Cochrane Collaboration aims to produce relevant and top priority evidence that responds to existing evidence gaps. Hence, research priority setting (RPS) is important to identify which potential research gaps are deemed most important. Moreover, RPS supports future health research to conform both health and health evidence needs. However, studies that are prioritising systematic review topics in public health are surprisingly rare. Therefore, to inform the research agenda of Cochrane Public Health Europe (CPHE), we introduce the protocol of a priority setting study on systematic review topics in several European countries, which is conceptualised as pilot. Methods and analysis We will conduct a two-round modified Delphi study in Switzerland, incorporating an anonymous web-based questionnaire, to assess which topics should be prioritised for systematic reviews in public health. In the first Delphi round public health stakeholders will suggest relevant assessment criteria and potential priority topics. In the second Delphi round the participants indicate their (dis)agreement to the aggregated results of the first round and rate the potential review topics with the predetermined criteria on a four-point Likert scale. As we invite a wide variety of stakeholders we will compare the results between the different stakeholder groups. Ethics and dissemination We have received ethical approval from the ethical board of the University of Bremen, Germany (principal investigation is conducted at the University of Bremen) and a certificate of non-objection from the Canton of Zurich, Switzerland (fieldwork will be conducted in Switzerland). The results of this study will be further disseminated through peer reviewed publication and will support systematic review author groups (i.a. CPHE) to improve the relevance of the groups´ future review work. Finally, the proposed priority setting study can be used as a framework by other systematic review groups when conducting a priority setting study in a different context.

Comparative evaluation of information products regarding cancer screening of German-speaking cancer organizations

BACKGROUND Evidence-based information materials about the pros and cons of cancer screening are important sources for men and women to decide for or against cancer screening. The aim of this paper was to compare recommendations from different cancer institutions in German-speaking countries (Austria, Germany, and Switzerland) regarding screening for breast, cervix, colon, and prostate cancer and to assess the quality and development process of patient information materials. METHODS Relevant information material was identified through web searches and personal contact with cancer institutions. To achieve our objective, we employed a qualitative approach. The quality of 22 patient information materials was analysed based on established guidance by Bunge et al. In addition, we conducted guided interviews about the process of developing information materials with decision-makers of cancer institutes. RESULTS Overall, major discrepancies in cancer screening recommendations exist among the Austrian, German, and Swiss cancer institutes. Process evaluation revealed that crucial steps of quality assurance, such as assembling a multi-disciplinary panel, assessing conflicts of interest, or transparency regarding funding sources, have frequently not been undertaken. All information materials had substantial quality deficits in multiple areas. Three out of four institutes issued information materials that met fewer than half of the quality criteria. CONCLUSION Most patient information materials of cancer institutes in German-speaking countries are fraught with substantial deficits and do not provide an objective source for patients to be able to make an informed decision for or against cancer screening.

Effects of lifestyle changes on adults with prediabetes: A systematic review and meta-analysis

AIMS To assess the efficacy, safety, and cost-effectiveness of lifestyle intervention, compared with treatment as usual in people with prediabetes as defined by the American Diabetes Association. For older studies, we used the 1985 World Health Organization definition. METHODS We systematically searched multiple electronic databases and referenced lists of pertinent review articles from January 1980 through November 2015. We performed an update search in MEDLINE on April 26, 2017. Based on a priori established eligibility criteria, we dually reviewed the literature, extracted data, and rated the risk of bias of included studies with validated checklists. To assess the efficacy of lifestyle intervention to prevent or delay further progression to type 2 diabetes, we conducted a random-effects meta-analysis. We assessed the certainty of evidence using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. RESULT Pooled results of 16 randomized controlled trials showed that people with prediabetes who received lifestyle intervention had a lower rate of progression to type 2 diabetes after one (4% vs. 10%, RR 0.46 [CI 0.32, 0.66]) and three years of follow-up (14% vs. 23%, RR 0.64 [95% CI 0.53, 0.77]). The majority of the studies also showed a greater weight loss in lifestyle intervention participants, with a great variation between studies. Costs per quality-adjusted life-year were lower when the benefits of lifestyle intervention were analyzed over a lifelong time horizon compared to only the period of lifestyle intervention (three years) or to modeling over a ten-year period. CONCLUSION Lifestyle intervention is an efficacious, safe, and cost-effective measure to reduce the risk of progression to type 2 diabetes in people diagnosed with prediabetes. More research is necessary to compare the efficacy of various modes, frequencies, and intensities of lifestyle intervention across studies.

Facilitating evidence uptake: development and user testing of a systematic review summary format to inform public health decision-making in German-speaking countries

BackgroundSystematic reviews are an important source of evidence for public health decision-making, but length and technical jargon tend to hinder their use. In non-English speaking countries, inaccessibility of information in the native language often represents an additional barrier. In line with our vision to strengthen evidence-based public health in the German-speaking world, we developed a German language summary format for systematic reviews of public health interventions and undertook user-testing with public health decision-makers in Germany, Austria and Switzerland.MethodsWe used several guiding principles and core elements identified from the literature to produce a prototype summary format and applied it to a Cochrane review on the impacts of changing portion and package sizes on selection and consumption of food, alcohol and tobacco. Following a pre-test in each of the three countries, we carried out 18 user tests with public health decision-makers in Germany, Austria and Switzerland using the ‘think-aloud’ method. We analysed participants’ comments according to the facets credibility, usability, understandability, usefulness, desirability, findability, identification and accessibility. We also identified elements that hindered the facile and satisfying use of the summary format, and revised it based on participants’ feedback.ResultsThe summary format was well-received; participants particularly appreciated receiving information in their own language. They generally found the summary format useful and a credible source of information, but also signalled several barriers to a positive user experience such as an information-dense structure and difficulties with understanding statistical terms. Many of the identified challenges were addressed through modifications of the summary format, in particular by allowing for flexible length, placing more emphasis on key messages and relevance for public health practice, expanding the interpretation aid for statistical findings, providing a glossary of technical terms, and only including graphical GRADE ratings. Some barriers to uptake, notably the participants’ wish for actionable recommendations and contextual information, could not be addressed.ConclusionsParticipants welcomed the initiative, but user tests also revealed their problems with understanding and interpreting the findings summarised in our prototype format. The revised summary format will be used to communicate the results of Cochrane reviews of public health interventions.

SchwerpunktVergleichende Evaluierung von Informationsprodukten zu Krebsscreening der deutschsprachigen Krebs-OrganisationenComparative evaluation of information products regarding cancer screening of German-speaking cancer organizations

BACKGROUND Evidence-based information materials about the pros and cons of cancer screening are important sources for men and women to decide for or against cancer screening. The aim of this paper was to compare recommendations from different cancer institutions in German-speaking countries (Austria, Germany, and Switzerland) regarding screening for breast, cervix, colon, and prostate cancer and to assess the quality and development process of patient information materials. METHODS Relevant information material was identified through web searches and personal contact with cancer institutions. To achieve our objective, we employed a qualitative approach. The quality of 22 patient information materials was analysed based on established guidance by Bunge et al. In addition, we conducted guided interviews about the process of developing information materials with decision-makers of cancer institutes. RESULTS Overall, major discrepancies in cancer screening recommendations exist among the Austrian, German, and Swiss cancer institutes. Process evaluation revealed that crucial steps of quality assurance, such as assembling a multi-disciplinary panel, assessing conflicts of interest, or transparency regarding funding sources, have frequently not been undertaken. All information materials had substantial quality deficits in multiple areas. Three out of four institutes issued information materials that met fewer than half of the quality criteria. CONCLUSION Most patient information materials of cancer institutes in German-speaking countries are fraught with substantial deficits and do not provide an objective source for patients to be able to make an informed decision for or against cancer screening.