Nicola Magrini

Grading quality of evidence and strength of recommendations.

Abstract Users of clinical practice guidelines and other recommendations need to know how much confidence they can place in the recommendations. Systematic and explicit methods of making judgments can reduce errors and improve communication. We have developed a system for grading the quality of evidence and the strength of recommendations that can be applied across a wide range of interventions and contexts. In this article we present a summary of our approach from the perspective of a guideline user. Judgments about the strength of a recommendation require consideration of the balance between benefits and harms, the quality of the evidence, translation of the evidence into specific circumstances, and the certainty of the baseline risk. It is also important to consider costs (resource utilisation) before making a recommendation. Inconsistencies among systems for grading the quality of evidence and the strength of recommendations reduce their potential to facilitate critical appraisal and improve communication of these judgments. Our system for guiding these complex judgments balances the need for simplicity with the need for full and transparent consideration of all important issues. Clinical guidelines are only as good as the evidence and judgments they are based on. The GRADE approach aims to make it easier for users to assess the judgments behind recommendations

Systems for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system

BackgroundSystems that are used by different organisations to grade the quality of evidence and the strength of recommendations vary. They have different strengths and weaknesses. The GRADE Working Group has developed an approach that addresses key shortcomings in these systems. The aim of this study was to pilot test and further develop the GRADE approach to grading evidence and recommendations.MethodsA GRADE evidence profile consists of two tables: a quality assessment and a summary of findings. Twelve evidence profiles were used in this pilot study. Each evidence profile was made based on information available in a systematic review. Seventeen people were given instructions and independently graded the level of evidence and strength of recommendation for each of the 12 evidence profiles. For each example judgements were collected, summarised and discussed in the group with the aim of improving the proposed grading system. Kappas were calculated as a measure of chance-corrected agreement for the quality of evidence for each outcome for each of the twelve evidence profiles. The seventeen judges were also asked about the ease of understanding and the sensibility of the approach. All of the judgements were recorded and disagreements discussed.ResultsThere was a varied amount of agreement on the quality of evidence for the outcomes relating to each of the twelve questions (kappa coefficients for agreement beyond chance ranged from 0 to 0.82). However, there was fair agreement about the relative importance of each outcome. There was poor agreement about the balance of benefits and harms and recommendations. Most of the disagreements were easily resolved through discussion. In general we found the GRADE approach to be clear, understandable and sensible. Some modifications were made in the approach and it was agreed that more information was needed in the evidence profiles.ConclusionJudgements about evidence and recommendations are complex. Some subjectivity, especially regarding recommendations, is unavoidable. We believe our system for guiding these complex judgements appropriately balances the need for simplicity with the need for full and transparent consideration of all important issues.

An emerging consensus on grading recommendations

Clinical practice guidelines have improved in quality over the past 10 years by adhering to a few basic principles, such as conducting thorough systematic reviews of relevant evidence and grading the recommendations and the quality of the underlying evidence. The large number of systems of measuring the quality of evidence and recommendations that have emerged are, however, confusing.1 The mission of the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) working group is to help resolve the confusion among the different systems of rating evidence and recommendations. The group has wide representation from many organisations including the Agency for Healthcare Research and Quality in the US, the National Institute for Clinical Excellence for England and Wales, and the World Health Organization. Developing a new uniform rating system is challenging because all systems have limitations and because many organisations have invested a great deal of time and effort to develop their rating systems and are understandably reluctant to adopt a new system. The GRADE working group first published the results of its work in 2004 in the BMJ.2 A simpler, clinically oriented description will soon be published.3 GRADE has taken care to ensure its suggested system is simple to use and applicable to a wide variety of clinical recommendations that span the full spectrum of medical specialties and clinical care. The GRADE system classifies recommendations in 1 of 2 levels—strong and weak—and quality of evidence into 1 of 4 levels—high, moderate, low, and very low. Evidence based on randomised controlled trials (RCTs) begins with a top rating on GRADE’s 4 level quality of evidence classification (table …

Discovery, research, and development of new antibiotics: the WHO priority list of antibiotic-resistant bacteria and tuberculosis

BACKGROUND The spread of antibiotic-resistant bacteria poses a substantial threat to morbidity and mortality worldwide. Due to its large public health and societal implications, multidrug-resistant tuberculosis has been long regarded by WHO as a global priority for investment in new drugs. In 2016, WHO was requested by member states to create a priority list of other antibiotic-resistant bacteria to support research and development of effective drugs. METHODS We used a multicriteria decision analysis method to prioritise antibiotic-resistant bacteria; this method involved the identification of relevant criteria to assess priority against which each antibiotic-resistant bacterium was rated. The final priority ranking of the antibiotic-resistant bacteria was established after a preference-based survey was used to obtain expert weighting of criteria. FINDINGS We selected 20 bacterial species with 25 patterns of acquired resistance and ten criteria to assess priority: mortality, health-care burden, community burden, prevalence of resistance, 10-year trend of resistance, transmissibility, preventability in the community setting, preventability in the health-care setting, treatability, and pipeline. We stratified the priority list into three tiers (critical, high, and medium priority), using the 33rd percentile of the bacteriums total scores as the cutoff. Critical-priority bacteria included carbapenem-resistant Acinetobacter baumannii and Pseudomonas aeruginosa, and carbapenem-resistant and third-generation cephalosporin-resistant Enterobacteriaceae. The highest ranked Gram-positive bacteria (high priority) were vancomycin-resistant Enterococcus faecium and meticillin-resistant Staphylococcus aureus. Of the bacteria typically responsible for community-acquired infections, clarithromycin-resistant Helicobacter pylori, and fluoroquinolone-resistant Campylobacter spp, Neisseria gonorrhoeae, and Salmonella typhi were included in the high-priority tier. INTERPRETATION Future development strategies should focus on antibiotics that are active against multidrug-resistant tuberculosis and Gram-negative bacteria. The global strategy should include antibiotic-resistant bacteria responsible for community-acquired infections such as Salmonella spp, Campylobacter spp, N gonorrhoeae, and H pylori. FUNDING World Health Organization.

Feasibility and effectiveness of a low cost campaign on antibiotic prescribing in Italy: community level, controlled, non-randomised trial.

Objectives To test the hypothesis that a multifaceted, local public campaign could be feasible and influence antibiotic prescribing for outpatients. Design Community level, controlled, non-randomised trial. Setting Provinces of Modena and Parma in Emilia-Romagna, northern Italy, November 2011 to February 2012. Population 1 150 000 residents of Modena and Parma (intervention group) and 3 250 000 residents in provinces in the same region but where no campaign had been implemented (control group). Interventions Campaign materials (mainly posters, brochures, and advertisements on local media, plus a newsletter on local antibiotic resistance targeted at doctors and pharmacists). General practitioners and paediatricians in the intervention area participated in designing the campaign messages. Main outcomes measures Primary outcome was the average change in prescribing rates of antibiotics for outpatient in five months, measured as defined daily doses per 1000 inhabitants/day, using health districts as the unit of analysis. Results Antibiotic prescribing was reduced in the intervention area compared with control area (−4.3%, 95% confidence interval −7.1% to −1.5%). This result was robust to “sensitivity analysis” modifying the baseline period from two months (main analysis) to one month. A higher decrease was observed for penicillins resistant to β lactamase and a lower decrease for penicillins susceptible to β lactamase, consistent with the content of the newsletter on antibiotic resistance directed at health professionals. The decrease in expenditure on antibiotics was not statistically significant in a district level analysis with a two month baseline period (main analysis), but was statistically significant in sensitivity analyses using either a one month baseline period or a more powered doctor level analysis. Knowledge and attitudes of the target population about the correct use of antibiotics did not differ between the intervention and control areas. Conclusions A local low cost information campaign targeted at citizens, combined with a newsletter on local antibiotic resistance targeted at doctors and pharmacists, was associated with significantly decreased total rates of antibiotic prescribing but did not affect the population’s knowledge and attitudes about antibiotic resistance. Trial registration ClinicalTrials.gov NCT01604096.

Proposing Essential Medicines to Treat Cancer: Methodologies, Processes, and Outcomes

PURPOSE A great proportion of the worlds cancer burden resides in low- and middle-income countries where cancer care infrastructure is often weak or absent. Although treatment of cancer is multidisciplinary, involving surgery, radiation, systemic therapies, pathology, radiology, and other specialties, selection of medicines that have impact and are affordable has been particularly challenging in resource-constrained settings. In 2014, at the invitation of the WHO, the Union for International Cancer Control convened experts to develop an approach to propose essential cancer medicines to be included in the WHO Model Essential Medicines Lists (EML) for Adults and for Children, as well as a resulting new list of cancer medicines. METHODS Experts identified 29 cancer types with potential for maximal treatment impact, on the basis of incidence and benefit of systemic therapies. More than 90 oncology experts from all continents drafted and reviewed disease-based documents outlining epidemiology, diagnostic needs, treatment options, and benefits and toxicities. RESULTS Briefing documents were created for each disease, along with associated standard treatment regimens, resulting in a list of 52 cancer medicines. A comprehensive application was submitted as a revision to the existing cancer medicines on the WHO Model Lists. In May 2015, the WHO announced the addition of 16 medicines to the Adult EML and nine medicines to the Childrens EML. CONCLUSION The list of medications proposed, and the ability to link each recommended medicine to specific diseases, should allow public officials to apply resources most effectively in developing and supporting nascent or growing cancer treatment programs.

Information from drug companies and opinion leaders

Double standards in information for medical journals and practitioners should go

The role of Cochrane reviews in informing international guidelines: a case study of using the Grading of Recommendations, Assessment, Development and Evaluation system to develop World Health Organization guidelines for the psychosocially assisted pharmacological treatment of opioid dependence

BACKGROUND AND AIMS The World Health Organization (WHO), and a growing number of other organizations, have adopted the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system in order to both assess the quality of research evidence and develop clinical practice guidelines. In 2009 WHO published a guideline on psychosocially assisted pharmacological treatment of opioid dependence, based on the results of Cochrane Reviews summarized using the GRADE methodology. The main features of this system are an a priori definition of outcomes and their relevance, and distinction between the quality of evidence (also referred to as confidence in the estimate of intervention effect) and the strength of recommendations. We consider how successful this approach has been. ANALYSIS AND EVIDENCE We discuss the merits and limitations of using Cochrane Reviews and GRADE framework in developing guidelines in the field of drug addiction. In 2009 a panel of multi-disciplinary international experts identified 15 clinical questions and eight relevant outcomes. Cochrane reviews were available for each clinical question and four outcomes. The panel formulated 15 recommendations. Eight recommendations were classified as strong, two of which were based on high-quality evidence and three on very low-quality evidence. For example, the strong recommendation to use methadone in adequate doses in preference to buprenorphine was based on high-quality evidence, while the strong recommendation not to use the combination of opioid antagonists with heavy sedation in the management of opioid withdrawal was based on low-quality evidence. CONCLUSIONS An explicit stepwise process of moving from evaluation of the quality of evidence to the definition of the strength of recommendations is important in providing practical and clear clinical guidance for practitioners and policy-makers in addiction.

Essential medicines for cancer: WHO recommendations and national priorities

Abstract Objective To examine, for essential anti-cancer medicines, the alignment of national lists of essential medicines and national reimbursable medicines lists with the World Health Organization’s (WHO’s) Model Lists. Methods National medicine lists for 135 countries with per-capita gross national incomes below 25 000 United States dollars in 2015 were compared with WHO’s 2013 and 2015 Model Lists of Essential Medicines. Correlations between numbers of anti-cancer medicines included in national lists and gross national income (GNI), government health expenditure and number of physicians per 1000 population were evaluated. Findings Of the 25 anti-cancer medicines on the 2013 Model List and the 16 added via the 2015 revision of the Model List, 0–25 (median: 17) and 0–15 (median: 3) appeared in national lists, respectively. There was considerable variability in these numbers within and between World Bank income groups. Of the 16 new medicines included in the 2015 Model List, for example, 0–10 (median: 1) and 2–15 (median: 10) were included in the national lists of low-income and high-income countries, respectively. The numbers of these new medicines included in national lists were significantly correlated (P ≤ 0.0001) with per-capita GNI (r = 0.45), per-capita annual government health expenditure (r = 0.33) and number of physicians per 1000 population (r = 0.48). Twenty-one countries (16%) included the targeted anti-cancer medicines imatinib, rituximab and trastuzumab in their national lists. Conclusion Substantial numbers of anti-cancer medicines are included in national lists of low- and middle-income countries but the availability, affordability, accessibility and administration feasibility of these medicines, at country-level, need assessment.

Updating clinical recommendations for breast, colorectal and lung cancer treatments: an opportunity to improve methodology and clinical relevance

BACKGROUND clinical guidelines can improve quality of care summarising available knowledge and proposing recommendations for health care decisions. Being up to date is one of their quality requisites. Little experience is available on when and how guidelines should be updated. We report on the update process of evidence-based clinical recommendations on anticancer drugs. METHODS three multidisciplinary panels, supported by methodology experts, updated the recommendations. The methodologists were in charge of the qualitative and quantitative synthesis of the evidence. The panels were responsible for the final decision about risk/benefit profile of the drugs and strength of the recommendations. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach was used. RESULTS six recommendations out of 15 were completely updated in 8 months time. In four cases, the strength of the recommendation changed; in two of them, we moved from a weak to a strong positive one. Despite the increased certainty about the positive risk/benefit profile, this was translated in a change in the strength of the recommendation only in one case out of three. Three recommendations were refined making them more clinically specific. CONCLUSIONS accumulation of evidence is an opportunity for guideline panels to refine methodological rigour, clinical relevance and to foster consensus on recommendations. This requires time and resource investments.