Dianne O'Connell

Grading quality of evidence and strength of recommendations.

Abstract Users of clinical practice guidelines and other recommendations need to know how much confidence they can place in the recommendations. Systematic and explicit methods of making judgments can reduce errors and improve communication. We have developed a system for grading the quality of evidence and the strength of recommendations that can be applied across a wide range of interventions and contexts. In this article we present a summary of our approach from the perspective of a guideline user. Judgments about the strength of a recommendation require consideration of the balance between benefits and harms, the quality of the evidence, translation of the evidence into specific circumstances, and the certainty of the baseline risk. It is also important to consider costs (resource utilisation) before making a recommendation. Inconsistencies among systems for grading the quality of evidence and the strength of recommendations reduce their potential to facilitate critical appraisal and improve communication of these judgments. Our system for guiding these complex judgments balances the need for simplicity with the need for full and transparent consideration of all important issues. Clinical guidelines are only as good as the evidence and judgments they are based on. The GRADE approach aims to make it easier for users to assess the judgments behind recommendations

Systems for grading the quality of evidence and the strength of recommendations I: Critical appraisal of existing approaches The GRADE Working Group

BackgroundA number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations.MethodsSix prominent systems for grading levels of evidence and strength of recommendations were selected and someone familiar with each system prepared a description of each of these. Twelve assessors independently evaluated each system based on twelve criteria to assess the sensibility of the different approaches. Systems used by 51 organisations were compared with these six approaches.ResultsThere was poor agreement about the sensibility of the six systems. Only one of the systems was suitable for all four types of questions we considered (effectiveness, harm, diagnosis and prognosis). None of the systems was considered usable for all of the target groups we considered (professionals, patients and policy makers). The raters found low reproducibility of judgements made using all six systems. Systems used by 51 organisations that sponsor clinical practice guidelines included a number of minor variations of the six systems that we critically appraised.ConclusionsAll of the currently used approaches to grading levels of evidence and the strength of recommendations have important shortcomings.

Efficacy of tricyclic drugs in treating child and adolescent depression: a meta-analysis

Abstract Objective: To examine whether tricyclic antidepressants are superior to placebo in the treatment of child and adolescent depression. Design: Meta-analysis of 12 randomised controlled trials comparing the efficacy of tricyclic antidepressants with placebo in depressed subjects aged 6-18 years. Main outcome measures: Most studies employed several depression rating scales. For each study the “best available” measure was chosen by using objective criteria, and individual and pooled effect sizes were calculated as the number of standard deviations by which the change scores for the treatment groups exceeded those for the control groups. Where authors had reported numbers “responding” to treatment we calculated individual and pooled ratios for the odds of improvement in treated compared with control subjects. Results: From the six studies presenting data which enabled an estimation of effect size the pooled effect size was 0.35 standard deviations (95% confidence interval of -0.16 to 0.86) indicating no significant benefit of treatment. From the five studies presenting data on the number of “responders” in each group, the ratio of the odds of a response in the treated compared with the control subjects was calculated and the pooled odds ratio was 1.08 (95% confidence interval of 0.53 to 2.17); again indicating no significant benefit of treatment. The pooled sample had more than an 80% chance of detecting a treatment effect of 0.5 standard deviations or greater. There was an inverse relation between study quality and estimated treatment effect. Conclusions: Tricyclic antidepressants appear to be no more effective than placebo in the treatment of depression in children and adolescents. Key messages Key messages Previous studies, and narrative reviews of the topic, have shown that tricyclic antidepressants are of equivocal benefit in juvenile depression * This meta-analysis of 12 randomised double blind placebo controlled trials found an overall small but clinically non-significant treatment effect Tricyclic drugs are not recommended as a first line treatment for depression in children and adolescents

Systems for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system

BackgroundSystems that are used by different organisations to grade the quality of evidence and the strength of recommendations vary. They have different strengths and weaknesses. The GRADE Working Group has developed an approach that addresses key shortcomings in these systems. The aim of this study was to pilot test and further develop the GRADE approach to grading evidence and recommendations.MethodsA GRADE evidence profile consists of two tables: a quality assessment and a summary of findings. Twelve evidence profiles were used in this pilot study. Each evidence profile was made based on information available in a systematic review. Seventeen people were given instructions and independently graded the level of evidence and strength of recommendation for each of the 12 evidence profiles. For each example judgements were collected, summarised and discussed in the group with the aim of improving the proposed grading system. Kappas were calculated as a measure of chance-corrected agreement for the quality of evidence for each outcome for each of the twelve evidence profiles. The seventeen judges were also asked about the ease of understanding and the sensibility of the approach. All of the judgements were recorded and disagreements discussed.ResultsThere was a varied amount of agreement on the quality of evidence for the outcomes relating to each of the twelve questions (kappa coefficients for agreement beyond chance ranged from 0 to 0.82). However, there was fair agreement about the relative importance of each outcome. There was poor agreement about the balance of benefits and harms and recommendations. Most of the disagreements were easily resolved through discussion. In general we found the GRADE approach to be clear, understandable and sensible. Some modifications were made in the approach and it was agreed that more information was needed in the evidence profiles.ConclusionJudgements about evidence and recommendations are complex. Some subjectivity, especially regarding recommendations, is unavoidable. We believe our system for guiding these complex judgements appropriately balances the need for simplicity with the need for full and transparent consideration of all important issues.

Quality of life three years after diagnosis of localised prostate cancer: population based cohort study

Objective To quantify the risk and severity of negative effects of treatment for localised prostate cancer on long term quality of life. Design Population based, prospective cohort study with follow-up over three years. Setting New South Wales, Australia. Participants Men with localised prostate cancer were eligible if aged less than 70 years, diagnosed between October 2000 and October 2002, and notified to the New South Wales central cancer registry. Controls were randomly selected from the New South Wales electoral roll and matched to cases by age and postcode. Main outcome measures General health specific and disease specific function up to three years after diagnosis, according to the 12 item short form health survey and the University of California, Los Angeles prostate cancer index. Results 1642 (64%) cases and 495 (63%) eligible and contacted controls took part in the study. After adjustment for confounders, all active treatment groups had low odds of having better sexual function than controls, in particular men on androgen deprivation therapy (adjusted odds ratio (OR) 0.02, 95% CI 0.01 to 0.07). Men treated surgically reported the worst urinary function (adjusted OR 0.17, 95% CI 0.13 to 0.22). Bowel function was poorest in cases who had external beam radiotherapy (adjusted OR 0.44, 95% CI 0.30 to 0.64). General physical and mental health scores were similar across treatment groups, but poorest in men who had androgen deprivation therapy. Conclusions The various treatments for localised prostate cancer each have persistent effects on quality of life. Sexual dysfunction three years after diagnosis was common in all treatment groups, whereas poor urinary function was less common. Bowel function was most compromised in those who had external beam radiotherapy. Men with prostate cancer and the clinicians who treat them should be aware of the effects of treatment on quality of life, and weigh them up against the patient’s age and the risk of progression of prostate cancer if untreated to make informed decisions about treatment.

Describing treatment effects to patients

OBJECTIVE: To examine the impact of different presentations of equivalent information (framing) on treatment decisions faced by patients.DESIGN: A systematic review of the published literature was conducted. English language publications allocating participants to different frames were retrieved using electronic and bibliographic searches. Two reviewers examined each article for inclusion, and assessed methodological quality. Study characteristics were tabulated and where possible, relative risks (RR; 95% confidence intervals) were calculated to estimate intervention effects.MEASUREMENTS AND MAIN RESULTS: Thirty-seven articles, yielding 40 experimental studies, were included. Studies examined treatment (N=24), immunization (N=5), or health behavior scenarios (N=11). Overall, active treatments were preferred when outcomes were described in terms of relative rather than absolute risk reductions or number needed to treat. Surgery was preferred to other treatments when treatment efficacy was presented in a positive frame (survival) rather than a negative frame (mortality) (relative risk [RR]=1.51, 95% confidence interval [CI], 1.39 to 1.64). Framing effects were less obvious for immunization and health behavior scenarios. Those with little interest in the behavior at baseline were influenced by framing, particularly when information was presented as gains. In studies judged to be of good methodological quality and/or examining actual decisions, the framing effect, although still evident, was less convincing compared to the results of all included studies.CONCLUSIONS: Framing effects varied with the type of scenario, responder characteristics, scenario manipulations, and study quality. When describing treatment effects to patients, expressing the information in more than one way may present a balanced view to patients and enable them to make informed decisions.

The spectrum of human immunodeficiency virus-associated cancers in a South African black population: results from a case-control study, 1995-2004

The effect of the evolving HIV epidemic on cancer has been sparsely documented in Africa. We report results on the risk of cancer associated with HIV‐1 infection using data from an ongoing study. A case–control analysis was used to estimate the relative risk (odds ratio, OR) of cancer types known to be AIDS defining: Kaposis sarcoma (n = 333), non‐Hodgkin lymphoma (NHL, n = 223) and cancers of the cervix (n = 1,586), and 11 cancer types possibly associated with HIV infection: Hodgkin lymphoma (n = 154), cancers of other anogenital organs (n = 157), squamous cell cancer of the skin (SCC, n = 70), oral cavity and pharynx (n = 319), liver (n = 83), stomach (n = 142), leukemia (n = 323), melanoma (n = 53), sarcomas other than Kaposis (n = 93), myeloma (n = 189) and lung cancer (n = 363). The comparison group comprised 3,717 subjects with all other cancer types and 682 subjects with vascular disease. ORs were adjusted for age, sex (except cervical cancer), year of diagnosis, education and number of sexual partners. Significantly increased risks associated with HIV‐1 infection were found for HIV/AIDS associated Kaposis sarcoma (OR = 47.1, 95% CI = 31.9–69.8), NHL (OR = 5.9, 95% CI = 4.3–8.1) and cancer of the cervix (OR = 1.6, 95% CI = 1.3–2.0); Hodgkins disease (OR = 1.6, 95% CI = 1.0–2.7), cancers of anogenital organs other than the cervix (OR = 2.2; 95% CI = 1.4–3.3) and SCC (OR = 2.6, 95% CI = 1.4–4.9) were also significantly increased. No significant associations were found between HIV and any of the other cancers examined. Risks for HIV‐related cancers are consistent with previous studies in Africa, and are lower when compared to those observed in developed countries.

A systematic review of psychosocial interventions for men with prostate cancer and their partners

OBJECTIVE To systematically review interventions aiming to improve adjustment in men with prostate cancer and their partners. METHODS Medline, EMBASE, CINAHL and PsycINFO databases were searched. Inclusion criteria were: randomized controlled trials; relevant to specified clinical questions; included men who had prostate cancer (at least 80% prostate cancer patients or prostate cancer sub-group analysis); published in English between December 1999 and December 2009. Trial quality was assessed. RESULTS 21 studies met inclusion criteria. Trial quality was low; had not improved over the study timeframe; men with advanced disease were not targeted; minority groups were seldom included. Group cognitive-behavioral and psycho-education interventions appear helpful in promoting better psychological adjustment and QOL for men with prostate cancer; coping skills training for patient-spouse dyads improved QOL for partners. CONCLUSION There are limitations in the research on effective ways to improve adjustment for men with prostate cancer of any stage and their partners; and scant research targeting minority groups and the concerns of men with advanced disease. PRACTICE IMPLICATIONS Interventions for men with advanced prostate cancer could usefully target the implications of advancing disease and caregiver burden. There is an urgent need for researchers to focus efforts specifically on such men and their families.

The Effects of Information Framing on the Practices of Physicians

OBJECTIVE: The presentation format of clinical trial results, or the “frame,” may influence perceptions about the worth of a treatment. The extent and consistency of that influence are unclear. We undertook a systematic review of the published literature on the effects of information framing on the practices of physicians.DESIGN: Relevant articles were retrieved using bibliographic and electronic searches. Information was extracted from each in relation to study design, frame type, parameter assessed, assessment scale, clinical setting, intervention, results, and factors modifying the frame effect.MAIN RESULTS: Twelve articles reported randomized trials investigating the effect of framing on doctors’ opinions or intended practices. Methodological shortcomings were numerous. Seven papers investigated the effect of presenting clinical trial results in terms of relative risk reduction, or absolute risk reductions or the number needing to be treated; gain/loss (positive/negative) terms were used in four papers; verbal/numeric terms in one. In simple clinical scenarios, results expressed in relative risk reduction or gain terms were viewed most positively by doctors. Factors that reduced the impact of framing included the risk of causing harm, preexisting prejudices about treatments, the type of decision, the therapeutic yield, clinical experience, and costs. No study investigated the effect of framing on actual clinical practice.CONCLUSIONS: While a framing effect may exist, particularly when results are presented in terms of proportional or absolute measures of gain or loss, it appears highly susceptible to modification, and even neutralization, by other factors that influence doctors’ decision making. Its effects on actual clinical practice are unknown.

Oral or intravenous N-acetylcysteine: which is the treatment of choice for acetaminophen (paracetamol) poisoning?

BACKGROUND The optimal route and duration of administration for N-acetyl-cysteine in the management of acetaminophen (paracetamol) poisoning are controversial. It has been stated on the basis of a selected post-hoc analysis that oral N-acetylcysteine is superior to intravenous N-acetylcysteine in presentations later than 15 hours. AIM OF STUDY To investigate the efficacy of intravenous or oral N-acetylcysteine. PATIENTS AND METHODS We analyzed a series of acetaminophen poisonings treated with a protocol including activated charcoal and intravenous N-acetylcysteine. The outcomes assessed included use of N-acetylcysteine, adverse effects of intravenous N-acetylcysteine, and the occurrence of hepatotoxicity (transaminase > 1000 U/L). We incorporated these results in a meta-analysis of previously reported series of acetaminophen poisonings to compare the outcomes from intravenous and oral N-acetylcysteine use. RESULTS Of 981 patients admitted over 10 years, 4% (40) presented later than 24 hours and 10% (100) had concentrations of acetaminophen that indicated a probable or high risk of hepatotoxicity. The 30 patients who developed hepatotoxicity presented later, took larger amounts, had higher concentrations, and received N-acetylcysteine later than those who did not. No patients received a liver transplant but 2 patients died (one after referral to a transplant unit and one just before). Adverse reactions to intravenous N-acetylcysteine occurred in 6% (12/205) of patients but none prevented completion of the treatment. In the meta-analysis, those with probable or high risk concentrations had similar outcomes with intravenous (pooled n = 341) and oral N-acetylcysteine (pooled n = 1462) administration. Rates of hepatotoxicity for those treated within 10 hours (3 and 6%), late (10-24 hours: 30 and 26%), and overall (0-24 hours: 16 and 19%) were all similar. The proportion of patients classified as presenting later than 10 hours is much greater in the oral N-acetylcysteine studies (64%) than in many of the intravenous N-acetylcysteine studies (38%, 44%, and 63%). CONCLUSIONS The differences claimed between oral and intravenous N-acetylcysteine regimes are probably artifactual and relate to inappropriate subgroup analysis. A shorter hospital stay, patient and doctor convenience, and the concerns over the reduction in bioavailability of oral N-acetylcysteine by charcoal and vomiting make intravenous N-acetylcysteine preferable for most patients with acetaminophen poisoning.