Henry J McQuay

Assessing the quality of reports of randomized clinical trials: Is blinding necessary?

It has been suggested that the quality of clinical trials should be assessed by blinded raters to limit the risk of introducing bias into meta-analyses and systematic reviews, and into the peer-review process. There is very little evidence in the literature to substantiate this. This study describes the development of an instrument to assess the quality of reports of randomized clinical trials (RCTs) in pain research and its use to determine the effect of rater blinding on the assessments of quality. A multidisciplinary panel of six judges produced an initial version of the instrument. Fourteen raters from three different backgrounds assessed the quality of 36 research reports in pain research, selected from three different samples. Seven were allocated randomly to perform the assessments under blind conditions. The final version of the instrument included three items. These items were scored consistently by all the raters regardless of background and could discriminate between reports from the different samples. Blind assessments produced significantly lower and more consistent scores than open assessments. The implications of this finding for systematic reviews, meta-analytic research and the peer-review process are discussed.

The visual analogue pain intensity scale: what is moderate pain in millimetres?

Abstract One way to ensure adequate sensitivity for analgesic trials is to test the intervention on patients who have established pain of moderate to severe intensity. The usual criterion is at least moderate pain on a categorical pain intensity scale. When visual analogue scales (VAS) are the only pain measure in trials we need to know what point on a VAS represents moderate pain, so that these trials can be included in meta‐analysis when baseline pain of at least moderate intensity is an inclusion criterion. To investigate this we used individual patient data from 1080 patients from randomised controlled trials of various analgesics. Baseline pain was measured using a 4‐point categorical pain intensity scale and a pain intensity VAS under identical conditions. The distribution of the VAS scores was examined for 736 patients reporting moderate pain and for 344 reporting severe pain. The VAS scores corresponding to moderate or severe pain were also examined by gender. Baseline VAS scores recorded by patients reporting moderate pain were significantly different from those of patients reporting severe pain. Of the patients reporting moderate pain 85% scored over 30 mm on the corresponding VAS, with a mean score of 49 mm. For those reporting severe pain 85% scored over 54 mm with a mean score of 75 mm. There was no difference between the corresponding VAS scores of men and women. Our results indicate that if a patient records a baseline VAS score in excess of 30 mm they would probably have recorded at least moderate pain on a 4‐point categorical scale.

Opioids in chronic non-cancer pain: systematic review of efficacy and safety

&NA; Opioids are used increasingly for chronic non‐cancer pain. Controversy exists about their effectiveness and safety with long‐term use. We analysed available randomised, placebo‐controlled trials of WHO step 3 opioids for efficacy and safety in chronic non‐cancer pain. The Oxford Pain Relief Database (1950–1994) and Medline, EMBASE and the Cochrane Library were searched until September 2003. Inclusion criteria were randomised comparisons of WHO step 3 opioids with placebo in chronic non‐cancer pain. Double‐blind studies reporting on pain intensity outcomes using validated pain scales were included. Fifteen randomised placebo‐controlled trials were included. Four investigations with 120 patients studied intravenous opioid testing. Eleven studies (1025 patients) compared oral opioids with placebo for four days to eight weeks. Six of the 15 included trials had an open label follow‐up of 6–24 months. The mean decrease in pain intensity in most studies was at least 30% with opioids and was comparable in neuropathic and musculoskeletal pain. About 80% of patients experienced at least one adverse event, with constipation (41%), nausea (32%) and somnolence (29%) being most common. Only 44% of 388 patients on open label treatments were still on opioids after therapy for between 7 and 24 months. The short‐term efficacy of opioids was good in both neuropathic and musculoskeletal pain conditions. However, only a minority of patients in these studies went on to long‐term management with opioids. The small number of selected patients and the short follow‐ups do not allow conclusions concerning problems such as tolerance and addiction.

A systematic review of antidepressants in neuropathic pain

&NA; The objective of this study was to review the effectiveness and safety of antidepressants in neuropathic pain. In a systematic review of randomised controlled trials, the main outcomes were global judgements, pain relief or fall in pain intensity which approximated to more than 50% pain relief, and information about minor and major adverse effects. Dichotomous data for effectiveness and adverse effects were analysed using odds ratio and number needed‐to‐treat (NNT) methods. Twenty‐one placebo‐controlled treatments in 17 randomised controlled trials were included, involving 10 antidepressants. In six of 13 diabetic neuropathy studies the odds ratios showed significant benefit compared with placebo. The combined odds ratio was 3.6 (95% CI 2.5–5.2), with a NNT for benefit of 3 (2.4–4). In two of three postherpetic neuralgia studies the odds ratios showed significant benefit, and the combined odds ratio was 6.8 (3.5–14.3), with a NNT of 2.3 (1.7–3.3). In two atypical facial pain studies the combined odds ratio for benefit was 4.1 (2.3–7.5), with a NNT of 2.8 (2–4.7). Only one of three central pain studies had analysable dichotomous data. The NNT point estimate was 1.7. Comparisons of tricyclic antidepressants did not show any significant difference between them; they were significantly more effective than benzodiazepines in the three comparisons available. Paroxetine and mianserin were less effective than imipramine. For 11 of the 21 placebo‐controlled treatments there was dichotomous information on minor adverse effects; combining across pain syndromes the NNT for minor (noted in published report) adverse effects was 3.7 (2.9–5.2). Information on major (drug‐related study withdrawal) adverse effects was available from 19 reports; combining across pain syndromes the NNT for major adverse effects was 22 (13.5–58). Antidepressants are effective in relieving neuropathic pain. Compared with placebo, of 100 patients with neuropathic pain who are given antidepressants, 30 will obtain more than 50% pain relief, 30 will have minor adverse reactions and four will have to stop treatment because of major adverse effects. With very similar results for anticonvulsants it is still unclear which drug class should be first choice. Treatment would be improved if we could harness the dramatic improvement seen on placebo in some of the trials.

Cannabinoids for control of chemotherapy induced nausea and vomiting: quantitative systematic review.

Abstract Objective: To quantify the antiemetic efficacy and adverse effects of cannabis used for sickness induced by chemotherapy. Design: Systematic review. Data sources: Systematic search (Medline, Embase, Cochrane library, bibliographies), any language, to August 2000. Studies: 30 randomised comparisons of cannabis with placebo or antiemetics from which dichotomous data on efficacy and harm were available (1366 patients). Oral nabilone, oral dronabinol (tetrahydrocannabinol), and intramuscular levonantradol were tested. No cannabis was smoked. Follow up lasted 24 hours. Results: Cannabinoids were more effective antiemetics than prochlorperazine, metoclopramide, chlorpromazine, thiethylperazine, haloperidol, domperidone, or alizapride: relative risk 1.38 (95% confidence interval 1.18 to 1.62), number needed to treat 6 for complete control of nausea; 1.28 (1.08 to 1.51), NNT 8 for complete control of vomiting. Cannabinoids were not more effective in patients receiving very low or very high emetogenic chemotherapy. In crossover trials, patients preferred cannabinoids for future chemotherapy cycles: 2.39 (2.05 to 2.78), NNT 3. Some potentially beneficial side effects occurred more often with cannabinoids: “high” 10.6 (6.86 to 16.5), NNT 3; sedation or drowsiness 1.66 (1.46 to 1.89), NNT 5; euphoria 12.5 (3.00 to 52.1), NNT 7. Harmful side effects also occurred more often with cannabinoids: dizziness 2.97 (2.31 to 3.83), NNT 3; dysphoria or depression 8.06 (3.38 to 19.2), NNT 8; hallucinations 6.10 (2.41 to 15.4), NNT 17; paranoia 8.58 (6.38 to 11.5), NNT 20; and arterial hypotension 2.23 (1.75 to 2.83), NNT 7. Patients given cannabinoids were more likely to withdraw due to side effects 4.67 (3.07 to 7.09), NNT 11. Conclusions: In selected patients, the cannabinoids tested in these trials may be useful as mood enhancing adjuvants for controlling chemotherapy related sickness. Potentially serious adverse effects, even when taken short term orally or intramuscularly, are likely to limit their widespread use. What is already known on this topic Requests have been made for legalisation of cannabis (marijuana) for medical use Long term smoking of cannabis can have physical and neuropsychiatric adverse effects Cannabis may be useful in the control of chemotherapy related sickness What this study adds Oral nabilone and dronabinol and intramuscular levonantradol are superior to conventional antiemetics (such as prochlorperazine or metoclopramide) in chemotherapy Side effects are common with cannabinoids, and although some may be potentially beneficial (euphoria, “high,” sedation), others are harmful (dysphoria, depression, hallucinations) Many patients have a strong preference for cannabinoids

Using Numerical Results from Systematic Reviews in Clinical Practice

As professionals, we want to use the best treatments; as patients, we want to be given them. Knowing whether an intervention works (or does not work) is fundamental to clinical decision making. However, clinical decision making involves more than simply taking published results of research directly to the bedside. Physicians need to consider how similar their patients are to those in the published studies, to take the values and preferences of their patients into account, and to consider their own experience with a given test or treatment. Evidence from clinical research is becoming increasingly important in medical-practice decisions as more and better evidence is published. But when is the evidence strong enough to justify changing a practice? Individual studies that involve only small numbers of patients may have results that are distorted by the random play of chance and thus lead to less than optimal decisions. As is clear from other papers in this series, systematic reviews identify, critically appraise, and review all the relevant studies on a clinical question and are more likely to give a valid answer. They use explicit methods and quality standards to reduce bias. Their results are the closest we can come to reaching the truth given our current state of knowledge. The questions about an intervention that a systematic review should answer are the following: 1. Does it work? 2. If it works, how well does it work in general and compared with placebo, no treatment, or other interventions that are currently in use? 3. Is it safe? 4. Will it be safe and effective for my patients? Whereas the critical appraisal and qualitative synthesis provided by review articles can be interpreted directly, the numerical products of quantitative reviews can be more difficult to understand and apply in daily clinical practice. This paper provides guidance on how to interpret the numerical and statistical results of systematic reviews, translate these results into more understandable terms, and apply them directly to individual patients. Many of these principles can also be used to interpret the numerical results of individual clinical studies. They are particularly relevant to systematic reviews, however, because such reviews contain more information than do primary studies and often exert greater influence than do individual studies. Making Sense of the Numerical Results of Clinical Studies Although the results of clinical studies can be expressed in intuitively meaningful ways, such results do not always easily translate into clinical decision making. For example, results are frequently expressed in terms of risk, which is an expression of the frequency of a given outcome. (Risks are probabilities, which can vary between 0.0 and 1.0. A probability of 0.0 means that the event will never happen, and a probability of 1.0 means that it always happens.) Consider a hypothetical study of the recurrence of migraine headaches in a control group receiving placebo and a treatment group receiving a new antimigraine preparation, drug M (a secondary prevention trial). Suppose that at the end of the trial, migraines recurred in 30% of the control group (the risk for recurrence was 0.30) but in only 5% of the drug M group (risk of 0.05) (Table 1). Table 1. Numerical Expression of Hypothetical Clinical Trial Results The outcomes of the study are clear enough for the two groups when they are examined separately. But clinicians and patients are more interested in the comparative results, that is, the outcome in one group relative to the outcome in the other group. This overall (comparative) result can be expressed in various ways. For example, the relative risk, which is the risk in the treatment group relative to that in the control group, is simply the ratio of the risks in the two groups. In other words, relative risk is the risk in the treatment group divided by that in the control group, 0.05 0.30, or 0.17. The comparison can also be expressed as the reduction in relative risk, which is the ratio between the decrease in risk (in the treatment group) and the risk in the control group, 0.25 0.30, or 0.83 (Table 1). (The relative risk reduction can also be calculated as 1 relative risk). Although the clinical meaning of relative risk (and relative risk reduction) is reasonably clear, relative risk has the distinct disadvantage that a given value (for example, 0.17) is the same whether the risk with treatment decreases from 0.80 to 0.14, from 0.30 to 0.05, from 0.001 to 0.00017, and so forth. The clinical implications of these changes clearly differ from one another enormously and depend on the specific disease and intervention. An important alternate expression of comparative results, therefore, is the absolute risk reduction. Absolute risk reduction is determined by subtracting the risk in one group from the risk in the other (for example, the risk in the treatment group is subtracted from the risk in the placebo group). In the case of our migraine study, the absolute risk reduction would be 0.30 0.05, which equals 0.25, or 25 percentage points. In contrast, for a study in which the risk decreased from 0.001 to 0.00017, the absolute risk reduction would be only 0.00083, or 0.083 percentage points, which is a trivial change in comparison (Table 1). This arithmetic emphasizes the difficulty of expressing the results of clinical studies in meaningful ways. Relative risk and relative risk reduction clearly give a quantitative sense of the effects of an intervention in proportional terms but provide no clue about the size of an effect on an absolute scale. In contrast, although it tells less about proportional effects, absolute risk says a great deal about whether an effect is likely to be clinically meaningful. Despite this benefit, even absolute risk is problematic because it is a dimensionless, abstract number; that is, it lacks a direct connection with the clinical situation in which the patient and physician exist. However, another way of expressing clinical research results can provide that clinical link: the number needed to treat (NNT). Number Needed To Treat The NNT for a given therapy is simply the reciprocal of the absolute risk reduction for that treatment [1, 2]. In the case of our hypothetical migraine study (in which risk decreased from 0.30 without treatment with drug M to 0.05 with treatment with drug M, for a relative risk of 0.17, a relative risk reduction of 0.83, and an absolute risk reduction of 0.25), the NNT would be 1 0.25, or 4. In concrete clinical terms, an NNT of 4 means that you would need to treat four patients with drug M to prevent migraine from recurring in one patient. To emphasize the difference between the concepts embodied in NNT and relative risk, recall the various situations mentioned above, in all of which the relative risk was 0.17 but in which the absolute risk decreased from 0.80 to 0.14 in one case and from 0.001 to 0.00017 in another. Note that the corresponding NNTs in these two other cases are 1.5 and 1204, respectively: that is, you would need to treat 1.5 and 1204 patients to obtain a therapeutic result in these two situations compared with 4 patients with drug M (Table 1). The NNT can be calculated easily and kept as a single numerical reminder of the effectiveness (or, as we will see, the potential for harm) of a particular therapy. As we suggested, the NNT has the crucial advantage of direct applicability to clinical practice because it shows the effort that is required to achieve a particular therapeutic target. The NNT has the additional advantage that it can be applied to any beneficial outcome or any adverse event (when it becomes the number needed to harm [NNH]). The concept of NNT always refers to a comparison group (in which patients receive placebo, no treatment, or some other treatment), a particular treatment outcome, and a defined period of treatment. In other words, the NNT is the number of patients that you will need to treat with drug or treatment A to achieve an improvement in outcome compared with drug or treatment B for a treatment period of C weeks (or other unit of time). To be fully specified, NNT and NNH must always specify the comparator, the therapeutic outcome, and the duration of treatment that is necessary to achieve the outcome. Important Qualities of the Number Needed To Treat The NNT is treatment specific. It describes the difference between treatment and control in achieving a particular clinical outcome. Table 2 shows NNTs from a selection of systematic reviews and large randomized, controlled trials. Table 2. Numbers Needed To Treat from Systematic Reviews and Randomized, Controlled Trials A very small NNT (that is, one that approaches 1) means that a favorable outcome occurs in nearly every patient who receives the treatment and in few patients in a comparison group. Although NNTs close to 1 are theoretically possible, they are almost never found in practice. However, small NNTs do occur in some therapeutic trials, such as those comparing antibiotics with placebo in the eradication of Helicobacter pylori infection or those examining the use of insecticide for head lice (Table 2). An NNT of 2 or 3 indicates that a treatment is quite effective. In contrast, such prophylactic interventions as adding aspirin to streptokinase to reduce 5-week vascular mortality rates after myocardial infarction may have NNTs as high as 20 to 40 and still be considered clinically effective. Limitations of the Number Needed To Treat Although NNTs are powerful instruments for interpreting clinical effects, they also have important limitations. First, an NNT is generally expressed as a single number, which is known as its point estimate. As with all experimental measurements, however, the true value of the NNT can be higher or lower than the point estimate determined through clinical studies. The 95% CIs of the NNT are useful in this regard because they provide an indication that,

Efficacy, dose-response, and safety of ondansetron in prevention of postoperative nausea and vomiting, A quantitative systematic review of randomized placebo-controlled trials

Objective: The authors reviewed efficacy and safety data for ondansetron for preventing postoperative nausea and vomiting (PONV). Methods: Systematically searched, randomized, controlled trials (obtained through MEDLINE, EMBASE, Biological Abstracts, manufacturers database, manual searching of journals, and article reference lists) were analyzed. Relevant end points were prevention of early PONV (within 6 h after surgery) and late PONV (within 48 h) and adverse effects. Relative benefit and number‐needed‐to‐treat were calculated. The number‐needed‐to‐treat indicated how many patients had to be exposed to ondansetron to prevent PONV in one of them who would have vomited or been nauseated had he or she received placebo. Results: Fifty‐three trials were found that had data from 7,177 patients receiving 24 different ondansetron regimens and from 5,712 controls receiving placebo or no treatment. Average early and late PONV incidences without ondansetron were 40% and 60%, respectively. There was a dose response for oral and intravenous ondansetron. Best number‐needed‐to‐treat to prevent PONV with the best documented regimens was between 5 and 6. This was achieved with an intravenous dose of 8 mg and an oral dose of 16 mg. Antivomiting efficacy was consistently better than antinausea efficacy. Efficacy in children was poorly documented. Ondansetron significantly increased the risk for elevated liver enzymes (number‐needed‐to‐harm was 31) and headache (number‐needed‐to‐harm was 36). Conclusions: If the risk of PONV is very high, for every 100 patients receiving an adequate dose of ondansetron 20 patients will not vomit who would have vomited had they received placebo. The antinausea effect is less pronounced. Of these 100, three will have elevated liver enzymes and three will have a headache who would not have had these adverse effects without the drug.

Are cannabinoids an effective and safe treatment option in the management of pain? A qualitative systematic review.

Abstract Objective: To establish whether cannabis is an effective and safe treatment option in the management of pain. Design: Systematic review of randomised controlled trials. Data sources: Electronic databases Medline, Embase, Oxford Pain Database, and Cochrane Library; references from identified papers; hand searches. Study selection: Trials of cannabis given by any route of administration (experimental intervention) with any analgesic or placebo (control intervention) in patients with acute, chronic non-malignant, or cancer pain. Outcomes examined were pain intensity scores, pain relief scores, and adverse effects. Validity of trials was assessed independently with the Oxford score. Data extraction: Independent data extraction; discrepancies resolved by consensus. Data synthesis: 20 randomised controlled trials were identified, 11 of which were excluded. Of the 9 included trials (222 patients), 5 trials related to cancer pain, 2 to chronic non-malignant pain, and 2 to acute postoperative pain. No randomised controlled trials evaluated cannabis; all tested active substances were cannabinoids. Oral delta-9-tetrahydrocannabinol (THC) 5-20 mg, an oral synthetic nitrogen analogue of THC 1 mg, and intramuscular levonantradol 1.5-3 mg were about as effective as codeine 50-120 mg, and oral benzopyranoperidine 2-4 mg was less effective than codeine 60-120 mg and no better than placebo. Adverse effects, most often psychotropic, were common. Conclusion: Cannabinoids are no more effective than codeine in controlling pain and have depressant effects on the central nervous system that limit their use. Their widespread introduction into clinical practice for pain management is therefore undesirable. In acute postoperative pain they should not be used. Before cannabinoids can be considered for treating spasticity and neuropathic pain, further valid randomised controlled studies are needed. What is already known on this topic Three quarters of British doctors surveyed in 1994 wanted cannabis available on prescription Humans have cannabinoid receptors in the central and peripheral nervous system In animal testing cannabinoids are analgesic and reduce signs of neuropathic pain Some evidence exists that cannabinoids may be analgesic in humans What this study adds No studies have been conducted on smoked cannabis Cannabinoids give about the same level of pain relief as codeine in acute postoperative pain They depress the central nervous system

Quantitative estimation of rare adverse events which follow a biological progression: a new model applied to chronic NSAID use.

Abstract Randomised controlled trials (RCTs) alone are unlikely to provide reliable estimates of the incidence of rare events because of their limited size. Cohort, case control, and other observational studies have large numbers but are vulnerable to various kinds of bias. Wanting to estimate the risk of death from bleeding or perforated gastroduodenal ulcers with chronic usage of non‐steroidal anti‐inflammatory drugs (NSAIDs) with greater precision, we developed a model to quantify the frequency of rare adverse events which follow a biological progression. The model combined data from both RCTs and observational studies. We searched systematically for any report of chronic (≥2 months) use of NSAIDs which gave information on gastroduodenal ulcer, bleed or perforation, death due to these complications, or progression from one level of harm to the next. Fifteen RCTs (19 364 patients exposed to NSAIDs for 2–60 months), three cohort studies (215 076 patients redeeming a NSAID prescription over a 3–12 month period), six case‐control studies (2957 cases) and 20 case series (7406), and case reports (4447) were analysed. In RCTs the incidence of bleeding or perforation in 6822 patients exposed to NSAIDs was 0.69%; two deaths occurred. Of 11 040 patients with bleeding or perforation with or without NSAID exposure across all reports, 6–16% (average 12%) died; the risk was lowest in RCTs and highest in case reports. Death from bleeding or perforation in all controls not exposed to NSAIDs occurred in 18 out of 849 489 (0.002%). From these numbers we calculated the number‐needed‐to‐treat for one patient to die due to gastroduodenal complications with chronic (≥2 months) NSAIDs as 1/((0.69×{6–16%, average 12%})−0.002%))=909–2500 (average 1220). On average 1 in 1200 patients taking NSAIDs for at least 2 months will die from gastroduodenal complications who would not have died had they not taken NSAIDs. This extrapolates to about 2000 deaths each year in the UK.

Opioids in pain management.

Opioids are our most powerful analgesics, but politics, prejudice, and our continuing ignorance still impede optimum prescribing. Just over 100 years ago, opium poppies were still grown on the Cambridgeshire fens in the UK to provide oblivion for the working man and his family, but the brewing lobby argued on thin evidence that their potions were less dangerous. The restriction of opioid availability to protect society and the individual continues in many countries. In this review I focus on chronic and cancer pain, but many of the principles apply in acute pain. The justification for this focus is that patients with chronic pain may suffer longer and unnecessarily if we prescribe and legislate badly.