Paula Williamson

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias

Background The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. Methodology/Principal Findings We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40–62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. Conclusions Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews

Objective To examine the prevalence of outcome reporting bias—the selection for publication of a subset of the original recorded outcome variables on the basis of the results—and its impact on Cochrane reviews. Design A nine point classification system for missing outcome data in randomised trials was developed and applied to the trials assessed in a large, unselected cohort of Cochrane systematic reviews. Researchers who conducted the trials were contacted and the reason sought for the non-reporting of data. A sensitivity analysis was undertaken to assess the impact of outcome reporting bias on reviews that included a single meta-analysis of the review primary outcome. Results More than half (157/283 (55%)) the reviews did not include full data for the review primary outcome of interest from all eligible trials. The median amount of review outcome data missing for any reason was 10%, whereas 50% or more of the potential data were missing in 70 (25%) reviews. It was clear from the publications for 155 (6%) of the 2486 assessable trials that the researchers had measured and analysed the review primary outcome but did not report or only partially reported the results. For reports that did not mention the review primary outcome, our classification regarding the presence of outcome reporting bias was shown to have a sensitivity of 88% (95% CI 65% to 100%) and specificity of 80% (95% CI 69% to 90%) on the basis of responses from 62 trialists. A third of Cochrane reviews (96/283 (34%)) contained at least one trial with high suspicion of outcome reporting bias for the review primary outcome. In a sensitivity analysis undertaken for 81 reviews with a single meta-analysis of the primary outcome of interest, the treatment effect estimate was reduced by 20% or more in 19 (23%). Of the 42 meta-analyses with a statistically significant result only, eight (19%) became non-significant after adjustment for outcome reporting bias and 11 (26%) would have overestimated the treatment effect by 20% or more. Conclusions Outcome reporting bias is an under-recognised problem that affects the conclusions in a substantial proportion of Cochrane reviews. Individuals conducting systematic reviews need to address explicitly the issue of missing outcome data for their review to be considered a reliable source of evidence. Extra care is required during data extraction, reviewers should identify when a trial reports that an outcome was measured but no results were reported or events observed, and contact with trialists should be encouraged.

The SANAD study of effectiveness of carbamazepine, gabapentin, lamotrigine, oxcarbazepine, or topiramate for treatment of partial epilepsy: an unblinded randomised controlled trial

BACKGROUND Carbamazepine is widely accepted as a drug of first choice for patients with partial onset seizures. Several newer drugs possess efficacy against these seizure types but previous randomised controlled trials have failed to inform a choice between these drugs. We aimed to assess efficacy with regards to longer-term outcomes, quality of life, and health economic outcomes. METHODS SANAD was an unblinded randomised controlled trial in hospital-based outpatient clinics in the UK. Arm A recruited 1721 patients for whom carbamazepine was deemed to be standard treatment, and they were randomly assigned to receive carbamazepine, gabapentin, lamotrigine, oxcarbazepine, or topiramate. Primary outcomes were time to treatment failure, and time to 12-months remission, and assessment was by both intention to treat and per protocol. This study is registered as an International Standard Randomised Controlled Trial, number ISRCTN38354748. FINDINGS For time to treatment failure, lamotrigine was significantly better than carbamazepine (hazard ratio [HR] 0.78 [95% CI 0.63-0.97]), gabapentin (0.65 [0.52-0.80]), and topiramate (0.64 [0.52-0.79]), and had a non-significant advantage compared with oxcarbazepine (1.15 [0.86-1.54]). For time to 12-month remission carbamazepine was significantly better than gabapentin (0.75 [0.63-0.90]), and estimates suggest a non-significant advantage for carbamazepine against lamotrigine (0.91 [0.77-1.09]), topiramate (0.86 [0.72-1.03]), and oxcarbazepine (0.92 [0.73-1.18]). In a per-protocol analysis, at 2 and 4 years the difference (95% CI) in the proportion achieving a 12-month remission (lamotrigine-carbamazepine) is 0 (-8 to 7) and 5 (-3 to 12), suggesting non-inferiority of lamotrigine compared with carbamazepine. INTERPRETATION Lamotrigine is clinically better than carbamazepine, the standard drug treatment, for time to treatment failure outcomes and is therefore a cost-effective alternative for patients diagnosed with partial onset seizures.

The SANAD study of effectiveness of valproate, lamotrigine, or topiramate for generalised and unclassifiable epilepsy: an unblinded randomised controlled trial.

BACKGROUND Valproate is widely accepted as a drug of first choice for patients with generalised onset seizures, and its broad spectrum of efficacy means it is recommended for patients with seizures that are difficult to classify. Lamotrigine and topiramate are also thought to possess broad spectrum activity. The SANAD study aimed to compare the longer-term effects of these drugs in patients with generalised onset seizures or seizures that are difficult to classify. METHODS SANAD was an unblinded randomised controlled trial in hospital-based outpatient clinics in the UK. Arm B of the study recruited 716 patients for whom valproate was considered to be standard treatment. Patients were randomly assigned to valproate, lamotrigine, or topiramate between Jan 12, 1999, and Aug 31, 2004, and follow-up data were obtained up to Jan 13, 2006. Primary outcomes were time to treatment failure, and time to 1-year remission, and analysis was by both intention to treat and per protocol. This study is registered as an International Standard Randomised Controlled Trial, number ISRCTN38354748. FINDINGS For time to treatment failure, valproate was significantly better than topiramate (hazard ratio 1.57 [95% CI 1.19-2.08]), but there was no significant difference between valproate and lamotrigine (1.25 [0.94-1.68]). For patients with an idiopathic generalised epilepsy, valproate was significantly better than both lamotrigine (1.55 [1.07-2.24] and topiramate (1.89 [1.32-2.70]). For time to 12-month remission valproate was significantly better than lamotrigine overall (0.76 [0.62-0.94]), and for the subgroup with an idiopathic generalised epilepsy 0.68 (0.53-0.89). But there was no significant difference between valproate and topiramate in either the analysis overall or for the subgroup with an idiopathic generalised epilepsy. INTERPRETATION Valproate is better tolerated than topiramate and more efficacious than lamotrigine, and should remain the drug of first choice for many patients with generalised and unclassified epilepsies. However, because of known potential adverse effects of valproate during pregnancy, the benefits for seizure control in women of childbearing years should be considered.

Developing core outcome sets for clinical trials: issues to consider

The selection of appropriate outcomes or domains is crucial when designing clinical trials in order to compare directly the effects of different interventions in ways that minimize bias. If the findings are to influence policy and practice then the chosen outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. There is a growing recognition that insufficient attention has been paid to the outcomes measured in clinical trials. These issues could be addressed through the development and use of an agreed standardized collection of outcomes, known as a core outcome set, which should be measured and reported, as a minimum, in all trials for a specific clinical area. Accumulating work in this area has identified the need for general guidance on the development of core outcome sets. Key issues to consider in the development of a core outcome set include its scope, the stakeholder groups to involve, choice of consensus method and the achievement of a consensus.

A Randomized Trial of Genotype-Guided Dosing of Warfarin

BACKGROUND The level of anticoagulation in response to a fixed-dose regimen of warfarin is difficult to predict during the initiation of therapy. We prospectively compared the effect of genotype-guided dosing with that of standard dosing on anticoagulation control in patients starting warfarin therapy. METHODS We conducted a multicenter, randomized, controlled trial involving patients with atrial fibrillation or venous thromboembolism. Genotyping for CYP2C9*2, CYP2C9*3, and VKORC1 (-1639G→A) was performed with the use of a point-of-care test. For patients assigned to the genotype-guided group, warfarin doses were prescribed according to pharmacogenetic-based algorithms for the first 5 days. Patients in the control (standard dosing) group received a 3-day loading-dose regimen. After the initiation period, the treatment of all patients was managed according to routine clinical practice. The primary outcome measure was the percentage of time in the therapeutic range of 2.0 to 3.0 for the international normalized ratio (INR) during the first 12 weeks after warfarin initiation. RESULTS A total of 455 patients were recruited, with 227 randomly assigned to the genotype-guided group and 228 assigned to the control group. The mean percentage of time in the therapeutic range was 67.4% in the genotype-guided group as compared with 60.3% in the control group (adjusted difference, 7.0 percentage points; 95% confidence interval, 3.3 to 10.6; P<0.001). There were significantly fewer incidences of excessive anticoagulation (INR ≥4.0) in the genotype-guided group. The median time to reach a therapeutic INR was 21 days in the genotype-guided group as compared with 29 days in the control group (P<0.001). CONCLUSIONS Pharmacogenetic-based dosing was associated with a higher percentage of time in the therapeutic INR range than was standard dosing during the initiation of warfarin therapy. (Funded by the European Commission Seventh Framework Programme and others; ClinicalTrials.gov number, NCT01119300.).

Adverse Drug Reactions in Hospital In-Patients: A Prospective Analysis of 3695 Patient-Episodes

Adverse drug reactions (ADRs) are a major cause of hospital admissions, but recent data on the incidence and clinical characteristics of ADRs which occur following hospital admission, are lacking. Patients admitted to twelve wards over a six-month period in 2005 were assessed for ADRs throughout their admission. Suspected ADRs were recorded and analysed for causality, severity and avoidability and whether they increased the length of stay. Multivariable analysis was undertaken to identify the risk factors for ADRs. The 5% significance level was used when assessing factors for inclusion in multivariable models. Out of the 3695 patient episodes assessed for ADRs, 545 (14.7%, 95% CI 13.6–15.9%) experienced one or more ADRs. Half of ADRs were definitely or possibly avoidable. The patients experiencing ADRs were more likely to be older, female, taking a larger number of medicines, and had a longer length of stay than those without ADRs. However, the only significant predictor of ADRs, from the multivariable analysis of a representative sample of patients, was the number of medicines taken by the patient with each additional medication multiplying the hazard of an ADR episode by 1.14 (95% CI 1.09, 1.20). ADRs directly increased length of stay in 147 (26.8%) patients. The drugs most frequently associated with ADRs were diuretics, opioid analgesics, and anticoagulants. In conclusion, approximately one in seven hospital in-patients experience an ADR, which is a significant cause of morbidity, increasing the length of stay of patients by an average of 0.25 days/patient admission episode. The overall burden of ADRs on hospitals is high, and effective intervention strategies are urgently needed to reduce this burden.

Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies.

Ian Sinha and colleagues advise that when using the Delphi process to develop core outcome sets for clinical trials, patients and clinicians be involved, researchers and facilitators avoid imposing their views on participants, and attrition of participants be minimized.

Expression of the uptake drug transporter hOCT1 is an important clinical determinant of the response to imatinib in chronic myeloid leukemia.

Some chronic myeloid leukemia (CML) patients do not respond to imatinib, whereas others lose an initial response. To identify potential imatinib failures, we investigated the expression of imatinib uptake transporter (human organic cation transporter 1; hOCT1) and efflux transporters (ATP‐binding cassette transporters ABCB1, ABCG2, and ABCC1) using real‐time quantitative reverse transcription‐polymerase chain reaction in 70 CML patients. Patients with high pretreatment hOCT1 expression had superior complete cytogenetic response (CCR) rates (P=0.008), progression‐free and overall survival (P=0.01 and 0.004). Pretreatment ABCB1, ABCG2, and ABCC1 levels did not correlate with treatment outcome. Regression analysis demonstrated that pretreatment hOCT1 expression was the most powerful predictor of CCR achievement at 6 months (P=0.002). Imatinib uptake into a CML cell line with high hOCT1 expression was greater than into those with modest or low expression (P=0.002). The expression of hOCT1, but not efflux transporters, is important in determining the clinical response to imatinib.

Cost of illness of inflammatory bowel disease in the UK: a single centre retrospective study

Background and aims: The potentially high costs of care associated with inflammatory bowel disease (IBD) are recognised but we have little knowledge of the scale, profile, or determinants of these costs in the UK. This study aimed to describe costs of illness for a group of IBD patients and determine factors associated with increased healthcare costs. Setting: A university hospital serving a target population of approximately 330 000. Patients and methods: A six month cohort of IBD patients receiving any form of secondary care was identified, comprising 307 cases of ulcerative (or indeterminate) colitis and 172 cases of Crohn’s disease. Demographic and clinical data were abstracted from clinical records and individual resource use was itemised for all attributable costs (including extraintestinal manifestations). Item costs were derived from national and local sources. Cost data were expressed as mean six month costs per patient (with 95% confidence interval (CI)) obtained using non-parametric bootstrapping. Determinants of cost were analysed using generalised linear regression modelling. A postal survey of patients was undertaken to examine indirect costs, out of pocket expenses, and primary care visits. Results: Inpatient services (medical and/or surgical) were required by 67 patients (14%) but accounted for 49% of total secondary care costs. Drug costs accounted for less than a quarter of total costs. Individual patient costs ranged from £73 to £33 254 per six months. Mean (95% CI) six month costs per patient were £1256 (£988, £1721) for colitis and £1652 (£1221, £2239) for Crohn’s disease. Hospitalisation, disease severity grade, and disease extent correlated positively with cost of illness but costs were independent of age or sex. Compared with quiescent cases of IBD, disease relapse was associated with a 2–3-fold increase in costs for non-hospitalised cases and a 20-fold increase in costs for hospitalised cases. Survey data suggested average six month costs were <£30 per patient for primary care visits (both diseases) and median loss of earnings were £239 for colitis and £299 for Crohn’s disease. Conclusions: This study represents the first detailed characterisation of the scale and determinants of costs of illness for IBD in a British hospital. Hospitalisation affected a minority of sufferers but accounted for half of the total direct costs falling on the healthcare system.