Susan Dutton

Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes.

CONTEXTnPrevious studies indicate that the interpretation of trial results can be distorted by authors of published reports.nnnOBJECTIVEnTo identify the nature and frequency of distorted presentation or spin (ie, specific reporting strategies, whatever their motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results) in published reports of randomized controlled trials (RCTs) with statistically nonsignificant results for primary outcomes.nnnDATA SOURCESnMarch 2007 search of MEDLINE via PubMed using the Cochrane Highly Sensitive Search Strategy to identify reports of RCTs published in December 2006.nnnSTUDY SELECTIONnArticles were included if they were parallel-group RCTs with a clearly identified primary outcome showing statistically nonsignificant results (ie, P > or = .05).nnnDATA EXTRACTIONnTwo readers appraised each selected article using a pretested, standardized data abstraction form developed in a pilot test.nnnRESULTSnFrom the 616 published reports of RCTs examined, 72 were eligible and appraised. The title was reported with spin in 13 articles (18.0%; 95% confidence interval [CI], 10.0%-28.9%). Spin was identified in the Results and Conclusions sections of the abstracts of 27 (37.5%; 95% CI, 26.4%-49.7%) and 42 (58.3%; 95% CI, 46.1%-69.8%) reports, respectively, with the conclusions of 17 (23.6%; 95% CI, 14.4%-35.1%) focusing only on treatment effectiveness. Spin was identified in the main-text Results, Discussion, and Conclusions sections of 21 (29.2%; 95% CI, 19.0%-41.1%), 31 (43.1%; 95% CI, 31.4%-55.3%), and 36 (50.0%; 95% CI, 38.0%-62.0%) reports, respectively. More than 40% of the reports had spin in at least 2 of these sections in the main text.nnnCONCLUSIONnIn this representative sample of RCTs published in 2006 with statistically nonsignificant primary outcomes, the reporting and interpretation of findings was frequently inconsistent with the results.

The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed.

Objectives To examine the reporting characteristics and methodological details of randomised trials indexed in PubMed in 2000 and 2006 and assess whether the quality of reporting has improved after publication of the Consolidated Standards of Reporting Trials (CONSORT) Statement in 2001. Design Comparison of two cross sectional investigations. Study sample All primary reports of randomised trials indexed in PubMed in December 2000 (n=519) and December 2006 (n=616), including parallel group, crossover, cluster, factorial, and split body study designs. Main outcome measures The proportion of general and methodological items reported, stratified by year and study design. Risk ratios with 95% confidence intervals were calculated to represent changes in reporting between 2000 and 2006. Results The majority of trials were two arm (379/519 (73%) in 2000 v 468/616 (76%) in 2006) parallel group studies (383/519 (74%) v 477/616 (78%)) published in specialty journals (482/519 (93%) v 555/616 (90%)). In both 2000 and 2006, a median of 80 participants were recruited per trial for parallel group trials. The proportion of articles that reported drug trials decreased between 2000 and 2006 (from 393/519 (76%) to 356/616 (58%)), whereas the proportion of surgery trials increased (51/519 (10%) v 128/616 (21%)). There was an increase between 2000 and 2006 in the proportion of trial reports that included details of the primary outcome (risk ratio (RR) 1.18, 95% CI 1.04 to 1.33), sample size calculation (RR 1.66, 95% CI 1.40 to 1.95), and the methods of random sequence generation (RR 1.62, 95% CI 1.32 to 1.97) and allocation concealment (RR 1.40, 95% CI 1.11 to 1.76). There was no difference in the proportion of trials that provided specific details on who was blinded (RR 0.91, 95% CI 0.75 to 1.10). Conclusions Reporting of several important aspects of trial methods improved between 2000 and 2006; however, the quality of reporting remains well below an acceptable level. Without complete and transparent reporting of how a trial was designed and conducted, it is difficult for readers to assess its conduct and validity.

External validation of multivariable prediction models: a systematic review of methodological conduct and reporting

BackgroundBefore considering whether to use a multivariable (diagnostic or prognostic) prediction model, it is essential that its performance be evaluated in data that were not used to develop the model (referred to as external validation). We critically appraised the methodological conduct and reporting of external validation studies of multivariable prediction models.MethodsWe conducted a systematic review of articles describing some form of external validation of one or more multivariable prediction models indexed in PubMed core clinical journals published in 2010. Study data were extracted in duplicate on design, sample size, handling of missing data, reference to the original study developing the prediction models and predictive performance measures.Results11,826 articles were identified and 78 were included for full review, which described the evaluation of 120 prediction models. in participant data that were not used to develop the model. Thirty-three articles described both the development of a prediction model and an evaluation of its performance on a separate dataset, and 45 articles described only the evaluation of an existing published prediction model on another dataset. Fifty-seven percent of the prediction models were presented and evaluated as simplified scoring systems. Sixteen percent of articles failed to report the number of outcome events in the validation datasets. Fifty-four percent of studies made no explicit mention of missing data. Sixty-seven percent did not report evaluating model calibration whilst most studies evaluated model discrimination. It was often unclear whether the reported performance measures were for the full regression model or for the simplified models.ConclusionsThe vast majority of studies describing some form of external validation of a multivariable prediction model were poorly reported with key details frequently not presented. The validation studies were characterised by poor design, inappropriate handling and acknowledgement of missing data and one of the most key performance measures of prediction models i.e. calibration often omitted from the publication. It may therefore not be surprising that an overwhelming majority of developed prediction models are not used in practice, when there is a dearth of well-conducted and clearly reported (external validation) studies describing their performance on independent participant data.

Reporting methods in studies developing prognostic models in cancer: a review

BackgroundDevelopment of prognostic models enables identification of variables that are influential in predicting patient outcome and the use of these multiple risk factors in a systematic, reproducible way according to evidence based methods. The reliability of models depends on informed use of statistical methods, in combination with prior knowledge of disease. We reviewed published articles to assess reporting and methods used to develop new prognostic models in cancer.MethodsWe developed a systematic search string and identified articles from PubMed. Forty-seven articles were included that satisfied the following inclusion criteria: published in 2005; aiming to predict patient outcome; presenting new prognostic models in cancer with outcome time to an event and including a combination of at least two separate variables; and analysing data using multivariable analysis suitable for time to event data.ResultsIn 47 studies, prospective cohort or randomised controlled trial data were used for model development in only 33% (15) of studies. In 30% (14) of the studies insufficient data were available, having fewer than 10 events per variable (EPV) used in model development. EPV could not be calculated in a further 40% (19) of the studies. The coding of candidate variables was only reported in 68% (32) of the studies. Although use of continuous variables was reported in all studies, only one article reported using recommended methods of retaining all these variables as continuous without categorisation. Statistical methods for selection of variables in the multivariate modelling were often flawed. A method that is not recommended, namely, using statistical significance in univariate analysis as a pre-screening test to select variables for inclusion in the multivariate model, was applied in 48% (21) of the studies.ConclusionsWe found that published prognostic models are often characterised by both use of inappropriate methods for development of multivariable models and poor reporting. In addition, models are limited by the lack of studies based on prospective data of sufficient sample size to avoid overfitting. The use of poor methods compromises the reliability of prognostic models developed to provide objective probability estimates to complement clinical intuition of the physician and guidelines.

The proportion of tumor-stroma as a strong prognosticator for stage II and III colon cancer patients: validation in the VICTOR trial

BACKGROUNDnThe intra-tumor stroma percentage in colon cancer (CC) patients has previously been reported by our group as a strong independent prognostic parameter. Patients with a high stroma percentage within the primary tumor have a poor prognosis.nnnPATIENTS AND METHODSnTissue samples from the most invasive part of the primary tumor of 710 patients (52% Stage II, 48% Stage III) participating in the VICTOR trial were analyzed for their tumor-stroma percentage. Stroma-high (>50%) and stroma-low (≤50%) groups were evaluated with respect to survival times.nnnRESULTSnOverall and disease-free survival times (OS and DFS) were significantly lower in the stroma-high group (OS P<0.0001, hazard ratio (HR)=1.96; DFS P<0.0001, HR=2.15). The 5-year OS was 69.0% versus 83.4% and DFS 58.6% versus 77.3% for stroma-high versus stroma-low patients.nnnCONCLUSIONnThis study confirms the intra-tumor stroma ratio as a prognostic factor. This parameter could be a valuable and low cost addition to the TNM status and next to current high-risk parameters such as microsatellite instability status used in routine pathology reporting. When adding the stroma-parameter to the ASCO criteria, the rate of undertreated patients dropped from 5.9% to 4.3%, the overtreated increased with 6.8% but the correctly classified increased with an additional 14%.

Reporting performance of prognostic models in cancer: a review.

BackgroundAppropriate choice and use of prognostic models in clinical practice require the use of good methods for both model development, and for developing prognostic indices and risk groups from the models. In order to assess reliability and generalizability for use, models need to have been validated and measures of model performance reported. We reviewed published articles to assess the methods and reporting used to develop and evaluate performance of prognostic indices and risk groups from prognostic models.MethodsWe developed a systematic search string and identified articles from PubMed. Forty-seven articles were included that satisfied the following inclusion criteria: published in 2005; aiming to predict patient outcome; presenting new prognostic models in cancer with outcome time to an event and including a combination of at least two separate variables; and analysing data using multivariable analysis suitable for time to event data.ResultsIn 47 studies, Cox models were used in 94% (44), but the coefficients or hazard ratios for the variables in the final model were reported in only 72% (34). The reproducibility of the derived model was assessed in only 11% (5) of the articles. A prognostic index was developed from the model in 81% (38) of the articles, but researchers derived the prognostic index from the final prognostic model in only 34% (13) of the studies; different coefficients or variables from those in the final model were used in 50% (19) of models and the methods used were unclear in 16% (6) of the articles. Methods used to derive prognostic groups were also poor, with researchers not reporting the methods used in 39% (14 of 36) of the studies and data derived methods likely to bias estimates of differences between risk groups being used in 28% (10) of the studies. Validation of their models was reported in only 34% (16) of the studies. In 15 studies validation used data from the same population and in five studies from a different population. Including reports of validation with external data from publications up to four years following model development, external validation was attempted for only 21% (10) of models. Insufficient information was provided on the performance of models in terms of discrimination and calibration.ConclusionsMany published prognostic models have been developed using poor methods and many with poor reporting, both of which compromise the reliability and clinical relevance of models, prognostic indices and risk groups derived from them.

Gefitinib for oesophageal cancer progressing after chemotherapy (COG): a phase 3, multicentre, double-blind, placebo-controlled randomised trial

BACKGROUNDnEvidence is scarce for the effectiveness of therapies for oesophageal cancer progressing after chemotherapy, and no randomised trials have been reported. We aimed to compare gefitinib with placebo in previously treated advanced oesophageal cancer.nnnMETHODSnFor this phase 3, parallel, randomised, placebo-controlled trial, eligible patients were adults with advanced oesophageal cancer or type I/II Siewert junctional tumours, histologically confirmed squamous-cell carcinoma or adenocarcinoma, who had progressed after chemotherapy, with WHO performance status 0-2, and with measurable or evaluable disease on CT scan. Participants were recruited from 48 UK centres and randomly assigned (1:1) to gefitinib (500 mg) or matching placebo by simple randomisation with no stratification factors. Patients, clinicians, and trial office staff were masked to treatment allocation. Treatment continued until disease progression, unacceptable toxicity, or patient choice. The primary outcome was overall survival, analysed by intention to treat. This trial is registered, number ISRCTN29580179.nnnFINDINGSnBetween March 30, 2009, and Nov 18, 2011, 450 patients were randomly assigned to treatment groups (one patient withdrew consent; 224 patients allocated gefitinib and 225 allocated placebo included in analyses). Overall survival did not differ between groups (median 3·73 months, 95% CI 3·23-4·50, for gefitinib vs 3·67 months, 95% CI 2·97-4·37, for placebo; hazard ratio [HR] 0·90, 95% CI 0·74-1·09, p=0·29). Among the prespecified patient-reported outcomes (110 patients on gefitinib and 121 on placebo completed both baseline and 4 week questionnaires and were included in analyses), odynophagia was significantly better in the gefitinib group (adjusted mean difference -8·61, 95% CI -14·49 to -2·73; n=227; p=0·004), whereas the other outcomes were not significantly improved compared with placebo: global quality of life (2·69, 95% CI -2·33 to 7·72, n=231, p=0·293), dysphagia (-3·18, 95% CI -8·36 to 2·00, n=231, p=0·228), and eating (-4·11, 95% CI -9·96 to 1·75, n=229, p=0·168). Median progression-free survival was marginally longer with gefitinib than it was with placebo (1·57 months, 95% CI 1·23-1·90 in the gefitinib group vs 1·17 months, 95% CI 1·07-1·37 in the placebo group; HR 0·80, 95% CI 0·66-0·96, p=0·020). The most common toxicities were diarrhoea (36 [16%] of 224 patients on gefitinib vs six [3%] of 225 on placebo) and skin toxicity (46 [21%] vs two [1%]), both mostly grade 2. The commonest grade 3-4 toxicities were fatigue (24 [11%] vs 13 [6%] patients) and diarrhoea (13 [6%] vs two [1%]). Serious adverse events were reported in 109 (49%) of 224 patients assigned to gefitinib and 101 (45%) of 225 on placebo. 54 (24%) of patients in the gefitinib group achieved disease control at 8 weeks, as did 35 (16%) of patients on placebo (p=0·023).nnnINTERPRETATIONnThe use of gefitinib as a second-line treatment in oesophageal cancer in unselected patients does not improve overall survival, but has palliative benefits in a subgroup of these difficult-to-treat patients with short life expectancy. Future research should focus on identification of predictive biomarkers to identify this subgroup of benefiting patients.nnnFUNDINGnCancer Research UK.

Multicenter Randomized Controlled Trial of Conventional Versus Laparoscopic Surgery for Colorectal Cancer Within an Enhanced Recovery Programme: EnROL

PURPOSEnLaparoscopic resection and a multimodal approach known as an enhanced recovery program (ERP) have been major changes in colorectal perioperative care that have improved clinical outcomes for colorectal cancer resection. EnROL (Enhanced Recovery Open Versus Laparoscopic) is a multicenter randomized controlled trial examining whether the benefits of laparoscopy still exist when open surgery is optimized within an ERP.nnnPATIENTS AND METHODSnAdults with colorectal cancer suitable for elective resection were randomly assigned at a ratio of 1:1 to laparoscopic or open surgery within an ERP, stratified by center, cancer site (colon v rectum), and age group (<66 v 66-75 v >75 years) using minimization. The primary outcome was physical fatigue at 1 month postsurgery. Secondary outcomes included hospital stay, complications, other patient-reported outcomes (PROs), and physical function. Patients and outcome assessors were blinded until 7 days postsurgery or discharge if earlier. Central independent and blinded pathologic assessment of surgical quality was undertaken.nnnRESULTSnA total of 204 patients (laparoscopy, n=103; open surgery, n=101) were recruited from 12 UK centers from July 2008 to April 2012. One-month physical fatigue scores were similar in both groups (mean: laparoscopy, 12.28; 95% CI, 11.37 to 13.19 v open surgery, 12.05; 95% CI, 11.14 to 12.96; adjusted mean difference, -0.23; 95% CI, -1.52 to 1.07). Median total hospital stay was significantly shorter after laparoscopic surgery (median: laparoscopy, 5; interquartile range [IQR], 4 to 9 v open surgery, 7; IQR, 5 to 11 days; P=.033). There were no differences in other secondary outcomes or in specimen quality after central pathologic review.nnnCONCLUSIONnIn patients treated by experienced surgeons within an ERP, physical fatigue and other PROs were similar in both groups, but laparoscopic surgery significantly reduced length of hospital stay.

Distribution of childhood leukaemias and non-Hodgkin's lymphomas near nuclear installations in England and Wales

Abstract Objective : To examine the relation between the risk of childhood leukaemia and non-Hodgkins lymphoma and proximity of residence to nuclear installations in England and Wales. Design : Observed and expected numbers of cases were calculated and analysed by standard methods based on ratios of observed to expected counts and by a new statistical test, the linear risk score test, based on ranks and designed to be sensitive to excess incidence in close proximity to a putative source of risk. Setting : Electoral wards within 25 km of 23 nuclear installations and six control sites that had been investigated for suitability for generating stations but never used. Subjects : Children below age 15 in England and Wales, 1966-87. Main outcome measure - Registration of any leukaemia or non-Hodgkins lymphoma. Results : In none of the 25 km circles around the installations was the incidence ratio significantly greater than 1.0. The only significant results for the linear risk score test were for Sellafield (P=0.00002) and Burghfield (P=0.031). The circles for Aldermaston and Burghfield overlap; the incidence ratio was 1.10 in each. One of the control sites gave a significant linear risk score test result (P=0.020). All the tests carried out were one sided with P values estimated by simulation. Conclusion : There is no evidence of a general increase of childhood leukaemia or non-Hodgkins lymphoma around nuclear installations. Apart from Sellafield, the evidence for distance related risk is very weak.

Modafinil for the Treatment of Fatigue in Lung Cancer: Results of a Placebo-Controlled, Double-Blind, Randomized Trial

PURPOSEnFatigue is a distressing symptom occurring in more than 60% of patients with cancer. The CNS stimulants modafinil and methylphenidate are recommended for the treatment of cancer-related fatigue, despite a limited evidence base. We aimed to evaluate the efficacy and tolerability of modafinil in the management of fatigue in patients with non-small-cell lung cancer (NSCLC).nnnPATIENTS AND METHODSnAdults with advanced NSCLC and performance status of 0 to 2, who were not treated with chemotherapy or radiotherapy within the last 4 weeks, were randomly assigned to daily modafinil (100 mg on days 1 to 14; 200 mg on days 15 to 28) or matched placebo. The primary outcome was change in Functional Assessment of Chronic Illness Therapy (FACIT) -Fatigue score from baseline to 28 days, adjusted for baseline fatigue and performance status. Secondary outcomes included safety and patient-reported measures of depression, daytime sleepiness, and quality of life.nnnRESULTSnA total of 208 patients were randomly assigned, and 160 patients (modafinil, n = 75; placebo, n = 85) completed questionnaires at both baseline and day 28 and were included in the modified intention-to-treat analysis. FACIT-Fatigue scores improved from baseline to day 28 (mean score change: modafinil, 5.29; 95% CI, 2.57 to 8.02; placebo, 5.09; 95% CI, 2.54 to 7.65), but there was no difference between treatments (0.20; 95% CI, -3.56 to 3.97). There was also no difference between treatments for the secondary outcomes; 47% of the modafinil group and 23% of the placebo group stated that the intervention was not helpful.nnnCONCLUSIONnModafinil had no effect on cancer-related fatigue and should not be prescribed outside a clinical trial setting. Its use was associated with a clinically significant placebo effect.