Shiyuan Zhang

A tutorial on sensitivity analyses in clinical trials: the what, why, when and how

BackgroundSensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations—such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers—on the overall conclusions of a study.The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.DiscussionIn this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.SummaryWhen reporting on a clinical trial, we recommend including planned or posthoc sensitivity analyses, the corresponding rationale and results along with the discussion of the consequences of these analyses on the overall findings of the study.

Efficacy of vitamin D supplementation in depression in adults: a systematic review.

CONTEXT Randomized controlled trials (RCTs) investigating the efficacy of vitamin D (Vit D) in depression provided inconsistent results. OBJECTIVE We aim to summarize the evidence of RCTs to assess the efficacy of oral Vit D supplementation in depression compared to placebo. DATA SOURCES We searched electronic databases, two conference proceedings, and gray literature by contacting authors of included studies. STUDY SELECTION We selected parallel RCTs investigating the effect of oral Vit D supplementation compared with placebo on depression in adults at risk of depression, with depression symptoms or a primary diagnosis of depression. DATA EXTRACTION Two reviewers independently extracted data from relevant literature. DATA SYNTHESIS Classical and Bayesian random-effects meta-analyses were used to pool relative risk, odds ratio, and standardized mean difference. The quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation tool. RESULTS Six RCTs were identified with 1203 participants (72% females) including 71 depressed patients; five of the studies involved adults at risk of depression, and one trial used depressed patients. Results of the classical meta-analysis showed no significant effect of Vit D supplementation on postintervention depression scores (standardized mean difference = -0.14, 95% confidence interval = -0.41 to 0.13, P = .32; odds ratio = 0.93, 95% confidence interval = 0.54 to 1.59, P = .79). The quality of evidence was low. No significant differences were demonstrated in subgroup or sensitivity analyses. Similar results were found when Bayesian meta-analyses were applied. CONCLUSIONS There is insufficient evidence to support the efficacy of Vit D supplementation in depression symptoms, and more RCTs using depressed patients are warranted.

A systematic scoping review of adherence to reporting guidelines in health care literature

Background Reporting guidelines have been available for the past 17 years since the inception of the Consolidated Standards of Reporting Trials statement in 1996. These guidelines were developed to improve the quality of reporting of studies in medical literature. Despite the widespread availability of these guidelines, the quality of reporting of medical literature remained suboptimal. In this study, we assess the current adherence practice to reporting guidelines; determine key factors associated with better adherence to these guidelines; and provide recommendations to enhance adherence to reporting guidelines for future studies. Methods We undertook a systematic scoping review of systematic reviews of adherence to reporting guidelines across different clinical areas and study designs. We searched four electronic databases (Cumulative Index to Nursing and Allied Health Literature, Web of Science, Embase, and Medline) from January 1996 to September 2012. Studies were included if they addressed adherence to one of the following guidelines: Consolidated Standards of Reporting Trials (CONSORT), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), Quality of Reporting of Meta-analysis (QUOROM), Transparent Reporting of Evaluations with Nonrandomized Designs (TREND), Meta-analysis Of Observational Studies in Epidemiology (MOOSE) and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). A protocol for this study was devised. A literature search, data extraction, and quality assessment were performed independently by two authors in duplicate. This study reporting follows the PRISMA guidelines. Results Our search retrieved 5159 titles, of which 50 were eligible. Overall, 86.0% of studies reported suboptimal levels of adherence to reporting guidelines. Factors associated with better adherence included journal impact factor and endorsement of guidelines, publication date, funding source, multisite studies, pharmacological interventions and larger studies. Conclusion Reporting guidelines in the clinical literature are important to improve the standards of reporting of clinical studies; however, adherence to these guidelines remains suboptimal. Action is therefore needed to enhance the adherence to these standards. Strategies to enhance adherence include journal editorial policies endorsing these guidelines.

Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: A systematic review

BACKGROUND We sought to determine if the publication of the Consolidated Standards of Reporting Trials (CONSORT)(1) extension for abstracts in 2008 had led to an improvement in reporting abstracts of randomized controlled trials (RCTs).(2) METHODS: We searched PubMed for RCTs published in 2007 and 2012 in top-tier general medicine journals. A random selection of 100 trial abstracts was obtained for each year. Data were extracted in duplicate on the adherence to the CONSORT extension for abstracts. The primary outcome was the mean number of items reported and the secondary outcome was the odds of reporting each item. We also estimated incidence rate ratios (IRRs).(3) RESULTS: Significantly more checklist items were reported in 2012 than in 2007: adjusted mean difference was 2.91 (95% confidence interval [CI](4) 2.35, 3.41; p<0.001). In 2012 there were significant improvements in reporting the study as randomized in the title, describing the trial design, the participants, and objectives and blinding. In the Results section, trial status and numbers analyzed were also reported better. The IRRs were significantly higher for 2012 (IRR 1.32; 95% CI 1.25, 1.39; p<0.001) and in multisite studies compared to single site studies (IRR 1.08; 95% CI 1.03, 1.15; p=0.006). CONCLUSIONS There was a significant improvement in the reporting of abstracts of RCTs in 2012 compared to 2007. However, there is still room for improvement as some items remain under-reported.

Empirical comparison of four baseline covariate adjustment methods in analysis of continuous outcomes in randomized controlled trials

Background Although seemingly straightforward, the statistical comparison of a continuous variable in a randomized controlled trial that has both a pre- and posttreatment score presents an interesting challenge for trialists. We present here empirical application of four statistical methods (posttreatment scores with analysis of variance, analysis of covariance, change in scores, and percent change in scores), using data from a randomized controlled trial of postoperative pain in patients following total joint arthroplasty (the Morphine COnsumption in Joint Replacement Patients, With and Without GaBapentin Treatment, a RandomIzed ControlLEd Study [MOBILE] trials). Methods Analysis of covariance (ANCOVA) was used to adjust for baseline measures and to provide an unbiased estimate of the mean group difference of the 1-year postoperative knee flexion scores in knee arthroplasty patients. Robustness tests were done by comparing ANCOVA with three comparative methods: the posttreatment scores, change in scores, and percentage change from baseline. Results All four methods showed similar direction of effect; however, ANCOVA (−3.9; 95% confidence interval [CI]: −9.5, 1.6; P=0.15) and the posttreatment score (−4.3; 95% CI: −9.8, 1.2; P=0.12) method provided the highest precision of estimate compared with the change score (−3.0; 95% CI: −9.9, 3.8; P=0.38) and percent change (−0.019; 95% CI: −0.087, 0.050; P=0.58). Conclusion ANCOVA, through both simulation and empirical studies, provides the best statistical estimation for analyzing continuous outcomes requiring covariate adjustment. Our empirical findings support the use of ANCOVA as an optimal method in both design and analysis of trials with a continuous primary outcome.

The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement

BackgroundRandomized controlled trials (RCTs) are routinely used in systematic reviews and meta-analyses that help inform healthcare and policy decision making. The proper reporting of RCTs is important because it acts as a proxy for health care providers and researchers to appraise the quality of the methodology, conduct and analysis of an RCT. The aims of this study are to analyse the overall quality of reporting in 23 RCTs that were used in a meta-analysis by assessing 3 key methodological items, and to determine factors associated with high quality of reporting. It is hypothesized that studies with larger sample sizes, that have funding reported, that are published in journals with a higher impact factor and that are in journals that have adopted or endorsed the CONSORT statement will be associated with better overall quality of reporting and reporting of key methodological items.MethodsWe systematically reviewed RCTs used within an anesthesiology related post-operative pain management meta-analysis. We included all of the 23 RCTs used, all of which were parallel design that addressed the use of femoral nerve block in improving outcomes after total knee arthroplasty. Data abstraction was done independently by two reviewers. The two main outcomes were: 1) 15 point overall quality of reporting score (OQRS) based on the Consolidated Standards for Reporting Trials (CONSORT) and 2) 3 point key methodological item score (KMIS) based on allocation concealment, blinding and intention-to-treat analysis.ResultsTwenty-three RCTs were included. The median OQRS was 9.0 (Interquartile Range = 3). A multivariable regression analysis did not show any significant association between OQRS or KMIS and our four predictor variables hypothesized to improve reporting. The direction and magnitude of our results when compared to similar studies suggest that the sample size and impact factor are associated with improved key methodological item reporting.ConclusionsThe quality of reporting of RCTs used within an anesthesia related meta-analysis is poor to moderate. The information gained from this study should be used by journals to register the urgency for RCTs to be clear and transparent in reporting to help make literature accessible and comparable.

Efficacy of vitamin D supplementation in depression in adults: a systematic review protocol

BackgroundThe role of vitamin D in management of depression is unclear. Results from observational and emerging randomized controlled trials (RCTs) investigating the efficacy of vitamin D in depression lack consistency - with some suggesting a positive association while others show a negative or inconclusive association.Methods/DesignThe primary aim of this study is to conduct a systematic review of RCTs to assess the effect of oral vitamin D supplementation versus placebo on depression symptoms measured by scales and the proportion of patients with symptomatic improvement according to the authors’ original definition. Secondary aims include assessing the change in quality of life, adverse events and treatment discontinuation. We will conduct the systematic review and meta-analysis according to the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions. We will search the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (1966 to present), EMBASE (1980 to present), CINAHL (1982 to present), PsychINFO (1967 to present) and ClinicalTrials.gov. Unpublished work will be identified by searching two major conferences: the International Vitamin Conference, the Anxiety Disorders and Depression Conference, while grey literature will be acquired by contacting authors of included studies. We will use the random-effects meta-analysis to synthesize the data by pooling the results of included studies.DiscussionThe results of this systematic review will be helpful in clarifying the efficacy of vitamin D supplementation and providing evidence to establish guidelines for implementation of vitamin D for depression in general practice and other relevant settings.Study registrationUnique identifier: CRD42013003849.

Although not consistently superior, the absolute approach to framing the minimally important difference has advantages over the relative approach

OBJECTIVES Using studies that established minimal important difference (MID) using anchor-based methods, we set out to address the relative merits of absolute and relative changes in establishing an instruments MID. STUDY DESIGN AND SETTING In seven data sets, we calculated correlations between global change ratings and absolute and relative score changes and conducted meta-analyses. We considered that the measure with the higher correlation represented the more valid approach. RESULTS The meta-analyses showed no significant difference between pooled correlations of absolute and relative difference on health-related quality of life instrument with global transition scores of symptoms, emotional function, physical function, and cognitive function. In four of five domains, there was at least one study in which the absolute was significantly superior to the relative; in one of these four, one study showed statistically significant superior performance of the relative. In an analysis restricted to patients with low baseline scores for the domain of cognitive function, the relative approach showed higher correlation with global rating than did the absolute approach. CONCLUSION Although we found no consistent superiority of either approach to establishing the MID, when differences existed they usually favored the absolute, which also has advantages of simplicity and ease of pooling across studies. Researchers may consider the absolute as a default but also compare both methods on an instrument by instrument basis.

Sample sizes for clinical trials using sputum eosinophils as a primary outcome

Clinical trials do not report sputum eosinophil data in a consistent method and this makes it difficult to compare across studies and to evaluate the sample sizes estimated in these studies. The objectives of the paper are: 1) to systematically review reporting of effect size and sample calculations in randomised controlled trials using sputum eosinophil count as a primary outcome and 2) to illustrate sample size estimation under different methods of data representation using data from an effective anti-eosinophil treatment strategy (mepolizumab). Randomised controlled trials in adults (excluding allergen provocation models) of treatment of asthma and chronic obstructive pulmonary disease for the past 10 years were searched in Ovid MEDLINE and 20 studies were identified that met all the inclusion criteria. Only nine studies discussed sample size calculation. Change from baseline was used as an outcome in 11 studies and was expressed as change in absolute percentage count, percentage change from baseline or as fold changes. Assuming a minimal clinically important reduction of 15% in absolute terms, 18 subjects in each arm will be required to achieve 80% power using an ANCOVA analysis, which we recommend, to detect significance with an alpha error of 0.05. Systematic review and illustration of sample size calculations in RCTs using sputum eosinophil count as a primary outcome http://ow.ly/mK9g3

Reanalysis of morphine consumption from two randomized controlled trials of gabapentin using longitudinal statistical methods

Background Postoperative pain management in total joint replacement surgery remains ineffective in up to 50% of patients and has an overwhelming impact in terms of patient well-being and health care burden. We present here an empirical analysis of two randomized controlled trials assessing whether addition of gabapentin to a multimodal perioperative analgesia regimen can reduce morphine consumption or improve analgesia for patients following total joint arthroplasty (the MOBILE trials). Methods Morphine consumption, measured for four time periods in patients undergoing total hip or total knee arthroplasty, was analyzed using a linear mixed-effects model to provide a longitudinal estimate of the treatment effect. Repeated-measures analysis of variance and generalized estimating equations were used in a sensitivity analysis to compare the robustness of the methods. Results There was no statistically significant difference in morphine consumption between the treatment group and a control group (mean effect size estimate 1.0, 95% confidence interval −4.7, 6.7, P=0.73). The results remained robust across different longitudinal methods. Conclusion The results of the current reanalysis of morphine consumption align with those of the MOBILE trials. Gabapentin did not significantly reduce morphine consumption in patients undergoing major replacement surgeries. The results remain consistent across longitudinal methods. More work in the area of postoperative pain is required to provide adequate management for this patient population.