Chenglin Ye

A tutorial on sensitivity analyses in clinical trials: the what, why, when and how

BackgroundSensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations—such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers—on the overall conclusions of a study.The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.DiscussionIn this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.SummaryWhen reporting on a clinical trial, we recommend including planned or posthoc sensitivity analyses, the corresponding rationale and results along with the discussion of the consequences of these analyses on the overall findings of the study.

Mobile phone text messages for improving adherence to antiretroviral therapy (ART): a protocol for an individual patient data meta-analysis of randomised trials

Objectives Our objectives were to analyse the effects of text messaging versus usual care in improving adherence to antiretroviral therapy (ART) in people living with HIV using individual patient data meta-analysis. Adjusted, sensitivity and subgroup analyses were conducted. Setting 3 randomised controlled trials conducted between 2010 and 2012 in rural and urban centres in Cameroon and Kenya (two studies) were used. Participants A total of 1166 participants were included in this analysis (Cameroon=200; Kenya=428 and 538). Primary and secondary outcomes The primary outcome was adherence to ART >95%. The secondary outcomes were mortality, losses to follow-up, transfers and withdrawals. Results Text messaging improved adherence to ART (OR 1.38; 95% CIs 1.08 to 1.78; p=0.012), even after adjustment for baseline covariates (OR 1.46; 95% CI 1.13 to 1.88; p=0.004). Primary education (compared with no formal education) was associated with a greater intervention effect on adherence (OR 1.65; 95% CI 1.10 to 2.48; p=0.016) and also showed a significant subgroup effect (p=0.039). In sensitivity analysis, our findings were robust to a modified threshold of adherence, multiple imputation for missing data and aggregate level data pooling, but not to fixed-effects meta-analyses using generalised estimation equations. There was a significant subgroup effect for long weekly (p=0.037), short weekly text messages (p=0.014) and interactive messaging (p=0.010). Text messaging did not significantly affect any of the secondary outcomes. Conclusions Text messaging has a significant effect on adherence to ART, and this effect is influenced by level of education, gender, timing (weekly vs daily) and interactivity. We recommend the use of interactive weekly text messaging to improve adherence to ART, which is most effective in those with at least a primary level of education.

Association between framing of the research question using the PICOT format and reporting quality of randomized controlled trials

BackgroundExperts recommend formulating a structured research question to guide the research design. However, the basis for this recommendation has not been formally evaluated. The aim of this study was to examine if a structured research question using the PICOT (Population, Intervention, Comparator, Outcome, Time-frame) format is associated with a better reporting quality of randomized controlled trials (RCTs).MethodsWe evaluated 89 RCTs reports published in three endocrinology journals in 2005 and 2006, the quality of reporting of which was assessed in a previous study. We examined whether the reports stated each of the five elements of a structured research question: population, intervention, comparator, outcome and time-frame. A PICOT score was created with a possible score between 0 and 5. Outcomes were: 1) a 14-point overall reporting quality score (OQS) based on the Consolidated Standards for Reporting Trials; and 2) a 3-point key score (KS), based on allocation concealment, blinding and use of intention-to-treat analysis. We conducted multivariable regression analyses using generalized estimating equations to determine if a higher PICOT score or the use of a structured research question were independently associated with a better reporting quality. Journal of publication, funding source and sample size were identified as factors associated with OQS in our previous report on this dataset, and therefore included in the model.ResultsA higher PICOT score was independently associated with OQS (incidence rate ratio (IRR) = 1.021, 95% CI: 1.012 to 1.029) and KS (IRR = 1.142, 95% CI: 1.079 to 1.210). A structured research question was present in 33.7% of the reports and it was associated with a better OQS (IRR = 1.095, 95% CI 1.059-1.132) and KS (IRR = 1.530, 95% CI 1.311-1.786).ConclusionsBetter framing of the research question using the PICOT format is independently associated with better overall reporting quality - although the effect is small - and better reporting of key methodologies.

Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: A systematic review

BACKGROUND We sought to determine if the publication of the Consolidated Standards of Reporting Trials (CONSORT)(1) extension for abstracts in 2008 had led to an improvement in reporting abstracts of randomized controlled trials (RCTs).(2) METHODS: We searched PubMed for RCTs published in 2007 and 2012 in top-tier general medicine journals. A random selection of 100 trial abstracts was obtained for each year. Data were extracted in duplicate on the adherence to the CONSORT extension for abstracts. The primary outcome was the mean number of items reported and the secondary outcome was the odds of reporting each item. We also estimated incidence rate ratios (IRRs).(3) RESULTS: Significantly more checklist items were reported in 2012 than in 2007: adjusted mean difference was 2.91 (95% confidence interval [CI](4) 2.35, 3.41; p<0.001). In 2012 there were significant improvements in reporting the study as randomized in the title, describing the trial design, the participants, and objectives and blinding. In the Results section, trial status and numbers analyzed were also reported better. The IRRs were significantly higher for 2012 (IRR 1.32; 95% CI 1.25, 1.39; p<0.001) and in multisite studies compared to single site studies (IRR 1.08; 95% CI 1.03, 1.15; p=0.006). CONCLUSIONS There was a significant improvement in the reporting of abstracts of RCTs in 2012 compared to 2007. However, there is still room for improvement as some items remain under-reported.

The effect of regular physical activity on bone mineral density in post-menopausal women aged 75 and over: a retrospective analysis from the Canadian multicentre osteoporosis study

BackgroundPhysical activity is known to benefit many physiological processes, including bone turnover. There are; however, currently no clinical guidelines regarding the most appropriate type, intensity and duration of activity to prevent bone loss.MethodsTo help address this gap in the literature, we performed a retrospective analysis of data from the Canadian Multicentre Osteoporosis Study (CaMos), a prospective cohort of 9423 adult patients, to determine the relationship between the amount of regular daily physical activity performed and bone mineral density. A total of 1169 female participants aged 75 and over provided information regarding their daily activity levels, including the amount of time spent each week performing physical activity at varying levels of intensity. Multiple and linear regression analyses were used to determine the effect of increasing amounts of this regular physical activity on bone mineral density.ResultsThe results indicate that a step increase in the amount of physical activity performed each day resulted in a positive effect on bone mineral density at the hip, Ward’s triangle, trochanter and femoral neck (B = 0.006 to 0.008, p < 0.05). Possible confounding factors such as the use of anti-resorptive therapy, body mass index and age were included in the analysis and suggested that age had a negative effect on bone density while body mass index had a positive effect. Anti-resorptive therapy provided a protective effect against loss of bone density.ConclusionsThe data indicate that a step increase in the amount of daily activity, using simple, daily performed tasks, can help prevent decreases in post-menopausal bone mineral density.

Estimating treatment effects in randomised controlled trials with non-compliance: a simulation study

Objective Randomised controlled trials (RCTs) are often considered as the gold standard for assessing new health interventions. Patients are randomly assigned to receive an intervention or control. The effect of the intervention can be estimated by comparing outcomes between groups, whose prognostic factors are expected to balance by randomisation. However, patients’ non-compliance with their assigned treatment will undermine randomisation and potentially bias the estimate of treatment effect. Through simulation, we aim to compare common approaches in analysing non-compliant data under different non-compliant scenarios. Settings Based on a real study, we simulated hypothetical trials by varying three non-compliant factors: the type, randomness and degree of non-compliance. We compared the intention-to-treat (ITT), as-treated (AT), per-protocol (PP), instrumental variable (IV) and complier average casual effect (CACE) analyses to estimate large (50% improvement over the control), moderate (25% improvement) and null (same as the control) treatment effects. Different approaches were compared by the bias of estimate, mean square error (MSE) and 95% coverage of the true value. Results For a large or moderate treatment effect, the ITT estimate was considerably biased in all scenarios. The AT, PP, IV and CACE estimates were unbiased when non-compliant behaviours were random. The IV estimate was unbiased when non-compliant behaviours were symmetrically dependent on patients’ conditions. The PP estimate was mostly unbiased when patients in the control group did not have access to the intervention. When the intervention was not different from the control, the ITT was less biased than the other approaches. Similar results were found when comparing the MSE and 95% coverage. Conclusions The standard ITT analysis under non-compliance is biased when the intervention has a moderate or large effect. Alternative analyses can provide unbiased or less biased estimates. Based on the results, we make some suggestions on choosing optimal approaches for analysing specific non-compliant scenarios.

System Integration and Its Influence on the Quality of Life of Children with Complex Needs

Purpose. To explore the interactions between child and parents psychosocial factors and team integration variables that may explain improvements in physical dimensions of the PEDS QL quality of life of children with complex needs after 2 years. Methods. In this 2-year study, parents were identified by the Childrens Treatment Network. Families were eligible if the child was aged 0–19 years, had physical limitations, resided in either Simcoe County or the Region of York, Ontario, and there were multiple other family needs. Regression analysis used to explore associations and interactions; n = 110. Results. A childs physical quality of life was affected by interacting factors including childs behavior, parenting, and integrated care. Statistically significant interactions between team integration, processes of care, and child/parent variables highlight the complexity of the rehabilitation approach in real-life situations. Conclusions. Rehabilitation providers working with children with complex needs and their families should also address child and parent problematic behaviors. When this was the case in high integrated teams, the childs physical quality of life improved after two years.

Performance of the high-sensitivity troponin assay in diagnosing acute myocardial infarction: systematic review and meta-analysis

BACKGROUND High-sensitivity cardiac troponin assays have been adopted by many clinical centres worldwide; however, clinicians are uncertain how to interpret the results. We sought to assess the utility of these assays in diagnosing acute myocardial infarction (MI). METHODS We carried out a systematic review and meta-analysis of studies comparing high-sensitivity with conventional assays of cardiac troponin levels among adults with suspected acute MI in the emergency department. We searched MEDLINE, EMBASE and Cochrane databases up to April 2013 and used bivariable random-effects modelling to obtain summary parameters for diagnostic accuracy. RESULTS We identified 9 studies that assessed the use of high-sensitivity troponin T assays (n = 9186 patients). The summary sensitivity of these tests in diagnosing acute MI at presentation to the emergency department was estimated to be 0.94 (95% confidence interval [CI] 0.89-0.97); for conventional tests, it was 0.72 (95% CI 0.63-0.79). The summary specificity was 0.73 (95% CI 0.64-0.81) for the high-sensitivity assay compared with 0.95 (95% CI 0.93-0.97) for the conventional assay. The differences in estimates of the summary sensitivity and specificity between the high-sensitivity and conventional assays were statistically significant (p < 0.01). The area under the curve was similar for both tests carried out 3-6 hours after presentation. Three studies assessed the use of high-sensitivity troponin I assays and showed similar results. INTERPRETATION Used at presentation to the emergency department, the high-sensitivity cardiac troponin assay has improved sensitivity, but reduced specificity, compared with the conventional troponin assay. With repeated measurements over 6 hours, the area under the curve is similar for both tests, indicating that the major advantage of the high-sensitivity test is early diagnosis.

The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement

BackgroundRandomized controlled trials (RCTs) are routinely used in systematic reviews and meta-analyses that help inform healthcare and policy decision making. The proper reporting of RCTs is important because it acts as a proxy for health care providers and researchers to appraise the quality of the methodology, conduct and analysis of an RCT. The aims of this study are to analyse the overall quality of reporting in 23 RCTs that were used in a meta-analysis by assessing 3 key methodological items, and to determine factors associated with high quality of reporting. It is hypothesized that studies with larger sample sizes, that have funding reported, that are published in journals with a higher impact factor and that are in journals that have adopted or endorsed the CONSORT statement will be associated with better overall quality of reporting and reporting of key methodological items.MethodsWe systematically reviewed RCTs used within an anesthesiology related post-operative pain management meta-analysis. We included all of the 23 RCTs used, all of which were parallel design that addressed the use of femoral nerve block in improving outcomes after total knee arthroplasty. Data abstraction was done independently by two reviewers. The two main outcomes were: 1) 15 point overall quality of reporting score (OQRS) based on the Consolidated Standards for Reporting Trials (CONSORT) and 2) 3 point key methodological item score (KMIS) based on allocation concealment, blinding and intention-to-treat analysis.ResultsTwenty-three RCTs were included. The median OQRS was 9.0 (Interquartile Range = 3). A multivariable regression analysis did not show any significant association between OQRS or KMIS and our four predictor variables hypothesized to improve reporting. The direction and magnitude of our results when compared to similar studies suggest that the sample size and impact factor are associated with improved key methodological item reporting.ConclusionsThe quality of reporting of RCTs used within an anesthesia related meta-analysis is poor to moderate. The information gained from this study should be used by journals to register the urgency for RCTs to be clear and transparent in reporting to help make literature accessible and comparable.

Data withdrawal in randomized controlled trials: Defining the problem and proposing solutions: A commentary

It is not uncommon for a participant to withdraw from a randomized controlled trial (RCT). The withdrawal of a participant results in missing data and the potential for withdrawal bias. Data withdrawal, or a request from a participant to withdraw all of their previously collected data from a study, is particularly problematic because it leaves little opportunity to characterize or statistically address those that have withdrawn to minimize withdrawal bias. The aim of this commentary is to (1) provide a synthesis of available information on the ethical and methodological issues related to data withdrawal in RCTs and (2) provide some suggestions on how to minimize the impact of data withdrawal during the execution or analysis phases of an RCT. We searched PubMed, EMBASE and JSTOR for published articles on data withdrawal. In addition, we used internet sources as an additional tool to identify content on data withdrawal from research ethics guidelines, legislation, research ethics boards, funding agencies, professional organizations and researchers. We did not find any definitive guidelines for dealing with data withdrawal. We propose recommendations for minimizing the occurrence of data withdrawal, including explicit and clear descriptions in consent forms of how data will be handled after participant withdrawal. We also suggest using imputation techniques to deal with the missing data during analysis. The current commentary can be used to minimize the impact of data withdrawal in RCTs.