Greta Castellini

Comment on: “Cell Therapy for Heart Disease: Trial Sequential Analyses of Two Cochrane Reviews”

Trial Sequential Analysis (TSA) is a frequentist method to help researchers control the risks of random errors in meta‐analyses.1 Fisher et al.2 used TSA on cell therapy for heart diseases. The present article discusses the usefulness of TSA and its dependence on the choice of the parameters for calculation of the required information size and the adjacent monitoring boundaries, and comments on the approach by Fisher et al.2

Survey of the Reporting Characteristics of Systematic Reviews in Rehabilitation

Background Systematic reviews (SRs) have become increasingly important for informing clinical practice; however, little is known about the reporting characteristics and the quality of the SRs relevant to the practice of rehabilitation health professionals. Objective The purpose of this study was to examine the reporting quality of a representative sample of published SRs on rehabilitation, focusing on the descriptive, reporting, and bias-related characteristics. Methods A cross-sectional study was conducted by searching MEDLINE for aggregative and configurative SRs indexed in 2011 that focused on rehabilitation as restorative of functional limitations. Two reviewers independently screened and selected the SRs and extracted data using a 38-item data collection form derived from Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The data were analyzed descriptively. Results Eighty-eight SRs published in 59 journals were sampled. The median compliance with the PRISMA items was 17 (63%) out of 27 items (interquartile ratio=13–22 [48%–82%]). Two thirds of the SRs (n=66) focused on interventions for which efficacy is best addressed through a randomized controlled trial (RCT) design, and almost all of these SRs included RCTs (63/66 [95%]). More than two thirds of the SRs assessed the quality of primary studies (74/88 [84%]). Twenty-eight reviews (28/88 [32%]) meta-analyzed the results for at least one outcome. One half of the SRs reported positive statistically significant findings (46%), whereas a detrimental result was present only in one review. Conclusions This sample of SRs in the rehabilitation field showed heterogeneous characteristics and a moderate quality of reporting. Poor control of potential source of bias might be improved if more widely agreed-upon evidence-based reporting guidelines will be actively endorsed and adhered to by authors and journals.

Constraint‐induced movement therapy for upper extremities in people with stroke

Graeme J. Hankey MD, FRACP, FRCP Section Editor Constraint-induced movement therapy (CIMT) is a current approach to stroke rehabilitation that implies the forced and prolonged use of the affected arm by restraining the unaffected arm by enhancing the recovery of the residual motor power. The enhancement of the residual motor power by compulsory exercise is controversial. The objective was to assess the efficacy of CIMT for arm management in hemiparetic patients. We searched the Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Library (Issue 1, 2008), the Cochrane Stroke Group trials register, MEDLINE, EMBASE, CINAHL, and PEDro (all to June 2008). We selected randomized trials comparing CIMT, modified CIMT, or forced use with other rehabilitative techniques or none in adults after …

Completeness of Outcomes Description Reported in Low Back Pain Rehabilitation Interventions: A Survey of 185 Randomized Trials

Purpose: To assess reporting completeness of the most frequent outcome measures used in randomized controlled trials (RCTs) of rehabilitation interventions for mechanical low back pain. Methods: We performed a cross-sectional study of RCTs included in all Cochrane systematic reviews (SRs) published up to May 2013. Two authors independently evaluated the type and frequency of each outcome measure reported, the methods used to measure outcomes, the completeness of outcome reporting using a eight-item checklist, and the proportion of outcomes fully replicable by an independent assessor. Results: Our literature search identified 11 SRs, including 185 RCTs. Thirty-six different outcomes were investigated across all RCTs. The 2 most commonly reported outcomes were pain (n=165 RCTs; 89.2%) and disability (n=118 RCTs; 63.8%), which were assessed by 66 and 44 measurement tools, respectively. Pain and disability outcomes were found replicable in only 10.3% (n=17) and 10.2% (n=12) of the RCTs, respectively. Only 40 RCTs (21.6%) distinguished between primary and secondary outcomes. Conclusions: A large number of outcome measures and a myriad of measurement instruments were used across all RCTs. The reporting was largely incomplete, suggesting an opportunity for a standardized approach to reporting in rehabilitation science.

Reporting of Rehabilitation Intervention for Low Back Pain in Randomized Controlled Trials: Is the Treatment Fully Replicable?

Study Design. Methodological review of randomized controlled trials (RCTs). Objective. To assess the quality of reporting of rehabilitation interventions for mechanical low back pain (LBP) in published RCTs. Summary of Background Data. Reporting of interventions in RCTs often focused on the outcome value and failed to describe interventions adequately. Methods. We systematically searched for all RCTs in Cochrane systematic reviews on LBP published in the Cochrane Database of Systematic Reviews until December 2013. The description of rehabilitation interventions of each RCT was evaluated independently by 2 of the investigators, using an ad hoc checklist of 7 items. The primary outcome was the number of items reported in sufficient details to be replicable in a new RCT or in everyday practice. Results. We found 11 systematic reviews, including 220 eligible RCTs, on LBP. Of those, 185 RCTs were included. The median publication year was 1998 (I-III quartiles, 1990 to 2004). The most reported items were the characteristics of participants (91.3%; 95% confidence interval [CI], 87.3–95.4), the intervention providers (81.1%; 95% CI, 75.4–86.7), and the intervention schedule (69.7%; 95% CI, 63–76). Based on the description of the intervention, less than one fifth would be replicable clinically. The proportion of trials providing all essential information about the participants and interventions increased from 14% (n = 7) in 1971 to 1980 to 20% (n = 75) in 2001 to 2010. Conclusion. Despite the remarkable amount of energy spent producing RCTs in LBP rehabilitation, the majority of RCTs failed to report sufficient information that would allow the intervention to be replicated in clinical practice. Improving the quality of intervention description is urgently needed to better transfer research into rehabilitation practices. Level of Evidence: 1

Constraint-induced movement therapy: trial sequential analysis applied to Cochrane collaboration systematic review results

BackgroundTrial sequential analysis (TSA) may establish when firm evidence about the efficacy of interventions is reached in a cumulative meta-analysis, combining a required information size with adjusted thresholds for conservative statistical significance. Our aim was to demonstrate TSA results on randomized controlled trials (RCTs) included in a Cochrane systematic review on the effectiveness of constraint-induced movement therapy (CIMT) for stroke patients.MethodsWe extracted data on the functional independence measure (FIM) and the action research arm test (ARAT) from RCTs that compared CIMT versus other rehabilitative techniques. Mean differences (MD) were analyzed using a random-effects model. We calculated the information size and the cumulative Z-statistic, applying the O’Brien-Fleming monitoring boundaries.ResultsWe included data from 14 RCTs. In the conventional meta-analysis (seven trials, 233 patients), the effect of CIMT on FIM was reported as significant (MD 2.88, 95% CI 0.08 to 5.68; P = 0.04). The diversity-adjusted required information size was 142 patients, and the cumulative Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.02 to 5.78). The effect of CIMT on ARAT (nine trials, 199 patients) was reported as significant (MD 7.78, 95% CI 1.19 to 14.37; P = 0.02). However, the diversity-adjusted required information size was 252 patients, and the Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.06 to 15.62).ConclusionsAlthough conventional meta-analyses of CIMT reached statistical significance, their overall results remain inconclusive and might be spurious. Researchers should not be overconfident on CIMT efficacy based on the results of meta-analyses and derived recommendations.

Improving Power and Sample Size Calculation in Rehabilitation Trial Reports: A Methodological Assessment

OBJECTIVE To systematically assess the reporting of sample size calculation in randomized controlled trials (RCTs) on rehabilitation interventions for mechanical low back pain. DATA SOURCES The Cochrane Database of Systematic Reviews was searched through February 2015. STUDY SELECTION We conducted an electronic database search for RCTs published from January 1, 1968 to February 28, 2015 and included in the Cochrane Systematic Reviews. DATA EXTRACTION Two investigators independently used an ad hoc 6-item checklist derived from the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement recommendations to extract data on sample size calculation. The primary outcome was the proportion of RCTs that reported sample size calculation; the secondary outcome was the completeness of sample size analysis reporting. We also evaluated improvement in reporting of sample size calculation over time. DATA SYNTHESIS Sample size calculation was reported in 80 (36.0%) of the 222 eligible RCTs included in 14 Cochrane Systematic Reviews. Only 13 (16.3%) of these RCT reports gave a complete description, and about half reported ≥4 of the 6 elements of sample size calculation (median, 4; interquartile range, 3-5). Completeness of reporting of sample size calculation improved from 1968 to 2013; since 2005, the number of RCTs reporting sample size calculation has increased compared with the number of RCTs not reporting it. CONCLUSIONS Despite improvement, reporting of sample size calculation and power analysis remains inadequate, limiting the readers ability to assess the quality and accuracy of rehabilitation studies.

Mechanical Low Back Pain: Secular Trend and Intervention Topics of Randomized Controlled Trials

PURPOSE To evaluate the number of published randomized controlled trials (RCTs) focusing on mechanical low back pain (MLBP) rehabilitation, the secular (i.e., long-term) trend, and the distribution of interventions studied. METHODS All included RCTs were extracted from all Cochrane systematic reviews focusing on rehabilitation therapies for MLBP, and two independent reviewers screened and analyzed the information on interventions. RESULTS After removal of duplicates, the data set consisted of 196 RCTs published between 1961 and 2010. The number of RCTs published increased consistently over time: 2 trials (1% of the total) were published in 1961-1970, 10 (5%) in 1971-1980, 41 (21%) in 1981-1990, 68 (35%) in 1991-2000, and 75 (38%) in 2001-2010. The intervention of interest in the majority of RCTs was exercise therapy (115/399; 29%), followed by spinal manipulation therapies (60/399; 15%). CONCLUSION The number of RCTs focusing on MLBP has risen over time; of all interventions studied, exercise therapy has attracted the most research interest.

Assessing imprecision in Cochrane systematic reviews : A comparison of GRADE and Trial Sequential Analysis

BackgroundThe evaluation of imprecision is a key dimension of the grading of the confidence in the estimate. Grading of Recommendations Assessment, Development and Evaluation (GRADE) gives recommendations on how to downgrade evidence for imprecision, but authors vary in their use. Trial Sequential Analysis (TSA) has been advocated for a more reliable assessment of imprecision. We aimed to evaluate reporting of and adherence to GRADE and to compare the assessment of imprecision of intervention effects assessed by GRADE and TSA in Cochrane systematic reviews.MethodsIn this cross-sectional study, we included 100 Cochrane reviews irrespective of type of intervention with a key dichotomous outcome meta-analyzed and assessed by GRADE. The methods and results sections of each review were assessed for adequacy of imprecision evaluation. We re-analyzed imprecision following the GRADE Handbook and the TSA Manual.ResultsOverall, only 13.0% of reviews stated the criteria they applied to assess imprecision. The most common dimensions were the 95% width of the confidence intervals and the optimal information size. Review authors downgraded 48.0% of key outcomes due to imprecision. When imprecision was re-analyzed following the GRADE Handbook, 64% of outcomes were downgraded. Agreement between review authors’ assessment and assessment by the authors of this study was moderate (kappa 0.43, 95% confidence interval [CI] 0.23 to 0.58). TSA downgraded 69.0% outcomes due to imprecision. Agreement between review authors’ GRADE assessment and TSA, irrespective of downgrading levels, was moderate (kappa 0.43, 95% CI 0.21 to 0.57). Agreement between our GRADE assessment following the Handbook and TSA was substantial (kappa 0.66, 95% CI 0.49 to 0.79).ConclusionsIn a sample of Cochrane reviews, methods for assessing imprecision were rarely reported. GRADE according to Handbook guidelines and TSA led to more severe judgment of imprecision rather than GRADE adopted by reviews’ authors. Cochrane initiatives to improve adherence to GRADE Handbook are warranted. TSA may transparently assist in such development.

Completeness of outcome description in studies for low back pain rehabilitation interventions: a survey of trials included in Cochrane reviews

95% CI 31.9% – 45.9%), and quality of life (45/185; 24,3%, 95% CI 18.1% – 30.5%) measured respectively by 70, 43, 41, 19 different measurement instruments (Figure 2). The procedure of blinding assessment was reported in 49.7% of the RCTs for pain (n= 82 RCTs) and 45% of RCTs for disability (n=53 RCTs). Pain, disability, range of motion, and quality of life outcomes were reported as fully replicable in 10.3% (n= 17 RCTs), 10.1% (n= 12 RCTs), 5.5% (n= 4 RCTs), and 6.6% (n= 3 RCTs) of the RCTs, respectively (Figure 3).