Yasushi Tsujimoto

Majority of systematic reviews published in high-impact journals neglected to register the protocols: a meta-epidemiological study

OBJECTIVES To describe the registration of systematic review (SR) protocols and examine whether or not registration reduced the outcome reporting bias in high-impact journals. STUDY DESIGN AND SETTING We searched MEDLINE via PubMed to identify SRs of randomized controlled trials of interventions. We included SRs published between August 2009 and June 2015 in the 10 general and internal medicinal journals with the highest impact factors in 2013. We examined the proportion of SR protocol registration and investigated the relationship between registration and outcome reporting bias using multivariable logistic regression. RESULTS Among the 284 included reviews, 60 (21%) protocols were registered. The proportion of registration increased from 5.6% in 2009 to 27% in 2015 (P for trend <0.001). Protocol registration was not associated with outcome reporting bias (adjusted odds ratio [OR] 0.85, 95% confidence interval [CI] 0.39-1.86). The association between Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) adherence and protocol registration was not statistically significant (OR 1.09, 95% CI 0.59-2.01). CONCLUSIONS Six years after the launch of the PRISMA statement, the proportion of protocol registration in high-impact journals has increased some but remains low. The present study found no evidence suggesting that protocol registration reduced outcome reporting bias.

Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

Objectives To investigate whether overstatements in abstract conclusions influence primary care physicians’ evaluations when they read reports of randomised controlled trials (RCTs) Design RCT setting: This study was a parallel-group randomised controlled survey, conducted online while masking the study hypothesis. Participants Volunteers were recruited from members of the Japan Primary Care Association in January 2017. We sent email invitations to 7040 primary care physicians. Among the 787 individuals who accessed the website, 622 were eligible and automatically randomised into ‘without overstatement’ (n=307) and ‘with overstatement’ (n=315) groups. Interventions We selected five abstracts from published RCTs with at least one non-significant primary outcome and overstatement in the abstract conclusion. To construct a version without overstatement, we rewrote the conclusion sections. The methods and results sections were standardised to provide the necessary information of primary outcome information when it was missing in the original abstract. Participants were randomly assigned to read an abstract either with or without overstatements and asked to evaluate the benefit of the intervention. Outcome measures The primary outcome was the participants’ evaluation of the benefit of the intervention discussed in the abstract, on a scale from 0 to 10. A secondary outcome was the validity of the conclusion. Results There was no significant difference between the groups with respect to their evaluation of the benefit of the intervention (mean difference: 0.07, 95% CI −0.28 to 0.42, p=0.69). Participants in the ‘without’ group considered the study conclusion to be more valid than those in the ‘with’ group (mean difference: 0.97, 95% CI 0.59 to 1.36, P<0.001). Conclusion The overstatements in abstract conclusions did not significantly influence the primary care physicians’ evaluations of the intervention effect when necessary information about the primary outcomes was distinctly reported. Trial registration number UMIN000025317; Pre-results.

Exercise can improve sleep quality: a systematic review and meta-analysis

Background Insomnia is common. However, no systematic reviews have examined the effect of exercise on patients with primary and secondary insomnia, defined as both sleep disruption and daytime impairment. This systematic review and meta-analysis aimed to examine the effectiveness/efficacy of exercise in patients with insomnia. Methods We searched the Cochrane Central Register of Controlled Trials, MEDLINE, Embase, PsycINFO, World Health Organization International Clinical Trials Registry Platform, and ClinicalTrials.gov to identify all randomized controlled trials that examined the effects of exercise on various sleep parameters in patients with insomnia. All participants were diagnosed with insomnia, using standard diagnostic criteria or predetermined criteria and standard measures. Data on outcome measures were subjected to meta-analyses using random-effects models. The Cochrane Risk of Bias Tool and Grading of Recommendations, Assessment, Development, and Evaluation approach were used to assess the quality of the individual studies and the body of evidence, respectively. Results We included nine studies with a total of 557 participants. According to the Pittsburgh Sleep Quality Index (mean difference [MD], 2.87 points lower in the intervention group; 95% confidence interval [CI], 3.95 points lower to 1.79 points lower; low-quality evidence) and the Insomnia Severity Index (MD, 3.22 points lower in the intervention group; 95% CI, 5.36 points lower to 1.07 points lower; very low-quality evidence), exercise was beneficial. However, exercise interventions were not associated with improved sleep efficiency (MD, 0.56% lower in the intervention group; 95% CI, 3.42% lower to 2.31% higher; moderate-quality evidence). Only four studies noted adverse effects. Most studies had a high or unclear risk of selection bias. Discussion Our findings suggest that exercise can improve sleep quality without notable adverse effects. Most trials had a high risk of selection bias. Higher quality research is needed.

Rehabilitation for patients with sepsis: A systematic review and meta-analysis

The objective of this systematic review was to determine whether rehabilitation impacts clinically relevant outcomes among adult patients with sepsis. Randomized controlled trials from the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PEDro, and the World Health Organization International Clinical Trials Platform Search Portal, as well as conference proceedings and reference lists of relevant articles were collected. Two reviewers independently identified randomized controlled trials on the rehabilitation of patients with sepsis, and the two reviewers independently abstracted trial level data including population characteristics, interventions, comparisons, and clinical outcomes. Our primary outcomes were quality of life (QOL), activity of daily living (ADL), and mortality. Our secondary outcomes were length of stay, return to work, muscle strength, delirium, and all adverse events. The quality of evidence was determined using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. We included two trials enrolling 75 patients. The mean difference (95% confidence interval [CI]) of physical function and physical role in QOL measured by SF-36 were 21.10 (95% CI: 6.57–35.63) and 44.40 (95% CI: 22.55–66.05), respectively. Rehabilitation did not significantly decrease intensive care unit (ICU) mortality (risk ratio, 2.02 [95% CI: 0.46–8.91], I2 = 0%; n = 75). ICU length of stay and hospital length of stay and muscle strength were not statistically significantly different and no adverse events were reported in both studies. The certainty of the evidence for these outcomes was “very low.” Data on ADL, return to work, and delirium were not available in any of the trials. Rehabilitation of patients with sepsis might not decrease ICU mortality, but might improve QOL. Further, well-designed trials measuring important outcomes will be needed to determine the benefit and harm of rehabilitation among patients with sepsis.

Variations and characteristics of quality indicators for maintenance hemodialysis patients: A systematic review

Several quality indicators (QIs) to improve the quality of practice for hemodialysis patients have been implemented. However, the variations and characteristics of these indicators in terms of their use and feasibility have not been investigated. We conducted a systematic review to evaluate the variations and characteristics of existing QIs for maintenance hemodialysis patients.

Comparative efficacy and safety of bone-modifying agents for the treatment of bone metastases in patients with advanced renal cell carcinoma: a systematic review and meta-analysis

Objective To determine the comparative efficacy and safety of all available bone-modifying agents (BMAs) for the treatment of bone metastases in patients with advanced renal cell carcinoma (RCC). Results Three studies (259 patients) were identified for the systematic review. Two studies that compared zoledronic acid with placebo or no zoledronic acid showed that zoledronic acid reduced the risk of skeletal-related events (SREs) by 68% (pooled hazard ratio [HR]: 0.32; 95% confidence interval [CI]: 0.19–0.55). The quality of evidence was moderate. The incidence of serious adverse events (AEs) was identical (80%) for both treatment arms in one study and not reported in the other study. In the third study that compared denosumab and zoledronic acid among patients with solid tumors or myeloma, a post-hoc subgroup analysis with individual patient data (155 patients) showed an HR of 0.71 (95% CI: 0.43–1.17) for SREs and a risk ratio of 0.86 (95% CI: 0.68–1.08) for serious AEs for denosumab compared to zoledronic acid. Materials and Methods We searched the MEDLINE database, Cochrane Library, WHO International Clinical Trials Registry Platform, and ClinicalTrials.gov database up to January 2017 without language restriction. Only randomized controlled trials were included. When relevant data were missing, we contacted the investigators of the original study. The Grading of Recommendation Assessment, Development, and Evaluation approach was used to assess the evidence certainty. Conclusions Our moderate-quality evidence indicates that zoledronic acid significantly reduces SREs risk among patients with bone metastases of RCC.

Association between statistical significance and time to publication among systematic reviews: a study protocol for a meta-epidemiological investigation

Introduction Many studies have indicated the impact of bias in dissemination and publication in medical research. Existence of such bias among clinical trials has been repeatedly pointed out, but it has not been well studied in the field of systematic reviews (SRs). We therefore aim to investigate whether or not time lag bias and publication bias in SRs based on statistical significance in results exist. In addition, we will examine at what stage of paper publication process such bias, if any, creeps in. Method and analysis The present study is a meta-epidemiological study. We will include all SRs of interventions registered in the international prospective register of SRs (PROSPERO) before December 2014 if the SR has completed its analysis irrespective of its publication status. All contact authors of eligible SRs will be asked to participate in a survey administered through the Internet. Our primary outcome is time from protocol registration to full publication of SR as a journal article, defined as time from the registration date to the acceptance date among all the relevant SRs. We will examine the impact of statistically significant findings on the primary outcomes through time to event analyses. Ethics and dissemination Ethics approval will be obtained from the Ethical Committee of the Kyoto University Graduate School of Medicine. This protocol has been registered in the University Hospital Medical Information Network Clinical Trials Registry. We will publish our findings in a peer-reviewed journal and also may present them at conferences. Trial registration number: UMIN000028325

Discrepancy of Odds Ratios and Inappropriate Presentation of the Results and Selection of Covariates Used in Multivariate Model.

Critical Care Medicine www.ccmjournal.org e587 The authors reply: We thank Tsujimoto et al (1) for their interest in our study (2). Three questions were pointed out. The disparity of odds ratio (OR) estimates between continuous predictor and dichotomized continuous predictor has been widely discussed previously (3–5). In terms of identifying independent predictors of acute kidney injury (AKI), postoperative albumin level was considered as a continuous variable. The results of multivariate regression analysis shown in Tables 3 and 4 in (2) were based on the lowest albumin level within two postoperative days. We found a description error of the title, “postoperative albumin level less than 3.0 g/dL within 2 days,” instead of “lowest albumin level within two postoperative days” in Tables 3 and 4 in (2). Obviously, the discrepancy of univariate ORs (3.70 vs 2.49 and 4.76 vs 2.61 in Tables 3 and 4 in [2], respectively) mentioned by Tsujimoto et al (1) occurred due to typographical error, not due to statistical error. Furthermore, we did not conclude based on univariate or multivariate analysis (as we assumed that the data for univariate analysis or crude estimate may have biased factors), but final results were assessed mainly with Discrepancy of Odds Ratios and Inappropriate Presentation of the Results and Selection of Covariates Used in Multivariate Model

Letter to the Editor: Lack of requirements for conducting meta-analysis to evaluate the association between the TLR2 Arg753Gln polymorphism and the risk of sepsis

We read the article published recently in Critical Care by Gao et al. [1] with great interest and appreciate their effort to evaluate the association between the TLR2 Arg753Gln polymorphism and the risk of sepsis. However, we would like to point out a lack of requirements for conducting meta-analysis in this study; quality assessment, appropriate search method and protocol registration. First, the authors did not assess the quality of studies included in their meta-analysis. In general, meta-analysis of studies that are at risk of bias may be seriously misleading. If bias is present in each (or some) of the individual studies, meta-analysis will simply compound the bias of individual studies, and produce a ‘wrong’ result that may be interpreted as having more credibility [2]. Even though some of the included studies have limited information, the authors should perform sensitivity analysis excluding studies with a high risk of bias in order to explore the influence of biased studies. Secondly, the authors explained that all available data related to potential links between the TLR2 Arg753Gln polymorphism and sepsis risk were pooled in this study. However, the authors only used free-text terms in their search strategy. To search all existent studies, it is generally recommended to use a combination of subject terms selected from controlled vocabulary and free-text terms. Using only free-text terms might reduce the search quality. In addition, the authors did not identify unpublished and ongoing studies. International trial registers should be searched to detect publication bias [2]. Lastly, the study was performed partially in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA), the protocol for which is not registered in the International Prospective Register of Systematic Reviews (PROSPERO). Registration of protocol details is now recognized as desirable in order to promote and maintain transparency in the process and to assist in minimizing the risk of selective outcome reporting bias [3]. Therefore, we conclude that Gao et al.’s meta-analysis has the potential for producing incorrect results due to a lack of quality assessment, an inappropriate search to identify relevant studies and an absence of protocol registration.

Risk of Misinterpretation and Limitation of the Diagnostic Accuracy

We read the article by Alan R. Schroeder with great interest, and appreciate the authors’ efforts to assess diagnostic accuracy of urinalysis for urinary tract infection (UTI) with blood stream infection.1 However, we would like to point out 3 concerns. First, we are concerned that the excellent sensitivity may … E-mail: hira\_to\_ber3598{at}yahoo.co.jp