Lorenzo Moja

Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study

Abstract Objectives To describe how the methodological quality of primary studies is assessed in systematic reviews and whether the quality assessment is taken into account in the interpretation of results. Data sources Cochrane systematic reviews and systematic reviews in paper based journals. Study selection 965 systematic reviews (809 Cochrane reviews and 156 paper based reviews) published between 1995 and 2002. Data synthesis The methodological quality of primary studies was assessed in 854 of the 965 systematic reviews (88.5%). This occurred more often in Cochrane reviews than in paper based reviews (93.9% v 60.3%, P < 0.0001). Overall, only 496 (51.4%) used the quality assessment in the analysis and interpretation of the results or in their discussion, with no significant differences between Cochrane reviews and paper based reviews (52% v 49%, P = 0.58). The tools and methods used for quality assessment varied widely. Conclusions Cochrane reviews fared better than systematic reviews published in paper based journals in terms of assessment of methodological quality of primary studies, although they both largely failed to take it into account in the interpretation of results. Methods for assessment of methodological quality by systematic reviews are still in their infancy and there is substantial room for improvement.

Timing Matters in Hip Fracture Surgery: Patients Operated within 48 Hours Have Better Outcomes. A Meta-Analysis and Meta-Regression of over 190,000 Patients

Background To assess the relationship between surgical delay and mortality in elderly patients with hip fracture. Systematic review and meta-analysis of retrospective and prospective studies published from 1948 to 2011. Medline (from 1948), Embase (from 1974) and CINAHL (from 1982), and the Cochrane Library. Odds ratios (OR) and 95% confidence intervals for each study were extracted and pooled with a random effects model. Heterogeneity, publication bias, Bayesian analysis, and meta-regression analyses were done. Criteria for inclusion were retro- and prospective elderly population studies, patients with operated hip fractures, indication of timing of surgery and survival status. Methodology/Principal Findings There were 35 independent studies, with 191,873 participants and 34,448 deaths. The majority considered a cut-off between 24 and 48 hours. Early hip surgery was associated with a lower risk of death (pooled odds ratio (OR) 0.74, 95% confidence interval (CI) 0.67 to 0.81; P<0.000) and pressure sores (0.48, 95% CI 0.38 to 0.60; P<0.000). Meta-analysis of the adjusted prospective studies gave similar results. The Bayesian probability predicted that about 20% of future studies might find that early surgery is not beneficial for decreasing mortality. None of the confounders (e.g. age, sex, data source, baseline risk, cut-off points, study location, quality and year) explained the differences between studies. Conclusions/Significance Surgical delay is associated with a significant increase in the risk of death and pressure sores. Conservative timing strategies should be avoided. Orthopaedic surgery services should ensure the majority of patients are operated within one or two days.

Probiotics supplementation during pregnancy or infancy for the prevention of atopic dermatitis: A meta-analysis

Background: The study of probiotics to prevent allergic conditions has yielded conflicting results in children. We undertook a meta-analysis of randomized controlled trials to investigate whether probiotic use during pregnancy and early life decreases the incidence of atopic dermatitis and immunoglobulin E (IgE)-associated atopic dermatitis in infants and young children. Methods: We performed a systematic literature search in Medline, Embase, and Cochrane Library, updated to October 2011. The intervention was diet supplementation with probiotics versus placebo. Primary outcomes were incidence of atopic dermatitis and IgE-associated atopic dermatitis. We calculated summary relative risks (RRs) and corresponding 95% confidence intervals (CIs), using both fixed- and random-effects models. We computed summary estimates across several strata, including study period, type of patient, dose, and duration of intervention, and we assessed the risk of bias within and across trials. Results: We identified 18 publications based on 14 studies. Meta-analysis demonstrated that probiotic use decreased the incidence of atopic dermatitis (RR = 0.79 [95% CI = 0.71−0.88]). Studies were fairly homogeneous (I2 = 24.0%). The corresponding RR of IgE-associated atopic dermatitis was 0.80 (95% CI = 0.66−0.96). No appreciable difference emerged across strata, nor was there evidence of publication bias. Conclusions: This meta-analysis provided evidence in support of a moderate role of probiotics in the prevention of atopic dermatitis and IgE-associated atopic dermatitis in infants. The favorable effect was similar regardless of the time of probiotic use (pregnancy or early life) or the subject(s) receiving probiotics (mother, child, or both).

Prevalence, Characteristics, and Publication of Discontinued Randomized Trials

IMPORTANCE The discontinuation of randomized clinical trials (RCTs) raises ethical concerns and often wastes scarce research resources. The epidemiology of discontinued RCTs, however, remains unclear. OBJECTIVES To determine the prevalence, characteristics, and publication history of discontinued RCTs and to investigate factors associated with RCT discontinuation due to poor recruitment and with nonpublication. DESIGN AND SETTING Retrospective cohort of RCTs based on archived protocols approved by 6 research ethics committees in Switzerland, Germany, and Canada between 2000 and 2003. We recorded trial characteristics and planned recruitment from included protocols. Last follow-up of RCTs was April 27, 2013. MAIN OUTCOMES AND MEASURES Completion status, reported reasons for discontinuation, and publication status of RCTs as determined by correspondence with the research ethics committees, literature searches, and investigator surveys. RESULTS After a median follow-up of 11.6 years (range, 8.8-12.6 years), 253 of 1017 included RCTs were discontinued (24.9% [95% CI, 22.3%-27.6%]). Only 96 of 253 discontinuations (37.9% [95% CI, 32.0%-44.3%]) were reported to ethics committees. The most frequent reason for discontinuation was poor recruitment (101/1017; 9.9% [95% CI, 8.2%-12.0%]). In multivariable analysis, industry sponsorship vs investigator sponsorship (8.4% vs 26.5%; odds ratio [OR], 0.25 [95% CI, 0.15-0.43]; P < .001) and a larger planned sample size in increments of 100 (-0.7%; OR, 0.96 [95% CI, 0.92-1.00]; P = .04) were associated with lower rates of discontinuation due to poor recruitment. Discontinued trials were more likely to remain unpublished than completed trials (55.1% vs 33.6%; OR, 3.19 [95% CI, 2.29-4.43]; P < .001). CONCLUSIONS AND RELEVANCE In this sample of trials based on RCT protocols from 6 research ethics committees, discontinuation was common, with poor recruitment being the most frequently reported reason. Greater efforts are needed to ensure the reporting of trial discontinuation to research ethics committees and the publication of results of discontinued trials.

Effectiveness of Computerized Decision Support Systems Linked to Electronic Health Records: A Systematic Review and Meta-Analysis

We systematically reviewed randomized controlled trials (RCTs) assessing the effectiveness of computerized decision support systems (CDSSs) featuring rule- or algorithm-based software integrated with electronic health records (EHRs) and evidence-based knowledge. We searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Abstracts of Reviews of Effects. Information on system design, capabilities, acquisition, implementation context, and effects on mortality, morbidity, and economic outcomes were extracted. Twenty-eight RCTs were included. CDSS use did not affect mortality (16 trials, 37395 patients; 2282 deaths; risk ratio [RR] = 0.96; 95% confidence interval [CI] = 0.85, 1.08; I(2) = 41%). A statistically significant effect was evident in the prevention of morbidity, any disease (9 RCTs; 13868 patients; RR = 0.82; 95% CI = 0.68, 0.99; I(2) = 64%), but selective outcome reporting or publication bias cannot be excluded. We observed differences for costs and health service utilization, although these were often small in magnitude. Across clinical settings, new generation CDSSs integrated with EHRs do not affect mortality and might moderately improve morbidity outcomes.

Biological Agents for Moderately to Severely Active Ulcerative Colitis: A Systematic Review and Network Meta-analysis

Ulcerative colitis (UC) is an idiopathic, remitting and relapsing, chronic inflammatory bowel disease. It is characterized by mucosal ulceration, rectal bleeding, diarrhea, abdominal cramps, urgency or tenesmus, fever, malaise, weight loss and fatigue, depending on the extent and severity of the disease. Worldwide, the estimated incidence of UC ranges from 1.2 to 20.3 cases per 100000 person-years and its prevalence ranges from 7.6 to 246.0 per 100000 persons (14). Pharmacologic management of UC aims at reducing inflammation and maintaining remission of symptoms and includes sulfasalazine, 5-aminosalicylates (mesalamine, olsalazine, and balsalazide), glucocorticoids, and immunosuppressants (azathioprine, 6-mercaptopurine, and cyclosporine) (5, 6). Despite progress, treatment options for moderately to severely active UC remain limited because conventional therapies inadequately control the disease in a substantial proportion of patients and often lead to adverse events (AEs). However, a series of biological agents have recently received regulatory approval or are under scrutiny; in 2013, those included the monoclonal antibodies adalimumab, golimumab, infliximab, and vedolizumab. Evidence on comparative effectiveness and harms of those treatments would be very useful to inform clinical decision making. We conducted a systematic review of randomized, controlled trials (RCTs) assessing biological agents as induction or maintenance therapy for moderately to severely active UC in adults to address this issue. We assessed their comparative clinical efficacy and harm using the method of multiple-treatment meta-analysis, also known as network meta-analysis or mixed-treatment comparison (710), based on the available evidence from RCTs. We aimed to compare the different treatment options and provide a clinically useful summary that can be used to support optimal decision making. Methods Our study protocol (11) was registered on PROSPERO (CRD42013005459). We followed standard methods for conducting and reporting systematic reviews and network meta-analyses (12, 13). Data Sources and Searches We systematically searched MEDLINE and EMBASE databases from inception to 31 December 2013. Search terms included biologic(al) agent(s), biologic(s), adalimumab, golimumab, infliximab, or vedolizumab, combined with ulcerative colitis. The search was limited to RCTs and humans. Language or age restrictions were not imposed. We also searched the Cochrane Library for any recent systematic review on the subject, the ClinicalTrials.gov database for completed but unpublished studies, and the European Medicines Agency and U.S. Food and Drug Administration Web sites to obtain details on study characteristics or outcomes if these data were missing or unclearly presented in the original articles. Study Selection The titles and abstracts of identified published articles were scanned to exclude irrelevant studies. The full text of the selected articles was retrieved and read. The bibliographies of the articles and of reviews and meta-analyses were scanned. We further asked field experts to provide additional evidence. Studies were eligible for inclusion if they were randomized, placebo-controlled or head-to-head trials assessing the efficacy or harm of biological agents for the treatment of adult patients with moderately to severely active UC. We considered only biological agents that were market-authorized by either the U.S. Food and Drug Administration or European Medicines Agency, as well as agents under review for the new drug application (United States) or marketing authorisation application (European Union). We defined moderately to severely active UC as a Mayo Clinic score (MCS) of 6 to 12 points, with an endoscopic subscore of 2 or 3. An MCS is a composite activity index ranging from 0 to 12, with higher scores indicating more severe disease activity. It is calculated as the sum of 4 items: stool frequency, rectal bleeding, endoscopic findings, and physicians global assessment (14). Randomized, controlled trials were eligible for inclusion in the network regardless of country, phase (2 or 3), or support. Data Extraction and Quality Assessment Two reviewers abstracted the data independently. The following information was collected from each study: publication data; trials acronym and identifier; first authors last name; geographic location of study; year of publication; study design; number of participants and population characteristics; and interventions variables, including drug, dose, and administration. We extracted data specific for patients who were naive to treatment with biological agents because the relative efficacy of biological agents should not be assessed by pooling patients for whom standard therapies were unsuccessful with patients for whom biological agents were unsuccessful (15, 16). Different doses of the same treatment were treated as separate interventions, and for biological agents that received regulatory approval, only data for dose and administration as approved in the respective summary of product characteristics were considered. The outcome measures were the odds ratios (ORs) for clinical response (primary outcome), clinical remission, and mucosal healing at the end of induction and at completion of each trials maintenance phase, calculated in accordance with the intention-to-treat principle (that is, total number of randomly assigned participants, regardless of how the original study investigators analyzed the data). Clinical response was defined as a decrease from baseline in the MCS of at least 3 points and at least 30%, with an accompanying decrease in the subscore for rectal bleeding of at least 1 point, or an absolute rectal bleeding subscore of 0 or 1. Clinical remission was defined as an MCS of 2 points or lower, with no individual subscore exceeding 1 point. Mucosal healing was defined as an absolute subscore for endoscopy of 0 or 1. We also examined the occurrence of serious adverse events (SAEs), which are defined as any untoward medical occurrence that results in death, requires hospital admission or prolongation of an existing hospital stay, causes persistent or significant disability or incapacity, or is life-threatening (17); the AEs leading to discontinuation of the study drug; the total number of AEs; the total number of infectious AEs; the number of serious infections; tuberculosis; and congestive heart failure. Reviewers assessed the risk of bias in the results of included studies by using the Cochrane Collaborations tool (18), which addresses the following key domains: sequence generation, allocation concealment, blinding of participants and personnel, incomplete outcome data, selective outcome reporting, and other sources of bias. These items are considered as key domains for risk-of-bias assessment and reclassified as adequate (low risk of bias), inadequate (high risk of bias), or unclear. Studies with adequate procedures in all domains are considered to have a low risk of bias, ones with inadequate procedures in 1 or more domains are considered to have a high risk of bias, and those with unclear procedures in 1 or more domains are considered to have unclear risk of bias. Disagreements among reviewers were discussed and agreement was reached by consensus. Data Synthesis and Analysis We conducted the network meta-analysis within a Bayesian framework using Markov chain Monte Carlo methods in WinBUGS (Medical Research Council Biostatistics Unit, Cambridge, United Kingdom) (19). Analysis was based on noninformative priors for relative-effect parameters (flat normal with mean of 0 and precision of 0.001) and between-study SD (a flat uniform distribution between 0 and 2). Convergence and lack of autocorrelation were checked and confirmed after a 5000-simulation burn-in phase without any thinning and using 4 chains with different initial values. Then, a burn-in phase of 20000 iterations was used, followed by 50000 iterations to estimate parameters. We did sensitivity analyses by setting the prior on the heterogeneity equal to a uniform (0, 100) and on the precision parameter equal to a (0.001, 0.001) to check the robustness of the model to different sets of priors. Model choice was based on the deviance information criterion, and a lower deviance information criterion by 5 or more units suggested a better model fit. We report estimates only for the default model because we did not find differences in model fit according to deviance information criterion. We used the resulting ORs and 95% credible intervals (CrIs), the Bayesian equivalent to CIs, to assess treatment effects. We also calculated fixed-effects ORs and 95% CIs (frequentist approach). Given that each pairwise comparison included a limited number of RCTs, we could not formally assess statistical heterogeneity and publication bias. Role of the Funding Source This study was funded by the Centro Ricerca e Cura delle Malattie Infiammatorie Croniche Intestinali, IRCCS Istituto Clinico Humanitas. The funding source had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to submit the manuscript for publication. Results Figure 1 summarizes the search and selection of evidence. We identified 6 publications (2025) reporting the results of 7 trials (ACT [Active Ulcerative Colitis Trial] 1, ACT 2, ULTRA [Ulcerative Colitis Long-Term Remission and Maintenance With Adalimumab] 1, ULTRA 2, PURSUIT-SC [Program of Ulcerative Colitis Research Studies Utilizing an Investigational TreatmentSubcutaneous], PURSUIT-M [Program of Ulcerative Colitis Research Studies Utilizing an Investigational TreatmentMaintenance], and GEMINI 1). One additional study, the NCT00853099 trial, was initially identified through ClinicalTrials.gov (26) and later in full-text publication (27). In total, 8 trials met the eligibility criteria, none of which were head-to-head comparisons of biological agents. Figure 1. Summ

A Review of Online Evidence-based Practice Point-of-Care Information Summary Providers

Background Busy clinicians need easy access to evidence-based information to inform their clinical practice. Publishers and organizations have designed specific tools to meet doctors’ needs at the point of care. Objective The aim of this study was to describe online point-of-care summaries and evaluate their breadth, content development, and editorial policy against their claims of being “evidence-based.” Methods We searched Medline, Google, librarian association websites, and information conference proceedings from January to December 2008. We included English Web-based point-of-care summaries designed to deliver predigested, rapidly accessible, comprehensive, periodically updated, evidence-based information to clinicians. Two investigators independently extracted data on the general characteristics and content presentation of summaries. We assessed and ranked point-of-care products according to: (1) coverage (volume) of medical conditions, (2) editorial quality, and (3) evidence-based methodology. We explored how these factors were associated. Results We retrieved 30 eligible summaries. Of these products, 18 met our inclusion criteria and were qualitatively described, and 16 provided sufficient data for quantitative evaluation. The median volume of medical conditions covered was 80.6% (interquartile range, 68.9% - 84.2%) and varied for the different products. Similarly, differences emerged for editorial policy (median 8.0, interquartile range 5.8 - 10.3) and evidence-based methodology scores (median 10.0, interquartile range 1.0 - 12.8) on a 15-point scale. None of these dimensions turned out to be significantly associated with the other dimensions (editorial quality and volume, Spearman rank correlation r = -0.001, P = .99; evidence-based methodology and volume, r = -0.19, P = .48; editorial and evidence-based methodology, r = 0.43, P =.09). Conclusions Publishers are moving to develop point-of-care summary products. Some of these have better profiles than others, and there is room for improved reporting of the strengths and weaknesses of these products.

Speed of updating online evidence based point of care summaries: Prospective cohort analysis

Objective To evaluate the ability of international point of care information summaries to update evidence relevant to medical practice. Design Prospective cohort bibliometric analysis. Setting Top five point of care information summaries (Clinical Evidence, EBMGuidelines, eMedicine, Dynamed, UpToDate) ranked for coverage of medical conditions, editorial quality, and evidence based methodology. Main outcome measures From June 2009 to May 2010 we measured the incidence of research findings relating to potentially eligible newsworthy evidence. As samples, we chose systematic reviews rated as relevant by international research networks (such as, Evidence-Based Medicine, ACP Journal Club, and the Cochrane Collaboration). Every month we assessed whether each sampled review was cited in at least one chapter of the five summaries. The cumulative updating rate was analysed with Kaplan-Meier curves. Results From April to December 2009, 128 reviews were retrieved; 53% (68) from the literature surveillance journals and 47% (60) from the Cochrane Library. At nine months, Dynamed had cited 87% of the sampled reviews, while the other summaries had cited less than 50%. The updating speed of Dynamed clearly led the others. For instance, the hazard ratios for citations in EBM Guidelines and Clinical Evidence versus the top performer were 0.22 (95% confidence interval 0.17 to 0.29) and 0.03 (0.01 to 0.05). Conclusions Point of care information summaries include evidence relevant to practice at different speeds. A qualitative analysis of updating mechanisms is needed to determine whether greater speed corresponds to more appropriate incorporation of new information.

Barriers toward epilepsy surgery. A survey among practicing neurologists

Purpose:  Guidelines for refractory epilepsy recommend timely referral of potential surgical candidates to an epilepsy center for evaluation. However, this approach is seldom a priority for treating neurologists, possibly because of inertia of previous practice and personal attitudes, leading to a buildup of psychosocial disabilities and increased risk of morbidity and mortality. The aim of this study was to assess knowledge and attitudes toward epilepsy surgery among practicing neurologists and identify the barriers that delay the treatment.

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications

Objective To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications. Design Cohort of protocols of randomised controlled trial and subsequent full journal publications. Setting Six research ethics committees in Switzerland, Germany, and Canada. Data sources 894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications. Results Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis. Conclusions Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.