Andrea C. Tricco

Epidemiology and reporting characteristics of systematic reviews.

Background Systematic reviews (SRs) have become increasingly popular to a wide range of stakeholders. We set out to capture a representative cross-sectional sample of published SRs and examine them in terms of a broad range of epidemiological, descriptive, and reporting characteristics, including emerging aspects not previously examined. Methods and Findings We searched Medline for SRs indexed during November 2004 and written in English. Citations were screened and those meeting our inclusion criteria were retained. Data were collected using a 51-item data collection form designed to assess the epidemiological and reporting details and the bias-related aspects of the reviews. The data were analyzed descriptively. In total 300 SRs were identified, suggesting a current annual publication rate of about 2,500, involving more than 33,700 separate studies including one-third of a million participants. The majority (272 [90.7%]) of SRs were reported in specialty journals. Most reviews (213 [71.0%]) were categorized as therapeutic, and included a median of 16 studies involving 1,112 participants. Funding sources were not reported in more than one-third (122 [40.7%]) of the reviews. Reviews typically searched a median of three electronic databases and two other sources, although only about two-thirds (208 [69.3%]) of them reported the years searched. Most (197/295 [66.8%]) reviews reported information about quality assessment, while few (68/294 [23.1%]) reported assessing for publication bias. A little over half (161/300 [53.7%]) of the SRs reported combining their results statistically, of which most (147/161 [91.3%]) assessed for consistency across studies. Few (53 [17.7%]) SRs reported being updates of previously completed reviews. No review had a registration number. Only half (150 [50.0%]) of the reviews used the term “systematic review” or “meta-analysis” in the title or abstract. There were large differences between Cochrane reviews and non-Cochrane reviews in the quality of reporting several characteristics. Conclusions SRs are now produced in large numbers, and our data suggest that the quality of their reporting is inconsistent. This situation might be improved if more widely agreed upon evidence-based reporting guidelines were endorsed and adhered to by authors and journals. These results substantiate the view that readers should not accept SRs uncritically.

Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis

BACKGROUND The effectiveness of quality improvement (QI) strategies on diabetes care remains unclear. We aimed to assess the effects of QI strategies on glycated haemoglobin (HbA(1c)), vascular risk management, microvascular complication monitoring, and smoking cessation in patients with diabetes. METHODS We identified studies through Medline, the Cochrane Effective Practice and Organisation of Care database (from inception to July 2010), and references of included randomised clinical trials. We included trials assessing 11 predefined QI strategies or financial incentives targeting health systems, health-care professionals, or patients to improve management of adult outpatients with diabetes. Two reviewers independently abstracted data and appraised risk of bias. FINDINGS We reviewed 48 cluster randomised controlled trials, including 2538 clusters and 84,865 patients, and 94 patient randomised controlled trials, including 38,664 patients. In random effects meta-analysis, the QI strategies reduced HbA(1c) by a mean difference of 0·37% (95% CI 0·28-0·45; 120 trials), LDL cholesterol by 0·10 mmol/L (0·05-0.14; 47 trials), systolic blood pressure by 3·13 mm Hg (2·19-4·06, 65 trials), and diastolic blood pressure by 1·55 mm Hg (0·95-2·15, 61 trials) versus usual care. We noted larger effects when baseline concentrations were greater than 8·0% for HbA(1c), 2·59 mmol/L for LDL cholesterol, and 80 mm Hg for diastolic and 140 mm Hg for systolic blood pressure. The effectiveness of QI strategies varied depending on baseline HbA(1c) control. QI strategies increased the likelihood that patients received aspirin (11 trials; relative risk [RR] 1·33, 95% CI 1·21-1·45), antihypertensive drugs (ten trials; RR 1·17, 1·01-1·37), and screening for retinopathy (23 trials; RR 1·22, 1·13-1·32), renal function (14 trials; RR 128, 1·13-1·44), and foot abnormalities (22 trials; RR 1·27, 1·16-1·39). However, statin use (ten trials; RR 1·12, 0·99-1·28), hypertension control (18 trials; RR 1·01, 0·96-1·07), and smoking cessation (13 trials; RR 1·13, 0·99-1·29) were not significantly increased. INTERPRETATION Many trials of QI strategies showed improvements in diabetes care. Interventions targeting the system of chronic disease management along with patient-mediated QI strategies should be an important component of interventions aimed at improving diabetes management. Interventions solely targeting health-care professionals seem to be beneficial only if baseline HbA(1c) control is poor. FUNDING Ontario Ministry of Health and Long-term Care and the Alberta Heritage Foundation for Medical Research (now Alberta Innovates--Health Solutions).

A comparison of indirect versus direct measures for assessing physical activity in the pediatric population: A systematic review

BACKGROUND Accurate assessment of physical activity (PA) in children and adolescents is required to establish PA levels, monitor changes and inform public healthy policy. This study systematically reviews the literature to determine the extent of agreement between indirect (e.g., questionnaire) and direct (e.g., accelerometry) assessments of PA in pediatric populations (<or=19 years). METHODS Literature was identified through searching electronic databases (e.g., MEDLINE, EMBASE), websites of relevant organizations and conference abstracts until April 2007. Studies were included if they collected indirect and direct measures of PA in pediatric populations and were reported in English. Quality of included studies was appraised using a modified Downs and Black tool. RESULTS A total of 83 studies were included; 24 describing comparable data and 59 including a correlation analysis. The majority of correlations reported between indirect and direct measures were low-to-moderate (range: -0.56 to 0.89). Overall, 72% of the indirect measures overestimated the directly measured values. Combined gender, as well as male- and female-only data comparing indirect measures to accelerometery, heart rate monitoring or direct observation, all reported an overestimation by indirect method. A similar trend was observed in combined gender data comparing indirect measures with doubly labelled water; however, the opposite trend was observed in the male- and female-only data with a slight underestimation by indirect measure. CONCLUSIONS Substantial discrepancies and moderate correlations between indirect methods and direct measures of assessing PA in pediatric populations are of concern, especially when trying to establish relationships with health outcomes.

Synthesis of recommendations for the assessment and management of low back pain from recent clinical practice guidelines

BACKGROUND CONTEXT Low back pain (LBP) is a prevalent, costly, and challenging condition to manage. Clinicians must choose among numerous assessment and management options. Several recent clinical practice guidelines (CPGs) on LBP have attempted to inform these decisions by evaluating and summarizing the best available supporting evidence. The quality and consistency of recommendations from these CPGs are currently unknown. PURPOSE To conduct a systematic review of recent CPGs and synthesize their recommendations on assessing and managing LBP for clinicians. STUDY DESIGN/SETTING Systematic review. METHODS Literature search using MEDLINE, National Guidelines Clearinghouse, National Institute for Clinical Excellence, Internet search engines, and references of known articles. Only CPGs related to both assessment and management of LBP written in English were eligible; CPGs that summarized evidence from before the year 2000 were excluded. Data related to methods and recommendations for assessment and management of LBP were abstracted independently by two reviewers. Methodological quality was assessed using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument by two reviewers. RESULTS The search uncovered 669 citations, of which 95 were potentially relevant and 10 were included in the review; 6 discussed acute LBP, 6 chronic LBP, and 6 LBP with neurologic involvement. Methods used to develop CPGs varied, but the overall methodological quality was high as defined by AGREE scores. Recommendations for assessment of LBP emphasized the importance of ruling out potentially serious spinal pathology, specific causes of LBP, and neurologic involvement, as well as identifying risk factors for chronicity and measuring the severity of symptoms and functional limitations, through the history, physical, and neurologic examination. Recommendations for management of acute LBP emphasized patient education, with short-term use of acetaminophen, nonsteroidal anti-inflammatory drugs, or spinal manipulation therapy. For chronic LBP, the addition of back exercises, behavioral therapy, and short-term opioid analgesics was suggested. Management of LBP with neurologic involvement was similar, with additional consideration given to magnetic resonance imaging or computed tomography to identify appropriate candidates willing to undergo epidural steroid injections or decompression surgery if more conservative approaches are not successful. CONCLUSIONS Recommendations from several recent CPGs regarding the assessment and management of LBP were similar. Clinicians who care for patients with LBP should endeavor to adopt these recommendations to improve patient care. Future CPGs may wish to invite coauthors from targeted clinician user groups, increase patient participation, update their literature searches before publication, conduct their own quality assessment of studies, and consider cost-effectiveness and other aspects in their recommendations more explicitly.

A systematic review of treatments for mild traumatic brain injury.

Objectives: To assess the effectiveness of interventions for mild traumatic brain injury (MTBI) in adults as found in the literature. Research design: Systematic review of the literature. Methods: Six electronic databases and 18 journals within the brain injury field were manually searched between the years 1980–2003. References from articles were scanned for further literature. Studies that met broad inclusion criteria were subjected to a formal test of relevance. Those found to be relevant were qualitatively tested for their methodological soundness. Results: One thousand and fifty-five studies were initially identified and 163 were assessed using the relevance tool, yielding 20 studies for review. Four categories of interventions were identified: Pharmacotherapy, Cognitive Rehabilitation, Patient Education and Other. The majority of studies were weak, however there is evidence to support the effectiveness of patient education interventions. Conclusions: There are few rigorous studies evaluating treatment of MTBI. Limitations of the current literature are presented.

Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis

BackgroundInfluenza vaccines are most effective when the antigens in the vaccine match those of circulating strains. However, antigens contained in the vaccines do not always match circulating strains. In the present work we aimed to examine the vaccine efficacy (VE) afforded by influenza vaccines when they are not well matched to circulating strains.MethodsWe identified randomized clinical trials (RCTs) through MEDLINE, EMBASE, the Cochrane Library, and references of included RCTs. RCTs reporting laboratory-confirmed influenza among healthy participants vaccinated with antigens of matching and non-matching influenza strains were included. Two independent reviewers screened citations/full-text articles, abstracted data, and appraised risk of bias. Conflicts were resolved by discussion. A random effects meta-analysis was conducted. VE was calculated using the following formula: (1 - relative risk × 100%).ResultsWe included 34 RCTs, providing data on 47 influenza seasons and 94,821 participants. The live-attenuated influenza vaccine (LAIV) showed significant protection against mismatched (six RCTs, VE 54%, 95% confidence interval (CI) 28% to 71%) and matched (seven RCTs, VE 83%, 95% CI 75% to 88%) influenza strains among children aged 6 to 36 months. Differences were observed between the point estimates for mismatched influenza A (five RCTs, VE 75%, 95% CI 41% to 90%) and mismatched influenza B (five RCTs, VE 42%, 95% CI 22% to 56%) estimates among children aged 6 to 36 months. The trivalent inactivated vaccine (TIV) also afforded significant protection against mismatched (nine RCTs, VE 52%, 95% CI 37% to 63%) and matched (eight RCTs, VE 65%, 95% CI 54% to 73%) influenza strains among adults. Numerical differences were observed between the point estimates for mismatched influenza A (five RCTs, VE 64%, 95% CI 23% to 82%) and mismatched influenza B (eight RCTs, VE 52%, 95% CI 19% to 72%) estimates among adults. Statistical heterogeneity was low (I2 <50%) across all meta-analyses, except for the LAIV meta-analyses among children (I2 = 79%).ConclusionsThe TIV and LAIV vaccines can provide cross protection against non-matching circulating strains. The point estimates for VE were different for matching versus non-matching strains, with overlapping CIs.

Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study.

Background Systematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs. Methods and Findings We searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics. Conclusions An increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.

What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review

BackgroundA knowledge synthesis attempts to summarize all pertinent studies on a specific question, can improve the understanding of inconsistencies in diverse evidence, and can identify gaps in research evidence to define future research agendas. Knowledge synthesis activities in healthcare have largely focused on systematic reviews of interventions. However, a wider range of synthesis methods has emerged in the last decade addressing different types of questions (e.g., realist synthesis to explore mediating mechanisms and moderators of interventions). Many different knowledge synthesis methods exist in the literature across multiple disciplines, but locating these, particularly for qualitative research, present challenges. There is a need for a comprehensive manual for synthesis methods (quantitative/qualitative or mixed), outlining how these methods are related, and how to match the most appropriate knowledge synthesis method to answer a research question. The objectives of this scoping review are to: 1) conduct a systematic search of the literature for knowledge synthesis methods across multi-disciplinary fields; 2) compare and contrast the different knowledge synthesis methods; and, 3) map out the specific steps to conducting the knowledge syntheses to inform the development of a knowledge synthesis methods manual/tool.MethodsWe will search relevant electronic databases (e.g., MEDLINE, CINAHL), grey literature, and discipline-based listservs. The scoping review will consider all study designs including qualitative and quantitative methodologies (excluding economic analysis or clinical practice guideline development), and identify knowledge synthesis methods across the disciplines of health, education, sociology, and philosophy. Two reviewers will pilot-test the screening criteria and data abstraction forms, and will independently screen the literature and abstract the data. A three-step synthesis process will be used to map the literature to our objectives.DiscussionThis project represents the first attempt to broadly and systematically identify, define and classify knowledge synthesis methods (i.e., less traditional knowledge synthesis methods). We anticipate that our results will lead to an accepted taxonomy for less traditional knowledge synthesis methods, and to the development and implementation of a methods manual for these reviews which will be relevant to a wide range of knowledge users, including researchers, funders, and journal editors.

Transfer of learning and patient outcome in simulated crisis resource management: a systematic review

PurposeSimulation-based learning is increasingly used by healthcare professionals as a safe method to learn and practice non-technical skills, such as communication and leadership, required for effective crisis resource management (CRM). This systematic review was conducted to gain a better understanding of the impact of simulation-based CRM teaching on transfer of learning to the workplace and subsequent changes in patient outcomes.SourceStudies on CRM, crisis management, crew resource management, teamwork, and simulation published up to September 2012 were searched in MEDLINE®, EMBASE™, CINAHL, Cochrane Central Register of Controlled Trials, and ERIC. All studies that used simulation-based CRM teaching with outcomes measured at Kirkpatrick Level 3 (transfer of learning to the workplace) or 4 (patient outcome) were included. Studies measuring only learners’ reactions or simple learning (Kirkpatrick Level 1 or 2, respectively) were excluded. Two authors independently reviewed all identified titles and abstracts for eligibility.Principal findingsNine articles were identified as meeting the inclusion criteria. Four studies measured transfer of simulation-based CRM learning into the clinical setting (Kirkpatrick Level 3). In three of these studies, simulation-enhanced CRM training was found significantly more effective than no intervention or didactic teaching. Five studies measured patient outcomes (Kirkpatrick Level 4). Only one of these studies found that simulation-based CRM training made a clearly significant impact on patient mortality.ConclusionsBased on a small number of studies, this systematic review found that CRM skills learned at the simulation centre are transferred to clinical settings, and the acquired CRM skills may translate to improved patient outcomes, including a decrease in mortality.RésuméObjectifL’apprentissage basé sur des simulations est de plus en plus utilisé par les professionnels de santé comme méthodes sécuritaires d’apprentissage et de pratique de compétences non techniques, comme la communication et le leadership, qui sont nécessaires pour une gestion efficace des ressources en situation de crise (CRM). Cette étude systématique a été menée pour mieux comprendre l’impact de l’enseignement à partir de simulations de la CRM sur le transfert des connaissances sur le lieu de travail et les changements ultérieurs sur l’évolution des patients.SourceLes études sur la CRM, gestion de crise, gestion de ressources d’équipes, travail d’équipe et simulation, publiées jusqu’en septembre 2012 ont été recherchées dans les bases de données MEDLINE®, EMBASE™, CINAHL, Cochrane Central Register of Controlled Trials et ERIC. Toutes les études utilisant un enseignement de la CRM à partir de simulations avec des résultats mesurés au niveau 3 de Kirkpatrick (transfert de l’apprentissage au milieu de travail) ou au niveau 4 (évolution du patient) ont été incluses. Toutes les études ne mesurant que les réactions des apprenants ou le seul apprentissage (respectivement, niveau 1 ou 2 de Kirkpatrick) ont été exclues. Deux auteurs ont revu de façon indépendante tous les titres et résumés identifiés pour évaluer leur admissibilité.Constatations principalesNeuf articles répondant aux critères d’inclusion ont été identifiés. Quatre études mesuraient le transfert d’apprentissage de la CRM à partir de simulations vers un cadre clinique (niveau 3 de Kirkpatrick). Dans trois de ces études, la formation à la CRM soutenue par des simulations s’est avérée significativement plus efficace que l’absence d’intervention ou un enseignement didactique. Cinq études mesuraient les résultats pour les patients (niveau 4 de Kirkpatrick). Une seule de ces études a trouvé que la formation à la CRM basée sur des simulations avait un impact clairement significatif sur la mortalité des patients.ConclusionsReposant sur un petit nombre d’études, cette analyse systématique a trouvé que les habiletés en matière de CRM apprises au centre de simulations sont transférées dans des cadres cliniques et que les habiletés acquises de CRM peuvent se traduire par une amélioration de l’évolution, y compris une baisse de la mortalité.

Harassment and discrimination in medical training: a systematic review and meta-analysis.

Purpose Harassment and discrimination include a wide range of behaviors that medical trainees perceive as being humiliating, hostile, or abusive. To understand the significance of such mistreatment and to explore potential preventive strategies, the authors conducted a systematic review and meta-analysis to examine the prevalence, risk factors, and sources of harassment and discrimination among medical trainees. Method In 2011, the authors identified relevant studies by searching MEDLINE and EMBASE, scanning reference lists of relevant studies, and contacting experts. They included studies that reported the prevalence, risk factors, and sources of harassment and discrimination among medical trainees. Two reviewers independently screened all articles and abstracted study and participant characteristics and study results. The authors assessed the methodological quality in individual studies using the Newcastle–Ottawa Scale. They also conducted a meta-analysis. Results The authors included 57 cross-sectional and 2 cohort studies in their review. The meta-analysis of 51 studies demonstrated that 59.4% of medical trainees had experienced at least one form of harassment or discrimination during their training (95% confidence interval [CI]: 52.0%–66.7%). Verbal harassment was the most commonly cited form of harassment (prevalence: 63.0%; 95% CI: 54.8%–71.2%). Consultants were the most commonly cited source of harassment and discrimination, followed by patients or patients’ families (34.4% and 21.9%, respectively). Conclusions This review demonstrates the surprisingly high prevalence of harassment and discrimination among medical trainees that has not declined over time. The authors recommend both drafting policies and promoting cultural change within academic institutions to prevent future abuse.