Thomas A Trikalinos

Replication validity of genetic association studies

The rapid growth of human genetics creates countless opportunities for studies of disease association. Given the number of potentially identifiable genetic markers and the multitude of clinical outcomes to which these may be linked, the testing and validation of statistical hypotheses in genetic epidemiology is a task of unprecedented scale. Meta-analysis provides a quantitative approach for combining the results of various studies on the same topic, and for estimating and explaining their diversity. Here, we have evaluated by meta-analysis 370 studies addressing 36 genetic associations for various outcomes of disease. We show that significant between-study heterogeneity (diversity) is frequent, and that the results of the first study correlate only modestly with subsequent research on the same association. The first study often suggests a stronger genetic effect than is found by subsequent studies. Both bias and genuine population diversity might explain why early association studies tend to overestimate the disease protection or predisposition conferred by a genetic polymorphism. We conclude that a systematic meta-analytic approach may assist in estimating population-wide effects of genetic risk factors in human disease.

Systematic Review: Vitamin D and Cardiometabolic Outcomes

BACKGROUND Vitamin D may modify risk for cardiometabolic outcomes (type 2 diabetes, hypertension, or cardiovascular disease). PURPOSE To examine the association between vitamin D status, including the effect of vitamin D supplementation, and cardiometabolic outcomes in generally healthy adults. DATA SOURCES English-language studies in MEDLINE (inception to 4 November 2009) and the Cochrane Central Register of Controlled Trials (fourth quarter of 2009). STUDY SELECTION 11 reviewers screened citations to identify longitudinal cohort studies that reported associations between vitamin D status and cardiometabolic outcomes, including randomized trials of vitamin D supplementation. DATA EXTRACTION 5 independent reviewers extracted data about study conduct, participant characteristics, outcomes, and quality. Differences were resolved by consensus. DATA SYNTHESIS 13 observational studies (14 cohorts) and 18 trials were eligible. Three of 6 analyses (from 4 different cohorts) reported a lower incident diabetes risk in the highest versus the lowest vitamin D status groups. Eight trials found no effect of vitamin D supplementation on glycemia or incident diabetes. In meta-analysis of 3 cohorts, lower 25-hydroxyvitamin D concentration was associated with incident hypertension (relative risk, 1.8 [95% CI, 1.3 to 2.4]). In meta-analyses of 10 trials, supplementation nonsignificantly reduced systolic blood pressure (weighted mean difference, -1.9 mm Hg [CI, -4.2 to 0.4 mm Hg]) and did not affect diastolic blood pressure (weighted mean difference, -0.1 mm Hg [CI, -0.7 to 0.5 mm Hg]). Lower 25-hydroxyvitamin D concentration was associated with incident cardiovascular disease in 5 of 7 analyses (6 cohorts). Four trials found no effect of supplementation on cardiovascular outcomes. LIMITATIONS Studies included primarily white participants. Observational studies were heterogeneous. Several trials reported post hoc analyses. CONCLUSION The association between vitamin D status and cardiometabolic outcomes is uncertain. Trials showed no clinically significant effect of vitamin D supplementation at the dosages given. PRIMARY FUNDING SOURCE National Institute of Diabetes and Digestive and Kidney Disease, the National Institutes of Health Office of Dietary Supplements, U.S. Food and Drug Administration, Agency for Healthcare Research and Quality, and Public Health Agency of Canada.

Genetic associations in large versus small studies: an empirical assessment

BACKGROUND Advances in human genetics could help us to assess prognosis on an individual basis and to optimise the management of complex diseases. However, different studies on the same genetic association sometimes have discrepant results. Our aim was to assess how often large studies arrive at different conclusions than smaller studies, and whether this situation arises more frequently when findings of first published studies disagree with those of subsequent research. METHODS We examined the results of 55 meta-analyses (579 study comparisons) of genetic associations and tested whether the magnitude of the genetic effect differs in large versus smaller studies. FINDINGS We noted significant between-study heterogeneity in 26 (47%) meta-analyses. The magnitude of the genetic effect differed significantly in large versus smaller studies in ten (18%), 20 (36%), and 21 (38%) meta-analyses with tests of rank correlation, regression on SE, and regression on inverse of variance, respectively. The largest studies generally yielded more conservative results than the complete meta-analyses, which included all studies (p=0.005). In 14 (26%) meta-analyses the proposed association was significantly stronger in the first studies than in subsequent research. Only in nine (16%) meta-analyses was the genetic association significant and replicated without hints of heterogeneity or bias. There was little concordance in first versus subsequent discrepancies, and large versus small discrepancies. INTERPRETATION Genuine heterogeneity and bias could affect the results of genetic association studies. Genetic risk factors for complex diseases should be assessed cautiously and, if possible, using large scale evidence.

The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey

Background: Statistical tests for funnel-plot asymmetry are common in meta-analyses. Inappropriate application can generate misleading inferences about publication bias. We aimed to measure, in a survey of meta-analyses, how frequently the application of these tests would be not meaningful or inappropriate. Methods: We evaluated all meta-analyses of binary outcomes with é 3 studies in the Cochrane Database of Systematic Reviews (2003, issue 2). A separate, restricted analysis was confined to the largest meta-analysis in each of the review articles. In each meta-analysis, we assessed whether criteria to apply asymmetry tests were met: no significant heterogeneity, I2 < 50%, é 10 studies (with statistically significant results in at least 1) and ratio of the maximal to minimal variance across studies > 4. We performed a correlation and 2 regression asymmetry tests and evaluated their concordance. Finally, we sampled 60 meta-analyses from print journals in 2005 that cited use of the standard regression test. Results: A total of 366 of 6873 (5%) and 98 of 846 meta-analyses (12%) in the wider and restricted Cochrane data set, respectively, would have qualified for use of asymmetry tests. Asymmetry test results were significant in 7%–18% of the meta-analyses. Concordance between the 3 tests was modest (estimated k 0.33–0.66). Of the 60 journal meta-analyses, 7 (12%) would qualify for asymmetry tests; all 11 claims for identification of publication bias were made in the face of large and significant heterogeneity. Interpretation: Statistical conditions for employing asymmetry tests for publication bias are absent from most meta-analyses; yet, in medical journals these tests are performed often and interpreted erroneously.

Differential Effects of NOD2 Variants on Crohn's Disease Risk and Phenotype in Diverse Populations: A Metaanalysis

OBJECTIVES:Three variants of the CARD15/NOD2 gene (SNP8, SNP12, and SNP13) have been associated with Crohns disease (CD). We assessed the impact of NOD2 variants on the CD risk across diverse populations and examined possible associations with disease phenotype.METHODS:We performed a metaanalysis searching MEDLINE and EMBASE (last search 05/2004) and contacting field experts.RESULTS:Forty-two eligible studies contributed data on 206 comparisons. No variants were detected in Asians. In non-Jewish descent Caucasians carriage of SNP8, SNP12, or SNP13 had an odds ratio (OR) for CD of 2.20 (95% CI: 1.84–2.62), 2.99 (95% CI: 2.38–3.74), and 4.09 (95% CI: 3.23–5.18), respectively. For Jewish descent patients the corresponding ORs were 1.74, 1.93, and 2.45, respectively. The OR in carriers of at least two alleles was 17.1 (95% CI: 10.7–27.2). Large studies tended to yield more conservative estimates than smaller studies, so publication or other bias cannot be excluded. Among CD patients, carrying at least one high-risk variant increased slightly the risk for familial disease (OR = 1.49, (95% CI: 1.18–1.87)), modestly the risk of stenosing CD (OR = 1.94, (95% CI: 1.61–2.34)), and more prominently the risk of small bowel involvement (OR = 2.53, (95% CI: 2.01–3.16)).CONCLUSIONS:SNP8, SNP12, and SNP13 have differential effects on CD risk, with SNP13 having the strongest genetic effect. These NOD2 variants are also significant risk factors for CD phenotype, in particular ileal location.

Meta-Analyst: software for meta-analysis of binary, continuous and diagnostic data

BackgroundMeta-analysis is increasingly used as a key source of evidence synthesis to inform clinical practice. The theory and statistical foundations of meta-analysis continually evolve, providing solutions to many new and challenging problems. In practice, most meta-analyses are performed in general statistical packages or dedicated meta-analysis programs.ResultsHerein, we introduce Meta-Analyst, a novel, powerful, intuitive, and free meta-analysis program for the meta-analysis of a variety of problems. Meta-Analyst is implemented in C# atop of the Microsoft .NET framework, and features a graphical user interface. The software performs several meta-analysis and meta-regression models for binary and continuous outcomes, as well as analyses for diagnostic and prognostic test studies in the frequentist and Bayesian frameworks. Moreover, Meta-Analyst includes a flexible tool to edit and customize generated meta-analysis graphs (e.g., forest plots) and provides output in many formats (images, Adobe PDF, Microsoft Word-ready RTF). The software architecture employed allows for rapid changes to be made to either the Graphical User Interface (GUI) or to the analytic modules.We verified the numerical precision of Meta-Analyst by comparing its output with that from standard meta-analysis routines in Stata over a large database of 11,803 meta-analyses of binary outcome data, and 6,881 meta-analyses of continuous outcome data from the Cochrane Library of Systematic Reviews. Results from analyses of diagnostic and prognostic test studies have been verified in a limited number of meta-analyses versus MetaDisc and MetaTest. Bayesian statistical analyses use the OpenBUGS calculation engine (and are thus as accurate as the standalone OpenBUGS software).ConclusionWe have developed and validated a new program for conducting meta-analyses that combines the advantages of existing software for this task.

Vitamin D With or Without Calcium Supplementation for Prevention of Cancer and Fractures: An Updated Meta-analysis for the U.S. Preventive Services Task Force

BACKGROUND Studies suggest that vitamin D supplementation may reduce cancer and fracture risks. PURPOSE To examine the benefits and harms of vitamin D with or without calcium supplementation on clinical outcomes of cancer and fractures in adults. DATA SOURCES English-language studies identified from MEDLINE and the Cochrane Central Register of Controlled Trials through July 2011. STUDY SELECTION Randomized, controlled trials (RCTs), prospective cohort studies, and nested case-control studies reporting incidence of or death from cancer and fracture outcomes. DATA EXTRACTION Multiple reviewers extracted details about participant characteristics, including baseline vitamin D status and use of supplements; details of statistical analyses, including adjustments for confounding; and methodological quality. Differences were resolved by consensus. DATA SYNTHESIS 19 RCTs (3 for cancer and 16 for fracture outcomes) and 28 observational studies (for cancer outcomes) were analyzed. Limited data from RCTs suggested that high-dose (1000 IU/d) vitamin D supplementation can reduce the risk for total cancer, and data from observational studies suggested that higher blood 25-hydroxyvitamin D (25-[OH]D) concentrations might be associated with increased risk for cancer. Mixed-effects dose-response meta-analyses showed that each 10-nmol/L increase in blood 25-(OH)D concentration was associated with a 6% (95% CI, 3% to 9%) reduced risk for colorectal cancer but no statistically significant dose-response relationships for prostate and breast cancer. Random-effects model meta-analysis showed that combined vitamin D and calcium supplementation reduced fracture risk (pooled relative risk, 0.88 [CI, 0.78 to 0.99]) in older adults, but the effects differed according to study setting: institution (relative risk, 0.71 [CI, 0.57 to 0.89]) versus community-dwelling (relative risk, 0.89 [CI, 0.76 to 1.04]). One RCT showed adverse outcomes associated with supplementation, including increased risk for renal and urinary tract stones. LIMITATIONS Most trial participants were older (aged≥65 years) postmenopausal women. Observational studies were heterogeneous and were limited by potential confounders. CONCLUSION Combined vitamin D and calcium supplementation can reduce fracture risk, but the effects may be smaller among community-dwelling older adults than among institutionalized elderly persons. Appropriate dose and dosing regimens, however, require further study. Evidence is not sufficiently robust to draw conclusions regarding the benefits or harms of vitamin D supplementation for the prevention of cancer. PRIMARY FUNDING SOURCE Agency for Healthcare Research and Quality.

A Summary of the Agency for Healthcare Research and Quality's Evidence Report on Breastfeeding in Developed Countries

OBJECTIVES This article summarizes the Agency for Healthcare Research and Qualitys evidence report on the effects of breastfeeding on term infant and maternal health outcomes in developed countries. EVIDENCE REPORT DATA SOURCES Medline, CINAHL, Cochrane Library, bibliographies of selected reviews, and suggestions from domain experts were surveyed. Searches were limited to English-language publications. EVIDENCE REPORT REVIEW METHODS Eligible comparisons examined the association between differential exposure to breastfeeding and health outcomes. We assessed 15 infant and six maternal outcomes. For four outcomes, we also updated previously published systematic reviews. For the rest of the outcomes, we either summarized previous systematic reviews or conducted new systematic reviews; randomized and non-randomized comparative trials, prospective cohorts, and case-control studies were included. Adjusted estimates were extracted from non-experimental designs. The studies were graded for methodological quality. We did not draw conclusions from poor quality studies. EVIDENCE REPORT RESULTS We screened over 9,000 abstracts. Thirty-two primary studies on term infant health outcomes, 43 primary studies on maternal health outcomes, and 28 systematic reviews or meta-analyses that covered approximately 400 individual studies were included in this review. A history of breastfeeding was associated with a reduction in the risk of acute otitis media, nonspecific gastroenteritis, severe lower respiratory tract infections, atopic dermatitis, asthma (young children), obesity, type 1 and 2 diabetes, childhood leukemia, and sudden infant death syndrome. There was no relationship between breastfeeding in term infants and cognitive performance. There were insufficient good quality data to address the relationship between breastfeeding and cardiovascular diseases and infant mortality. For maternal outcomes, a history of lactation was associated with a reduced risk of type 2 diabetes, breast, and ovarian cancer. Early cessation of breastfeeding or no breastfeeding was associated with an increased risk of maternal postpartum depression. There was no relationship between a history of lactation and the risk of osteoporosis. The effect of breastfeeding in mothers on return-to-prepregnancy weight was negligible, and the effect of breastfeeding on postpartum weight loss was unclear. EVIDENCE REPORT CONCLUSIONS A history of breastfeeding is associated with a reduced risk of many diseases in infants and mothers. Future research would benefit from clearer selection criteria, definitions of breastfeeding exposure, and adjustment for potential confounders. Matched designs such as sibling analysis may provide a method to control for hereditary and household factors that are important in certain outcomes.

An exploratory test for an excess of significant findings

Background The published clinical research literature may be distorted by the pursuit of statistically significant results. Purpose We aimed to develop a test to explore biases stemming from the pursuit of nominal statistical significance. Methods The exploratory test evaluates whether there is a relative excess of formally significant findings in the published literature due to any reason (e.g., publication bias, selective analyses and outcome reporting, or fabricated data). The number of expected studies with statistically significant results is estimated and compared against the number of observed significant studies. The main application uses α = 0.05, but a range of α thresholds is also examined. Different values or prior distributions of the effect size are assumed. Given the typically low power (few studies per research question), the test may be best applied across domains of many meta-analyses that share common characteristics (interventions, outcomes, study populations, research environment). Results We evaluated illustratively eight meta-analyses of clinical trials with >50 studies each and 10 meta-analyses of clinical efficacy for neuroleptic agents in schizophrenia; the 10 meta-analyses were also examined as a composite domain. Different results were obtained against commonly used tests of publication bias. We demonstrated a clear or possible excess of significant studies in 6 of 8 large meta-analyses and in the wide domain of neuroleptic treatments. Limitations The proposed test is exploratory, may depend on prior assumptions, and should be applied cautiously. Conclusions An excess of significant findings may be documented in some clinical research fields. Clinical Trials 2007; 4: 245—253; http://ctj.sagepub.com

Large-Scale Analysis of Association Between LRP5 and LRP6 Variants and Osteoporosis

CONTEXT Mutations in the low-density lipoprotein receptor-related protein 5 (LRP5) gene cause rare syndromes characterized by altered bone mineral density (BMD). More common LRP5 variants may affect osteoporosis risk in the general population. OBJECTIVE To generate large-scale evidence on whether 2 common variants of LRP5 (Val667Met, Ala1330Val) and 1 variant of LRP6 (Ile1062Val) are associated with BMD and fracture risk. DESIGN AND SETTING Prospective, multicenter, collaborative study of individual-level data on 37,534 individuals from 18 participating teams in Europe and North America. Data were collected between September 2004 and January 2007; analysis of the collected data was performed between February and May 2007. Bone mineral density was assessed by dual-energy x-ray absorptiometry. Fractures were identified via questionnaire, medical records, or radiographic documentation; incident fracture data were available for some cohorts, ascertained via routine surveillance methods, including radiographic examination for vertebral fractures. MAIN OUTCOME MEASURES Bone mineral density of the lumbar spine and femoral neck; prevalence of all fractures and vertebral fractures. RESULTS The Met667 allele of LRP5 was associated with reduced lumbar spine BMD (n = 25,052 [number of participants with available data]; 20-mg/cm2 lower BMD per Met667 allele copy; P = 3.3 x 10(-8)), as was the Val1330 allele (n = 24,812; 14-mg/cm2 lower BMD per Val1330 copy; P = 2.6 x 10(-9)). Similar effects were observed for femoral neck BMD, with a decrease of 11 mg/cm2 (P = 3.8 x 10(-5)) and 8 mg/cm2 (P = 5.0 x 10(-6)) for the Met667 and Val1330 alleles, respectively (n = 25 193). Findings were consistent across studies for both LRP5 alleles. Both alleles were associated with vertebral fractures (odds ratio [OR], 1.26; 95% confidence interval [CI], 1.08-1.47 for Met667 [2001 fractures among 20 488 individuals] and OR, 1.12; 95% CI, 1.01-1.24 for Val1330 [1988 fractures among 20,096 individuals]). Risk of all fractures was also increased with Met667 (OR, 1.14; 95% CI, 1.05-1.24 per allele [7876 fractures among 31,435 individuals)]) and Val1330 (OR, 1.06; 95% CI, 1.01-1.12 per allele [7802 fractures among 31 199 individuals]). Effects were similar when adjustments were made for age, weight, height, menopausal status, and use of hormone therapy. Fracture risks were partly attenuated by adjustment for BMD. Haplotype analysis indicated that Met667 and Val1330 variants both independently affected BMD. The LRP6 Ile1062Val polymorphism was not associated with any osteoporosis phenotype. All aforementioned associations except that between Val1330 and all fractures and vertebral fractures remained significant after multiple-comparison adjustments. CONCLUSIONS Common LRP5 variants are consistently associated with BMD and fracture risk across different white populations. The magnitude of the effect is modest. LRP5 may be the first gene to reach a genome-wide significance level (a conservative level of significance [herein, unadjusted P < 10(-7)] that accounts for the many possible comparisons in the human genome) for a phenotype related to osteoporosis.