Cheryl A. Roe

Statistical Comparison of Two ROC-curve Estimates Obtained from Partially-paired Datasets

The authors propose a new generalized method for ROC-curve fitting and statistical testing that allows researchers to utilize all of the data collected in an experimental comparison of two diagnostic modalities, even if some patients have not been studied with both modalities. Their new algorithm, ROCKIT, subsumes previous algorithms as special cases. It conducts all analyses available from previous ROC software and provides 95% confidence intervals for all estimates. ROCKIT was tested on more than half a million computer-simulated datasets of various sizes and configurations repre senting a range of population ROC curves. The algorithm successfully converged for more than 99.8% of all datasets studied. The type I error rates of the new algorithms statistical test for differences in Az estimates were excellent for datasets typically en countered in practice, but diverged from alpha for datasets arising from some extreme situations. Key words. receiver operating characteristic (ROC) analysis, maximum-like lihood estimation; partially-paired data; missing data. (Med Decis Making 1998;18: 110-121)

Genetic Architecture of Transcript-Level Variation in Humans

We report here the results of testing the pairwise association of 12,747 transcriptional gene-expression values with more than two million single-nucleotide polymorphisms (SNPs) in samples of European (CEPH from Utah; CEU) and African (Yoruba from Ibadan; YRI) ancestry. We found 4,677 and 5,125 significant associations between expression quantitative nucleotides (eQTNs) and transcript clusters in the CEU and the YRI samples, respectively. The physical distance between an eQTN and its associated transcript cluster was referred to as the intrapair distance. An association with 4 Mb or less intrapair distance was defined as local; otherwise, it was defined as distant. The enrichment analysis of functional categories shows that genes harboring the local eQTNs are enriched in the categories related to nucleosome and chromatin assembly; the genes harboring the distant eQTNs are enriched in the categories related to transmembrane signal transduction, suggesting that these biological pathways are likely to play a significant role in regulation of gene expression. We highlight in the EPHX1 gene a deleterious nonsynonymous SNP that is distantly associated with gene expression of ORMDL3, a susceptibility gene for asthma.

Identification of Type 2 Diabetes Genes in Mexican Americans Through Genome-wide Association Studies

OBJECTIVE—The objective of this study was to identify DNA polymorphisms associated with type 2 diabetes in a Mexican-American population. RESEARCH DESIGN AND METHODS—We genotyped 116,204 single nucleotide polymorphisms (SNPs) in 281 Mexican Americans with type 2 diabetes and 280 random Mexican Americans from Starr County, Texas, using the Affymetrix GeneChip Human Mapping 100K set. Allelic association exact tests were calculated. Our most significant SNPs were compared with results from other type 2 diabetes genome-wide association studies (GWASs). Proportions of African, European, and Asian ancestry were estimated from the HapMap samples using structure for each individual to rule out spurious association due to population substructure. RESULTS—We observed more significant allelic associations than expected genome wide, as empirically assessed by permutation (14 below a P of 1 × 10−4 [8.7 expected]). No significant differences were observed between the proportion of ancestry estimates in the case and random control sets, suggesting that the association results were not likely confounded by substructure. A query of our top ∼1% of SNPs (P < 0.01) revealed SNPs in or near four genes that showed evidence for association (P < 0.05) in multiple other GWAS interrogated: rs979752 and rs10500641 near UBQLNL and OR52H1 on chromosome 11, rs2773080 and rs3922812 in or near RALGPS2 on chromosome 1, and rs1509957 near EGR2 on chromosome 10. CONCLUSIONS—We identified several SNPs with suggestive evidence for replicated association with type 2 diabetes that merit further investigation.

Dorfman-Berbaum-Metz method for statistical analysis of multireader, multimodality receiver operating characteristic data: Validation with computer simulation

RATIONALE AND OBJECTIVES The authors examined the relationship between the critical P value (alpha) and the empirical type I error rate when using the Dorfman-Berbaum-Metz (DMB) method for analysis of variance in multireader, multimodality receiver operating characteristic (ROC) data. METHODS The authors developed a linear mixed-effect model to generate continuous, normally distributed random decision variables containing multiple sources (components) of variation. A range of magnitudes for these variance components was used to stimulate experiments in which multiple readers (three or five) read imaged obtained with two modalities from the same set of cases with no re-reading. Three binormal population ROC curves, with areas of 0.962, 0.855, and 0.702, were included. Case-sample sizes ranged from 50 to 400, and either 50% or 10% of cases were actually positive. For each experiment, 2,000 data sets were analyzed by the computer program, and the proportion of 2,000 modality differences that was found to be statistically significant at an alpha level of .05 was tubulated. RESULTS The test for modality difference performed well for the low and intermediate ROC curves, even with small case samples. For the high ROC curve, the small-sample results were conservative. No relationship between observed type I error rate and the magnitude of data correlation was evident. CONCLUSION For typical ROC curves, the DBM method is robust in testing for modality effects in the null case, given a sufficient sample size. Instructions for obtaining a free copy of the software are given.

Variance-component modeling in the analysis of receiver operating characteristic index estimates

RATIONALE AND OBJECTIVES The authors performed this study to clarify and systematize the large number of variances and correlations observable with variance-component models of receiver operating characteristic (ROC) index estimates. MATERIALS AND METHODS The authors present a variance-component model for ROC index estimates (and for differences between estimates) and show correspondences between the method of experimental replication and the random components in the model. The authors introduce a notation that identifies both the method of replication and, when examining estimate differences, the estimate pairing scheme. RESULTS For models with three factors (modality, reader, case sample), the authors delineated four methods of replication and eight pairing schemes for generating estimate differences. For each of the resulting 32 replication-pairing combinations, the authors gave expressions for the variance of the difference and for the correlation between the two ROC index estimates. CONCLUSION The variance-component approach is a useful statistical tool for modeling different sources of variation that contribute to the overall variance of ROC data and index estimates derived from those data.

Spoiling the Whole Bunch: Quality Control Aimed at Preserving the Integrity of High-Throughput Genotyping

False-positive or false-negative results attributable to undetected genotyping errors and confounding factors present a constant challenge for genome-wide association studies (GWAS) given the low signals associated with complex phenotypes and the noise associated with high-throughput genotyping. In the context of the genetics of kidneys in diabetes (GoKinD) study, we identify a source of error in genotype calling and demonstrate that a standard battery of quality-control (QC) measures is not sufficient to detect and/or correct it. We show that, if genotyping and calling are done by plate (batch), even a few DNA samples of marginally acceptable quality can profoundly alter the allele calls for other samples on the plate. In turn, this leads to significant differential bias in estimates of allele frequency between plates and, potentially, to false-positive associations, particularly when case and control samples are not sufficiently randomized to plates. This problem may become widespread as investigators tap into existing public databases for GWAS control samples. We describe how to detect and correct this bias by utilizing additional sources of information, including raw signal-intensity data.

Gender-specific differences in expression in human lymphoblastoid cell lines

Women and men have different risks for certain diseases and they often respond differently to treatment. These differences could be due to the sex-specific differences in the expression of genes related to primary disease susceptibility or pharmacodynamic targets. To evaluate the sex-specific pattern of gene expression, we compared gene expression levels using a publicly available microarray dataset of 233 (115 women and 118 men) lymphoblastoid cell lines. From the 4799 probes meeting a specified minimal level of expression, 10 genes (P<0.005, permutation adjusted false discovery rate less than 50%) located on autosomal chromosomes were identified using a permutation-based approach. These genes were found to be over-represented in certain gene ontology terms of biological process (cell adhesion, apoptosis, transcription and signal transduction), and molecular function (structural molecule activity, zinc ion binding, transcription factor activity and protein binding). A Kyoto Encyclopedia of Genes and Genomes pathway analysis indicated that two known pathways are over-represented: adherens junction and cytokine–cytokine receptor interaction.

Cleft Palate, Retrognathia and Congenital Heart Disease in Velo-Cardio-Facial Syndrome: A Phenotype Correlation Study

OBJECTIVE Velo-cardio-facial syndrome (VCFS) is caused by a microdeletion of approximately 40 genes from one copy of chromosome 22. Expression of the syndrome is a variable combination of over 190 phenotypic characteristics. As of yet, little is known about how these phenotypes correlate with one another or whether there are predictable patterns of expression. Two of the most common phenotypic categories, congenital heart disease and cleft palate, have been proposed to have a common genetic relationship to the deleted T-box 1 gene (TBX1). The purpose of this study is to determine if congenital heart disease and cleft palate are correlated in a large cohort of human subjects with VCFS. METHODS This study is a retrospective chart review including 316 Caucasian non-Hispanic subjects with FISH or CGH microarray confirmed chromosome 22q11.2 deletions. All subjects were evaluated by the interdisciplinary team at the Velo-Cardio-Facial Syndrome International Center at Upstate Medical University, Syracuse, NY. Each combination of congenital heart disease, cleft palates, and retrognathia was analyzed by Chi square or Fisher exact test. RESULTS For all categories of congenital heart disease and cleft palate or retrognathia no significant associations were found, with the exception of submucous cleft palate and retrognathia (nominal p=0.0325) and occult submucous cleft palate and retrognathia (nominal p=0.000013). CONCLUSIONS Congenital heart disease and cleft palate do not appear to be correlated in human subjects with VCFS despite earlier suggestions from animal models. Possible explanations include modification of the effect of TBX1 by genes outside of the 22q11.2 region that may further influence the formation of the palate or heart, or the presence of epigenetic factors that may effect genes within the deleted region, modifying genes elsewhere, or polymorphisms on the normal copy of chromosome 22. Lastly, it is possible that TBX1 plays a role in palate formation in some species, but not in humans. In VCFS, retrognathia is caused by an obtuse angulation of the skull base. It is unknown if the correlation between retrognathia and cleft palate in VCFS indicates a developmental sequence related to skull morphology, or direct gene effects of both anomalies. Much work remains to be done to fully understand the complex relationships between phenotypic characteristics in VCFS.

Technology insight: microarrays--research and clinical applications.

For microarrays, the transition from research to clinical and diagnostic applications is well underway. Microarrays use a range of specific probes that are immobilized in known locations on a support matrix; this technique can measure levels of specific DNA, RNA and proteins, as well as carbohydrates and lipids. It is anticipated that analysis of these levels will lead to identification of biomarkers for the diagnosis, treatment and prognosis of a wide range of diseases. So far, this type of analysis has been particularly useful in clinical oncology, but the technology is being actively and successfully explored for diseases such as diabetes, endocrine tumors and endocrine modulators of tumors. There are now many commercial sources of microarrays, which have robust quality-control procedures in place. Progress will be enhanced when biomarkers can be established, statistical approaches can be refined and when we better understand the interactions of genes and of particular gene loci in disease progression.