Cesare Furlanello

Repeatability of published microarray gene expression analyses

Given the complexity of microarray-based gene expression studies, guidelines encourage transparent design and public data availability. Several journals require public data deposition and several public databases exist. However, not all data are publicly available, and even when available, it is unknown whether the published results are reproducible by independent scientists. Here we evaluated the replication of data analyses in 18 articles on microarray-based gene expression profiling published in Nature Genetics in 2005–2006. One table or figure from each article was independently evaluated by two teams of analysts. We reproduced two analyses in principle and six partially or with some discrepancies; ten could not be reproduced. The main reason for failure to reproduce was data unavailability, and discrepancies were mostly due to incomplete data annotation or specification of data processing and analysis. Repeatability of published microarray studies is apparently limited. More strict publication rules enforcing public data availability and explicit description of data processing and analysis should be considered.

The concordance between RNA-seq and microarray data depends on chemical treatment and transcript abundance

The concordance of RNA-sequencing (RNA-seq) with microarrays for genome-wide analysis of differential gene expression has not been rigorously assessed using a range of chemical treatment conditions. Here we use a comprehensive study design to generate Illumina RNA-seq and Affymetrix microarray data from the same liver samples of rats exposed in triplicate to varying degrees of perturbation by 27 chemicals representing multiple modes of action (MOAs). The cross-platform concordance in terms of differentially expressed genes (DEGs) or enriched pathways is linearly correlated with treatment effect size (R20.8). Furthermore, the concordance is also affected by transcript abundance and biological complexity of the MOA. RNA-seq outperforms microarray (93% versus 75%) in DEG verification as assessed by quantitative PCR, with the gain mainly due to its improved accuracy for low-abundance transcripts. Nonetheless, classifiers to predict MOAs perform similarly when developed using data from either platform. Therefore, the endpoint studied and its biological complexity, transcript abundance and the genomic application are important factors in transcriptomic research and for clinical and regulatory decision making.

Entropy-based gene ranking without selection bias for the predictive classification of microarray data.

BackgroundWe describe the E-RFE method for gene ranking, which is useful for the identification of markers in the predictive classification of array data. The method supports a practical modeling scheme designed to avoid the construction of classification rules based on the selection of too small gene subsets (an effect known as the selection bias, in which the estimated predictive errors are too optimistic due to testing on samples already considered in the feature selection process).ResultsWith E-RFE, we speed up the recursive feature elimination (RFE) with SVM classifiers by eliminating chunks of uninteresting genes using an entropy measure of the SVM weights distribution. An optimal subset of genes is selected according to a two-strata model evaluation procedure: modeling is replicated by an external stratified-partition resampling scheme, and, within each run, an internal K-fold cross-validation is used for E-RFE ranking. Also, the optimal number of genes can be estimated according to the saturation of Zipfs law profiles.ConclusionsWithout a decrease of classification accuracy, E-RFE allows a speed-up factor of 100 with respect to standard RFE, while improving on alternative parametric RFE reduction strategies. Thus, a process for gene selection and error estimation is made practical, ensuring control of the selection bias, and providing additional diagnostic indicators of gene importance.

Mitigation Measures for Pandemic Influenza in Italy: An Individual Based Model Considering Different Scenarios

Background Individual-based models can provide the most reliable estimates of the spread of infectious diseases. In the present study, we evaluated the diffusion of pandemic influenza in Italy and the impact of various control measures, coupling a global SEIR model for importation of cases with an individual based model (IBM) describing the Italian epidemic. Methodology/Principal Findings We co-located the Italian population (57 million inhabitants) to households, schools and workplaces and we assigned travel destinations to match the 2001 census data. We considered different R0 values (1.4; 1.7; 2), evaluating the impact of control measures (vaccination, antiviral prophylaxis -AVP-, international air travel restrictions and increased social distancing). The administration of two vaccine doses was considered, assuming that first dose would be administered 1-6 months after the first world case, and different values for vaccine effectiveness (VE). With no interventions, importation would occur 37–77 days after the first world case. Air travel restrictions would delay the importation of the pandemic by 7–37 days. With an R0 of 1.4 or 1.7, the use of combined measures would reduce clinical attack rates (AR) from 21–31% to 0.3–4%. Assuming an R0 of 2, the AR would decrease from 38% to 8%, yet only if vaccination were started within 2 months of the first world case, in combination with a 90% reduction in international air traffic, closure of schools/workplaces for 4 weeks and AVP of household and school/work close contacts of clinical cases. Varying VE would not substantially affect the results. Conclusions This IBM, which is based on country-specific demographic data, could be suitable for the real-time evaluation of measures to be undertaken in the event of the emergence of a new pandemic influenza virus. All preventive measures considered should be implemented to mitigate the pandemic.

A Comparison of MCC and CEN Error Measures in Multi-Class Prediction

We show that the Confusion Entropy, a measure of performance in multiclass problems has a strong (monotone) relation with the multiclass generalization of a classical metric, the Matthews Correlation Coefficient. Analytical results are provided for the limit cases of general no-information (n-face dice rolling) of the binary classification. Computational evidence supports the claim in the general case.

Gut microbiota profiling of pediatric nonalcoholic fatty liver disease and obese patients unveiled by an integrated meta-omics-based approach

There is evidence that nonalcoholic fatty liver disease (NAFLD) is affected by gut microbiota. Therefore, we investigated its modifications in pediatric NAFLD patients using targeted metagenomics and metabolomics. Stools were collected from 61 consecutive patients diagnosed with nonalcoholic fatty liver (NAFL), nonalcoholic steatohepatitis (NASH), or obesity and 54 healthy controls (CTRLs), matched in a case‐control fashion. Operational taxonomic units were pyrosequenced targeting 16S ribosomal RNA and volatile organic compounds determined by solid‐phase microextraction gas chromatography‐mass spectrometry. The α‐diversity was highest in CTRLs, followed by obese, NASH, and NAFL patients; and β‐diversity distinguished between patients and CTRLs but not NAFL and NASH. Compared to CTRLs, in NAFLD patients Actinobacteria were significantly increased and Bacteroidetes reduced. There were no significant differences among the NAFL, NASH, and obese groups. Overall NAFLD patients had increased levels of Bradyrhizobium, Anaerococcus, Peptoniphilus, Propionibacterium acnes, Dorea, and Ruminococcus and reduced proportions of Oscillospira and Rikenellaceae compared to CTRLs. After reducing metagenomics and metabolomics data dimensionality, multivariate analyses indicated a decrease of Oscillospira in NAFL and NASH groups and increases of Ruminococcus, Blautia, and Dorea in NASH patients compared to CTRLs. Of the 292 volatile organic compounds, 26 were up‐regulated and 2 down‐regulated in NAFLD patients. Multivariate analyses found that combination of Oscillospira, Rickenellaceae, Parabacteroides, Bacteroides fragilis, Sutterella, Lachnospiraceae, 4‐methyl‐2‐pentanone, 1‐butanol, and 2‐butanone could discriminate NAFLD patients from CTRLs. Univariate analyses found significantly lower levels of Oscillospira and higher levels of 1‐pentanol and 2‐butanone in NAFL patients compared to CTRLs. In NASH, lower levels of Oscillospira were associated with higher abundance of Dorea and Ruminococcus and higher levels of 2‐butanone and 4‐methyl‐2‐pentanone compared to CTRLs. Conclusion: An Oscillospira decrease coupled to a 2‐butanone up‐regulation and increases in Ruminococcus and Dorea were identified as gut microbiota signatures of NAFL onset and NAFL‐NASH progression, respectively. (Hepatology 2017;65:451‐464)

Comparison of RNA-seq and microarray-based models for clinical endpoint prediction

BackgroundGene expression profiling is being widely applied in cancer research to identify biomarkers for clinical endpoint prediction. Since RNA-seq provides a powerful tool for transcriptome-based applications beyond the limitations of microarrays, we sought to systematically evaluate the performance of RNA-seq-based and microarray-based classifiers in this MAQC-III/SEQC study for clinical endpoint prediction using neuroblastoma as a model.ResultsWe generate gene expression profiles from 498 primary neuroblastomas using both RNA-seq and 44 k microarrays. Characterization of the neuroblastoma transcriptome by RNA-seq reveals that more than 48,000 genes and 200,000 transcripts are being expressed in this malignancy. We also find that RNA-seq provides much more detailed information on specific transcript expression patterns in clinico-genetic neuroblastoma subgroups than microarrays. To systematically compare the power of RNA-seq and microarray-based models in predicting clinical endpoints, we divide the cohort randomly into training and validation sets and develop 360 predictive models on six clinical endpoints of varying predictability. Evaluation of factors potentially affecting model performances reveals that prediction accuracies are most strongly influenced by the nature of the clinical endpoint, whereas technological platforms (RNA-seq vs. microarrays), RNA-seq data analysis pipelines, and feature levels (gene vs. transcript vs. exon-junction level) do not significantly affect performances of the models.ConclusionsWe demonstrate that RNA-seq outperforms microarrays in determining the transcriptomic characteristics of cancer, while RNA-seq and microarray-based models perform similarly in clinical endpoint prediction. Our findings may be valuable to guide future studies on the development of gene expression-based predictive models and their implementation in clinical practice.

Clinical value of prognosis gene expression signatures in colorectal cancer: a systematic review.

Introduction The traditional staging system is inadequate to identify those patients with stage II colorectal cancer (CRC) at high risk of recurrence or with stage III CRC at low risk. A number of gene expression signatures to predict CRC prognosis have been proposed, but none is routinely used in the clinic. The aim of this work was to assess the prediction ability and potential clinical usefulness of these signatures in a series of independent datasets. Methods A literature review identified 31 gene expression signatures that used gene expression data to predict prognosis in CRC tissue. The search was based on the PubMed database and was restricted to papers published from January 2004 to December 2011. Eleven CRC gene expression datasets with outcome information were identified and downloaded from public repositories. Random Forest classifier was used to build predictors from the gene lists. Matthews correlation coefficient was chosen as a measure of classification accuracy and its associated p-value was used to assess association with prognosis. For clinical usefulness evaluation, positive and negative post-tests probabilities were computed in stage II and III samples. Results Five gene signatures showed significant association with prognosis and provided reasonable prediction accuracy in their own training datasets. Nevertheless, all signatures showed low reproducibility in independent data. Stratified analyses by stage or microsatellite instability status showed significant association but limited discrimination ability, especially in stage II tumors. From a clinical perspective, the most predictive signatures showed a minor but significant improvement over the classical staging system. Conclusions The published signatures show low prediction accuracy but moderate clinical usefulness. Although gene expression data may inform prognosis, better strategies for signature validation are needed to encourage their widespread use in the clinic.

Algebraic stability indicators for ranked lists in molecular profiling

MOTIVATION We propose a method for studying the stability of biomarker lists obtained from functional genomics studies. It is common to adopt resampling methods to tune and evaluate marker-based diagnostic and prognostic systems in order to prevent selection bias. Such caution promotes honest estimation of class prediction, but leads to alternative sets of solutions. In microarray studies, the difference in lists may be bewildering, also due to the presence of modules of functionally related genes. Methods for assessing stability understand the dependency of the markers on the data or on the predictors type and help selecting solutions. RESULTS A computational framework for comparing sets of ranked biomarker lists is presented. Notions and algorithms are based on concepts from permutation group theory. We introduce several algebraic indicators and metric methods for symmetric groups, including the Canberra distance, a weighted version of Spearmans footrule. We also consider distances between partial lists and an aggregation of sets of lists into an optimal list based on voting theory (Borda count). The stability indicators are applied in practical situations to several synthetic, cancer microarray and proteomics datasets. The addressed issues are predictive classification, presence of modules, comparison of alternative biomarker lists, outlier removal, control of selection bias by randomization techniques and enrichment analysis. AVAILABILITY Supplementary Material and software are available at the address http://biodcv.fbk.eu/listspy.html

An accelerated procedure for recursive feature ranking on microarray data

We describe a new wrapper algorithm for fast feature ranking in classification problems. The Entropy-based Recursive Feature Elimination (E-RFE) method eliminates chunks of uninteresting features according to the entropy of the weights distribution of a SVM classifier. With specific regard to DNA microarray datasets, the method is designed to support computationally intensive model selection in classification problems in which the number of features is much larger than the number of samples. We test E-RFE on synthetic and real data sets, comparing it with other SVM-based methods. The speed-up obtained with E-RFE supports predictive modeling on high dimensional microarray data.