Fatemeh Seyednasrollah

Comparison of software packages for detecting differential expression in RNA-seq studies

RNA-sequencing (RNA-seq) has rapidly become a popular tool to characterize transcriptomes. A fundamental research problem in many RNA-seq studies is the identification of reliable molecular markers that show differential expression between distinct sample groups. Together with the growing popularity of RNA-seq, a number of data analysis methods and pipelines have already been developed for this task. Currently, however, there is no clear consensus about the best practices yet, which makes the choice of an appropriate method a daunting task especially for a basic user without a strong statistical or computational background. To assist the choice, we perform here a systematic comparison of eight widely used software packages and pipelines for detecting differential expression between sample groups in a practical research setting and provide general guidelines for choosing a robust pipeline. In general, our results demonstrate how the data analysis tool utilized can markedly affect the outcome of the data analysis, highlighting the importance of this choice.

ROTS: reproducible RNA-seq biomarker detector—prognostic markers for clear cell renal cell cancer

Recent comprehensive assessments of RNA-seq technology support its utility in quantifying gene expression in various samples. The next step of rigorously quantifying differences between sample groups, however, still lacks well-defined best practices. Although a number of advanced statistical methods have been developed, several studies demonstrate that their performance depends strongly on the data under analysis, which compromises practical utility in real biomedical studies. As a solution, we propose to use a data-adaptive procedure that selects an optimal statistic capable of maximizing reproducibility of detections. After demonstrating its improved sensitivity and specificity in a controlled spike-in study, the utility of the procedure is confirmed in a real biomedical study by identifying prognostic markers for clear cell renal cell carcinoma (ccRCC). In addition to identifying several genes previously associated with ccRCC prognosis, several potential new biomarkers among genes regulating cell growth, metabolism and solute transport were detected.

Comparison of methods to detect differentially expressed genes between single-cell populations.

Abstract We compared five statistical methods to detect differentially expressed genes between two distinct single-cell populations. Currently, it remains unclear whether differential expression methods developed originally for conventional bulk RNA-seq data can also be applied to single-cell RNA-seq data analysis. Our results in three diverse comparison settings showed marked differences between the different methods in terms of the number of detections as well as their sensitivity and specificity. They, however, did not reveal systematic benefits of the currently available single-cell-specific methods. Instead, our previously introduced reproducibility-optimization method showed good performance in all comparison settings without any single-cell-specific modifications.

ROTS: An R package for reproducibility-optimized statistical testing

Differential expression analysis is one of the most common types of analyses performed on various biological data (e.g. RNA-seq or mass spectrometry proteomics). It is the process that detects features, such as genes or proteins, showing statistically significant differences between the sample groups under comparison. A major challenge in the analysis is the choice of an appropriate test statistic, as different statistics have been shown to perform well in different datasets. To this end, the reproducibility-optimized test statistic (ROTS) adjusts a modified t-statistic according to the inherent properties of the data and provides a ranking of the features based on their statistical evidence for differential expression between two groups. ROTS has already been successfully applied in a range of different studies from transcriptomics to proteomics, showing competitive performance against other state-of-the-art methods. To promote its widespread use, we introduce here a Bioconductor R package for performing ROTS analysis conveniently on different types of omics data. To illustrate the benefits of ROTS in various applications, we present three case studies, involving proteomics and RNA-seq data from public repositories, including both bulk and single cell data. The package is freely available from Bioconductor (https://www.bioconductor.org/packages/ROTS).

Prediction of Adulthood Obesity Using Genetic and Childhood Clinical Risk Factors in the Cardiovascular Risk in Young Finns StudyCLINICAL PERSPECTIVE

Background— Obesity is a known risk factor for cardiovascular disease. Early prediction of obesity is essential for prevention. The aim of this study is to assess the use of childhood clinical factors and the genetic risk factors in predicting adulthood obesity using machine learning methods. Methods and Results— A total of 2262 participants from the Cardiovascular Risk in YFS (Young Finns Study) were followed up from childhood (age 3–18 years) to adulthood for 31 years. The data were divided into training (n=1625) and validation (n=637) set. The effect of known genetic risk factors (97 single-nucleotide polymorphisms) was investigated as a weighted genetic risk score of all 97 single-nucleotide polymorphisms (WGRS97) or a subset of 19 most significant single-nucleotide polymorphisms (WGRS19) using boosting machine learning technique. WGRS97 and WGRS19 were validated using external data (n=369) from BHS (Bogalusa Heart Study). WGRS19 improved the accuracy of predicting adulthood obesity in training (area under the curve [AUC=0.787 versus AUC=0.744, P<0.0001) and validation data (AUC=0.769 versus AUC=0.747, P=0.026). WGRS97 improved the accuracy in training (AUC=0.782 versus AUC=0.744, P<0.0001) but not in validation data (AUC=0.749 versus AUC=0.747, P=0.785). Higher WGRS19 associated with higher body mass index at 9 years and WGRS97 at 6 years. Replication in BHS confirmed our findings that WGRS19 and WGRS97 are associated with body mass index. Conclusions— WGRS19 improves prediction of adulthood obesity. Predictive accuracy is highest among young children (3–6 years), whereas among older children (9–18 years) the risk can be identified using childhood clinical factors. The model is helpful in screening children with high risk of developing obesity.

How Reliable are Trial-based Prognostic Models in Real-world Patients with Metastatic Castration-resistant Prostate Cancer?

Robust prognostic factors are crucial for improving clinical trial design and later assisting treatment decision-making. The Dialogue for Reverse Engineering Assessments and Methods committee recently organized a crowdsourced, international competition to develop a new prognostic benchmark for predicting overall survival (OS) of metastatic castration-resistant prostate cancer (mCRPC) patients in docetaxel arms of randomized controlled trials (RCTs) [1]. However, utility of these trial-tailored prognostic models lacks confirmation in everyday practice. RCTs are gold standard for efficacy assessment of cancer therapies [2], but agreement of results between RCTs and real-world (RW) patients remains controversial. RCTs have high internal, but limited external, validity as RCT participants may poorly represent the RW population [3]. Inspired by promising results from the Dialogue for Reverse Engineering Assessments and Methods Challenge, we investigated both consistency between RCT and RW patients, and the applicability of RCT-based models to RW patients. The RCT data included four independent phase 3 clinical trials from the Challenge (n = 2070). The RW data included all mCRPC patients (n = 289) treated with first-line docetaxel at Turku University Hospital, Finland, in 2004– 2015 (Supplementary data). Over 150 clinical variables were available (Supplementary Table 1). As previously reported [3,4], RW patients tended to be older and had worse Eastern Cooperative Oncology Group status than RCT patients (p < 0.001; Supplementary Table 2). However, principal component analysis suggested high similarity between the cohorts in terms of variables of the Challenge reference model, Halabi et al. [5] (Fig. 1A), supported by consistent hazard ratios for OS across cohorts (Supplementary Table 3). Contrary to previous studies, OS was not significantly different between the cohorts (p = 0.11; Supplementary Fig. 1).

A DREAM Challenge to Build Prediction Models for Short-Term Discontinuation of Docetaxel in Metastatic Castration-Resistant Prostate Cancer

Purpose Docetaxel has a demonstrated survival benefit for patients with metastatic castration-resistant prostate cancer (mCRPC); however, 10% to 20% of patients discontinue docetaxel prematurely because of toxicity-induced adverse events, and the management of risk factors for toxicity remains a challenge. Patients and Methods The comparator arms of four phase III clinical trials in first-line mCRPC were collected, annotated, and compiled, with a total of 2,070 patients. Early discontinuation was defined as treatment stoppage within 3 months as a result of adverse treatment effects; 10% of patients discontinued treatment. We designed an open-data, crowd-sourced DREAM Challenge for developing models with which to predict early discontinuation of docetaxel treatment. Clinical features for all four trials and outcomes for three of the four trials were made publicly available, with the outcomes of the fourth trial held back for unbiased model evaluation. Challenge participants from around the world trained models and submitted their predictions. Area under the precision-recall curve was the primary metric used for performance assessment. Results In total, 34 separate teams submitted predictions. Seven models with statistically similar area under precision-recall curves (Bayes factor ≤ 3) outperformed all other models. A postchallenge analysis of risk prediction using these seven models revealed three patient subgroups: high risk, low risk, or discordant risk. Early discontinuation events were two times higher in the high-risk subgroup compared with the low-risk subgroup. Simulation studies demonstrated that use of patient discontinuation prediction models could reduce patient enrollment in clinical trials without the loss of statistical power. Conclusion This work represents a successful collaboration between 34 international teams that leveraged open clinical trial data. Our results demonstrate that routinely collected clinical features can be used to identify patients with mCRPC who are likely to discontinue treatment because of adverse events and establishes a robust benchmark with implications for clinical trial design.

Reply to Tuomas Mirtti and Tero Aittokallio's Letter to the Editor re: Fatemeh Seyednasrollah, Mehrad Mahmoudian, Liisa Rautakorpi, et al. How Reliable are Trial-based Prognostic Models in Real-world Patients with Metastatic Castration-resistant Prostate Cancer? Eur Urol. 2017;71:838–40. Clinical Utility of Trial-estimated Prognostic Models

The letter from Mirtti and Aittokallio raises important issues regarding reproducible research and the practical utility of models in clinical decision-making. However, their request to consult model developers for model application seems unwarranted. Benchmark models introduced in high-impact journals, such as the one by Aittokallio et al [1], should be usable by others. The DREAM challenge organizers made invaluable efforts to guarantee reproducibility, which was also a main rule for challenge participation. Duty falls on the developers themselves to ensure models can be easily and correctly applied by others. Missing values are intrinsic to clinical data, and clinically relevant models should be able to robustly deal with any missing values. Conventions in the real world (RW) and randomized clinical trials (RCTs) are not consistent and there are no strict universal rules. For instance, in the DREAM challenge, two important predictors in the Halabi reference model [2], lactate dehydrogenase and albumin, were entirely missing in the VENICE and ASCENT2 trials, respectively. Regarding imputation, we indeed first carried out imputation using each team’s own approach (teams 1 and 2, model-based imputation; team 3, median imputation); only persisting missing values were medianimputed. Of note, Aittokallio and co-authors used median imputation for the Halabi model in [1], although the original study used improved model-based imputation [2]. The assumption that all patients had bone metastases was based on their high prevalence in metastatic castrationresistant prostate cancer (mCRPC), supported both by literature and DREAM challenge data; 89% of patients had bone metastases (ENTHUSE 33 and ENTHUSE M1 100%). This high prevalence also drove the decision by Aittokallio’s team 1 to remove bone lesions from their model. Finally, Mirtti and Aittokallio describe our RW data as ‘‘limited’’ data for a heterogeneous patient subpopulation not attractive for application of mCRPC prognostic models.

End-of-life chemotherapy use at a Finnish university hospital: a retrospective cohort study

Abstract Background: Recent trends in the end-of-life (EOL) cancer care have suggested that the levels of treatment are becoming more aggressive. The aim of this single-center study was to evaluate the time from the last intravenous (IV) chemotherapy treatment to death and identify factors correlating with treatment closer to death. Material and methods: The study included all patients diagnosed with cancer at Turku University Central Hospital between the years 2005 and 2013 (N = 38,982) who received IV chemotherapy during the last year of life (N = 3285). The cohort of patients and their respective clinical information were identified from electronic medical records. Statistical analysis was performed to assess and compare the treatment strategies, taking into account the patient’s age, the year they were treated, and the type of cancer they were diagnosed with. Results: A total of 11,250 cancer patients died during the observation time and one-third (N = 3285, 29.2%) of them had received IV chemotherapy during the last year of life. The time from the last IV chemotherapy regimen to death remained consistent across the follow-up time. During the last month of life, every third patient under the age of 50 years and only one-tenth of patients over the age of 80 years received IV chemotherapy. Hematological malignancies and lymphomas were treated closer to death when compared to other diagnostic groups. Conclusions: During the period of 9 years, the pattern of EOL IV chemotherapy treatment remained stable. Every third patient died at tertiary care. Only 7.2% of patients who received IV chemotherapy during the last year of life were treated 14 days before death, which is in line with international recommendations. However, significant variation in EOL treatment strategies between different diagnosis and age groups were identified.

Assessing the utilization of radiotherapy near end of life at a Finnish University Hospital: a retrospective cohort study

Abstract Background: Palliative radiotherapy can improve quality of life for cancer patients during the last months of life. However, very short life expectancy may devastate the benefit of the treatment. This single center study assesses the utilization of radiotherapy during the last weeks of life. Material and methods: All cancer patients (N = 38,982) treated with radiotherapy (N = 11,395) in Turku University Central Hospital during 2005–2013 were identified in the database consisting of electronic patient records. One fourth (N = 2904, 25.5%) of the radiotherapy treatments were given during the last year of life. The last radiotherapy treatments and the time from the last radiotherapy treatment to death were assessed in regards to patients’ age, cancer diagnosis, domicile, place of death and the treatment year. Treatments given during the last two weeks of life were also assessed regarding the goal of treatment and the reason for possible discontinuation. Results: The median time from the last fraction of radiotherapy to death was 84 d. During the last two weeks before death (N = 340), pain (29.4%) was the most common indication for radiotherapy. Treatment was discontinued in 40.6% of the patients during the last two weeks of life, and worsening of general condition was the most common reason for discontinuity (70.3%). The patients receiving radiotherapy during the last weeks of life were more likely to die in tertiary care unit. During the last year of life single-fraction treatment was used only in 7% of all therapy courses. There was a statistically significant (p < .05) decrease in the median number of fractions in the last radiotherapy treatment between 2005–2007 (8 fractions) and 2011–2013 (6 fractions). Conclusions: Up to 70% of the treatments during the last two weeks of life were not delivered to alleviate pain and utilization of single fraction radiotherapy during the last year of life was infrequent. These observations suggest that practice of radiotherapy during the last weeks of life should be revisited.