Alex Zwanenburg

Responsible Radiomics Research for Faster Clinical Translation

It is now recognized that intratumoral heterogeneity is associated with more aggressive tumor phenotypes leading to poor patient outcomes (1). Medical imaging plays a central role in related investigations, because radiologic images are routinely acquired during cancer management. Imaging modalities such as 18F-FDG PET, CT, and MRI are minimally invasive and would constitute an immense source of potential data for decoding tumor phenotypes (2). Computer-aided diagnosis methods and systems exploiting medical images have been developed for decades, but their wide clinical implementation has been hampered by false-positive rates (3). As a consequence, routine clinical exploitation of images still consists mostly of visual or manual assessments. Today, the development of machine-learning techniques and the rise of computational power allow for the exploitation of a large number of quantitative features (4). This ability has led to a new incarnation of computer-aided diagnosis, “radiomics,” which refers to the characterization of tumor phenotypes via the extraction of highdimensional mineable data—for example, morphologic, intensitybased, fractal-based, and textural features—from medical images and whose subsequent analysis aims at supporting clinical decision making. A first proof-of-concept study dedicated to the prediction of tumor outcomes using PET radiomics-based multivariable models built via machine learning was published in 2009 (5). The term radiomics was then first used in 2010 to describe how imaging features can reflect gene expression (6). Other early radiomics studies followed (7,8), including some highlighting early on that the reliability of existing features is affected by acquisition protocol, reconstruction, test–retest consistency, preprocessing, and segmentation (9–13). The overall framework of radiomics was then explicitly described in 2012 (14), and in the years that followed, this emerging field experienced exponential growth (15). In the context of precision oncology, the radiomics workflow for the construction of predictive or prognostic models consists of 3 major steps (Fig. 1A): medical image acquisition, computation of radiomics features, and statistical analysis and machine learning. To apply the models to new patients for treatment personalization, a prospective model evaluation (preferably in a multicenter setup) is necessary. Radiomics research has already shown great promise for supporting clinical decision making. However, the fact that radiomicsbased strategies have not yet been translated to routine practice can be partly attributed to the low reproducibility of most current studies. The workflow for computing features is complex and involves many steps (Fig. 1B), often leading to incomplete reporting of methodologic information (e.g., texture matrix design choices and gray-level discretization methods). As a consequence, few radiomics studies in the current literature can be reproduced from start to end. Other major issues include the limited number of patients available for radiomics research, the high false-positive rates (similar to those of analogous computer-aided diagnosis methods), and the reporting of overly optimistic results, all of which affect the generalizability of the conclusions reached in current studies. Medical imaging journals are currently overwhelmed by a large volume of radiomics-related articles of variable quality and associated clinical value. The aim of this editorial is to present guidelines that we think can improve the reporting quality and therefore the reproducibility of radiomics studies, as well as the statistical quality of radiomics analyses. These guidelines can serve not only the authors of such studies but also the reviewers who assess their appropriateness for publication.

A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling

Radiomics applies machine learning algorithms to quantitative imaging data to characterise the tumour phenotype and predict clinical outcome. For the development of radiomics risk models, a variety of different algorithms is available and it is not clear which one gives optimal results. Therefore, we assessed the performance of 11 machine learning algorithms combined with 12 feature selection methods by the concordance index (C-Index), to predict loco-regional tumour control (LRC) and overall survival for patients with head and neck squamous cell carcinoma. The considered algorithms are able to deal with continuous time-to-event survival data. Feature selection and model building were performed on a multicentre cohort (213 patients) and validated using an independent cohort (80 patients). We found several combinations of machine learning algorithms and feature selection methods which achieve similar results, e.g., MSR-RF: C-Index = 0.71 and BT-COX: C-Index = 0.70 in combination with Spearman feature selection. Using the best performing models, patients were stratified into groups of low and high risk of recurrence. Significant differences in LRC were obtained between both groups on the validation cohort. Based on the presented analysis, we identified a subset of algorithms which should be considered in future radiomics studies to develop stable and clinically relevant predictive models for time-to-event endpoints.

Development and Validation of a Gene Signature for Patients with Head and Neck Carcinomas Treated by Postoperative Radio(chemo)therapy

Purpose: The aim of this study was to identify and independently validate a novel gene signature predicting locoregional tumor control (LRC) for treatment individualization of patients with locally advanced HPV-negative head and neck squamous cell carcinomas (HNSCC) who are treated with postoperative radio(chemo)therapy (PORT-C). Experimental Design: Gene expression analyses were performed using NanoString technology on a multicenter training cohort of 130 patients and an independent validation cohort of 121 patients. The analyzed gene set was composed of genes with a previously reported association with radio(chemo)sensitivity or resistance to radio(chemo)therapy. Gene selection and model building were performed comparing several machine-learning algorithms. Results: We identified a 7-gene signature consisting of the three individual genes HILPDA, CD24, TCF3, and one metagene combining the highly correlated genes SERPINE1, INHBA, P4HA2, and ACTN1. The 7-gene signature was used, in combination with clinical parameters, to fit a multivariable Cox model to the training data (concordance index, ci = 0.82), which was successfully validated (ci = 0.71). The signature showed improved performance compared with clinical parameters alone (ci = 0.66) and with a previously published model including hypoxia-associated genes and cancer stem cell markers (ci = 0.65). It was used to stratify patients into groups with low and high risk of recurrence, leading to significant differences in LRC in training and validation (P < 0.001). Conclusions: We have identified and validated the first hypothesis-based gene signature for HPV-negative HNSCC treated by PORT-C including genes related to several radiobiological aspects. A prospective validation is planned in an ongoing prospective clinical trial before potential application in clinical trials for patient stratification. Clin Cancer Res; 24(6); 1364–74. ©2018 AACR.

Why validation of prognostic models matters

Prognostic models are powerful tools for treatment personalisation. However, not all proposed models work well when validated using new data, despite impressive results being reported initially. Here, we will use a hands-on approach to highlight important aspects of prognostic modelling, as well as to demonstrate methods to generate generalisable models.

CT imaging during treatment improves radiomic models for patients with locally advanced head and neck cancer

BACKGROUND AND PURPOSE The development of radiomic risk models to predict clinical outcome is usually based on pre-treatment imaging, such as computed tomography (CT) scans used for radiation treatment planning. Imaging data acquired during the course of treatment may improve their prognostic performance. We compared the performance of radiomic risk models based on the pre-treatment CT and CT scans acquired in the second week of therapy. MATERIAL AND METHODS Treatment planning and second week CT scans of 78 head and neck squamous cell carcinoma patients treated with primary radiochemotherapy were collected. 1538 image features were extracted from each image. Prognostic models for loco-regional tumour control (LRC) and overall survival (OS) were built using 6 feature selection methods and 6 machine learning algorithms. Prognostic performance was assessed using the concordance index (C-Index). Furthermore, patients were stratified into risk groups and differences in LRC and OS were evaluated by log-rank tests. RESULTS The performance of radiomic risk model in predicting LRC was improved using the second week CT scans (C-Index: 0.79), in comparison to the pre-treatment CT scans (C-Index: 0.65). This was confirmed by Kaplan-Meier analyses, in which risk stratification based on the second week CT could be improved for LRC (p = 0.002) compared to pre-treatment CT (p = 0.063). CONCLUSION Incorporation of imaging during treatment may be a promising way to improve radiomic risk models for clinical treatment adaption, i.e., to select patients that may benefit from dose modification.