DARWIN: A Highly Flexible Platform for Imaging Research in Radiology
Lufan Chang, Wenjing Zhuang, Richeng Wu, Sai Feng, Hao Liu, Jing Yu, Jia Ding, Ziteng Wang, Jiaqi Zhang
11 DARWIN: A Highly Flexible Platform for ImagingResearch in Radiology
Lufan Chang, Wenjing Zhuang, Richeng Wu, Sai Feng, Hao Liu, Jing Yu, Jia Ding, Ziteng Wang, Jiaqi Zhang
Abstract —To conduct a radiomics or deep learning researchexperiment, the radiologists or physicians need to grasp theneeded programming skills, which, however, could be frustratingand costly when they have limited coding experience. In thispaper, we present DARWIN, a flexible research platform witha graphical user interface for medical imaging research. Ourplatform is consists of a radiomics module and a deep learningmodule. The radiomics module can extract more than 1000 di-mension features(first-, second-, and higher-order) and providedmany draggable supervised and unsupervised machine learningmodels. Our deep learning module integrates state of the artarchitectures of classification, detection, and segmentation tasks.It allows users to manually select hyperparameters, or choose analgorithm to automatically search for the best ones. DARWINalso offers the possibility for users to define a custom pipeline fortheir experiment. These flexibilities enable radiologists to carryout various experiments easily.
Index Terms —GUI platform, radiomics, deep learning, hyper-parameters tuning,
I. I
NTRODUCTION M ACHINE learning becomes ubiquitous in recent med-ical imaging analysis research. The radiomics anddeep learning techniques have facilitated numerous diseasediagnosis with high precision [1]–[3]. There has been aseries of applications of machine learning to medical imaging.Physicians can use machine learning techniques to explorethe relationships between images processed using radiomics,clinical outcomes and radiation dose data to help improvecancer treatment with radiotherapy [2]. Bayesian Network(BN), decision tree (DT), and Support Vector Machine (SVM)have been employed in various studies to predict 2-yearsurvival [4]–[6]. Numerous machine learning models and fea-ture selection techniques for Non-small-cell Lung Carcinoma(NSCLC) cancer patients has been investigated in [7]. Machinelearning models also facilitate many tasks such as lung nodulesclassification [8], [9], lesions detection [10], segmentation[11], [12], and image registration [13].Many machine learning platforms for general tasks havebeen developed by some high technology companies suchas Google, Amazon, and Microsoft. These platforms greatlyreduce duplication of efforts for developers. However, codingexperience is needed for conducting experiments on theseplatforms, which makes it difficult for physicians with limitedprogramming skills. Furthermore, these platforms do not focus
Authors are with the Yizhun Medical AI Co.Ltd, Beijing, ChinaEmails: { lufan.chang, wenjing.zhuang, richeng.wu, sai.feng, hao.liu,jing.yu, jia.ding, ziteng.wang, jiaqi.zhang } @yizhun-ai.comManuscript received April 19, 2005; revised August 26, 2015. on medical images, some requirements from physicians orradiologists may not be covered.A few platforms targeting medical images have come outto facilitate radiologists to carry out their research [14]–[17].Imaging Biomarker Explorer (IBEX) [15] is a graphical-user-interface (GUI) based platform for medical image analysis.It integrates ROI labeler, radiomic feature extractor, andfeature/model viewer. IBEX offers the flexibility for usersto choose features, however, it only works on the Windowssystem, and the dependency files make the installation ratherdifficult. RayPlus [17] has been proposed to solve this prob-lem. RayPlus is a web-based platform across all systems. Thefeatures provided include five image filters with histogram,volumetric, morphologic, and texture features. These featurescould be downloaded and further analyzed in R or SPSS.These GUI based platforms enable users to do imageanalysis with minimal coding experience, however, still havenotable weaknesses: 1) Only certain types of images suchas CT, PET are supported, which cannot satisfy the needsof physicians to study various types of images. 2) Limitednumbers of features are included, making it hard to generalizewell on different tasks. 3) Only radiomic features provided,and users still need to write programs on machine learningmodels in a way they can make clinical predictions using thesefeatures. 4) State of the art deep learning methods are notcovered in these platforms.Therefore we developed DARWIN: an artificial intelligenceresearch platform for medical imaging. The platform integratesnumerous data preprocessing methods, machine learning mod-els, and data visualization methods with a web-based GUI. Theplatform does not have steep learning curves for radiologiststo master it. The main contributions of the platform mainlylie on the following four aspects: 1) incorporated PyRadiomicslibrary [18] to extract abundant radiomics features. 2)providedseveral deep learning models with hyperparameter tuningalgorithms 3) provided customized pipelines for differentapplications. 4) provided a user-friendly interaction mode.II. O VERALL F EATURES OF THE P LATFORM
DARWIN research platform is a software mainly designedfor radiologists to conduct machine learning research. It isconvenient for physicians or radiologists to build an ex-periment pipeline to validate their hypotheses immediately.Fig 1 presents the architecture of DARWIN. It is based onBrowser/Server (B/S) structure. Users can access the platformwith a web browser on any operating system (Mac OSX,Windows, and Linux). It is implemented using Javascript,Python, and C++. a r X i v : . [ ee ss . I V ] S e p Fig. 1. Computational Architecture of the DARWIN Platform. Users can use the web browser to upload the experiment images, annotate region of interests(ROIs), construct a computational graph, download results, and retrieve history records. The four databases at the server-side store the images, masks, radiomicfeatures ,and models respectively.
The main functionalities of DARWIN include the following: • Image Labeler: A GUI for DICOM image loading anddisplay. Users can also draw a region of interest (ROI)and annotate it with a self-defined label (E.g. malignantor benign). • Computational Graph: A GUI to build a computationalpipeline, which includes image loader, preprocessing, fea-ture extraction, visualization, etc (Fig 2). Deep learningand radiomic experiments have separate computationalgraphs. Each computational graph is a tree structure withcomputational nodes.
Fig. 3. Semi-automatic segmentation tools. The first image is a coarse ROIdrawn by the user on T1WI. The second image shows the ROI generatedby the ’Fit Boundary’ tool. The third one is the same ROI automaticallytransferred to T2WI via the ’Copy/Paste’ tool.
III. I
MAGE L ABELER AND R ADIOMIC F EATURE E XTRACTOR
The image labeler can load multimodal DICOM imagesincluding CT, MRI, CR/DXR, PET, and so on. Like PictureArchiving and Communication System (PACS), we can notonly view pictures on it, but also do window level/widthadjusting, image scaling, and 3D-reconstruction. ROI is alwaysnecessary for analyzing, and we offer tools for ROI loading,drawing, and modifying. For breast cancer and pulmonarynodules, we have built-in convolutional neural networks thatcan automatically detect and segment lesions. For other typesof images and ROIs, the regions can also be drawn manuallywith the 2D and 3D drawing tool. To help physicians draw theROIs effortlessly, we provided semi-automatic segmentationtools (Fig. 3). The ’Fit Boundary’ tool implement using theseeded region growing algorithm [19] helps to segment outthe bright/dark ROI within a manually drawn curve, andwill spread out to 3D neighboring slices if allowed. The’Copy/Paste’ tool allows users to copy an ROI to differentseries in the same study with automatically registered.A 2D ROI in our system is a polygon, thus we saveit using the position of all its vertexes. Accordingly, it isstraightforward for users to modify any vertex. Furthermore,for each ROI, we can annotate labels on it, such as malignantand benign or BI-RADS (0,1,2,3,4,5). An important thing for
Fig. 2. The computational graph of a radiomic experiment. The left sidebar menu presents entries to Laboratory, Image Gallery, and other Projects. The leftpart of the page is the node-selecting area. Users can drag the nodes to the center and build up a computational graph. All the parameters could be changedon the right side. our labeler is that we can manually or automatically establishsome mappings between ROIs. For example, in some cases,we could have Cranial-Caudal (CC) and mediolateral-oblique(MLO) views for mammograms, and usually, a lesion appearson both views. In our system, We can link the two ROIs todistinguish whether they are from the same lesion, while theyshare the same annotation information and this relationshipshould be emphasized in analysis. All of the operations abovewill be sent to the server and stored in the database for futureuse.When the user finished labeling one ROI and submitted theresults, our system will automatically extract the radiomicsfeatures and save them in the database. This operation reducedthe delay in radiomic experiments and remarkably improveduser experience. We offered more than 1300 dimensions ofradiomics features for each ROI. The features consist of onaverage ∼
90 features (Table.II) for the original and derivedimages (8 levels of Wavelet decompositions; Laplacian ofGaussian, Square, Square Root, Logarithm and Exponentialfilters). These feature extractors are implemented using thePyRadiomics [18] library. We also provided a robust featureextraction choice (Fig. 4). Users can perform perturbations onROI and extract perilesional features to get more stable results.IV. C
OMPUTATIONAL G RAPH
A Computational Graph is a directed graph represents anexperimental pipeline. The users are able to build a graph usingour GUI in several minutes instead of writing scripts in some
Fig. 4. Robust feature extraction. The left one is an example of ROIperturbation. The middle and right ones are visualizations of perilesionalfeatures. programming languages, which is non-trivial for radiologistswith limited coding experience. Once the graph is established,we can still easily modify it by adding/deleting or replacingsome nodes on it.
A. Graph Nodes
Each node in the graph is a basic computational unit, it hasspecified input and output format. A node receives data fromone or more nodes and the output will be passed to severalnodes. When an error raised on one node, it does not haveinfluences on other independent ones. E.g., the error occurredon leaf nodes will not influence others. Deep learning andradiomics laboratories have separate classes of nodes.
1) Radiomics:a) Dataloader:
The dataloader class is used to loadimage features. Users can choose to upload the existingfeatures or use the ’Annotated Data Input’ node to read thefeature information from our database which is generated bythe image labeler. The dataloader will also read the annotation and linking information. The data loaded will be split intoa training set and a validation set randomly. Users can alsoassign each image for training or validation manually. b) Preprocessing:
Data preprocessing is usually neededbefore feeding data to models. Our platform provides morethan five different nodes for data preprocessing. The mostcommon ones are normalizing to [ − , or [0 , which are’Max-Abs Scaler’ and ’Min-Max Scaler’ respectively. We alsopresent ’Standard Scaler’ and ’Normalizer’ which representfor projecting all data onto a unit ball and a standard normaldistribution separately. In addition, users are also allowedto write a self-defined transformation function in Python forpersonal use in the ’Custom Transformer’ node. c) Feature Selection: The radiomics feature extractorwill output more than 1300 dimensions of features whichis usually redundant. Feature selection is an important stepthat will heavily influence the model performance. To selectout the most useful features, we provide numerous featureselection methods. Specifically, we have five different kindsof nodes. (i) Variance Threshold: choosing the features withhigher variance. (ii) Select K Best / Select K Percentile: userscan set how many features s/he wants and what measure touse. The measure can be Chi-Square statistics, ANOVA-Fstatistics, or mutual information [20] between each feature andannotated label. (iii) Select From Model: selecting the featuresby the weights from a model. We have L1 Lasso, SVM,and many decision tree-based methods. (iv) Recursive FeatureElimination: recursively decrease the feature size according toa model. SVM and other tree-based methods are supported. (v)Select Stable Feature: select k features that stable to differentfeature selection algorithms. For any feature selection node,one can single-click the node result to check the features nameand corresponding importance score they get from the nodewhen the computation finished. d) Visualization:
Heatmap and dimension reduction arecommon ways to visualize the overall distribution of highdimension( ≥
3) data. They will give a comprehensive im-pression of the goodness of feature selection results. Thesedimension reduction methods can also be used to show theresults of hierarchical data clustering which is a commonunsupervised method for data analyzing. We provided 2 nodesfor data visualization, heatmap and T-SNE [21]. e) Machine Learning Models:
The key component in thecomputational graph is to choose the best machine learningmodel to learn the relationship between our processed featuresand the human annotated labels. We have implemented manyclassical classifiers including SVM, Logistic Regression, De-cision Tree, Gradient Boosting Decision Tree, Random Forest,XGboost, and Adaboost. As for the hyperparameter of eachmodel, we automatically apply grid search cross validationto choose the best ones which will be very convenient forthose who are not so familiar with these ML models. Aftertraining the model, We will draw the ROC curve and computethe AUC and AP score for the model. Also, the p-values ofthe hypothesis test for AUC and AP will be reported to showstatistical significance.
2) Deep learning:
Radiomic Models Classification SVMLogistic RegressionDecision TreeRandom ForestGBDTXGBoostRegression NomogramClustering KmeansHDBSCANDeep Learning Models Classification VGG-16,19Resnet-18,34,50,101Densnet-121,169Resnext-50, 101Incpetion V4Inception-Resnet-V2XceptionDetection SSDYOLOv2,v3Faster RCNNMask RCNNRetinaNetSegmentation U-netFCNMask RCNNTABLE IM
ODELS P ROVIDED IN D ARWIN P LATFORM . B
OTH AND IMAGESARE SUPPORTED a) Image Input:
Different from the radiomics modulewhich takes features as input, deep learning module has to takeimages as input. The Image Input class provides two kinds ofnodes, which allows users to choose to use the whole images oronly ROI regions for the experiment. Similar to the radiomicsmodule, the training and validation set will be randomly split. b) Preprocessing:
Deep learning preprocessing class in-cludes the normalization and standardization node similarto radiomics. It also provides a feature-based image align-ment node for multi-modal images. Augmentation is also anessential part of deep learning experiments. We integratedalbumentations library [22] for 2D cases, and implement 3Daugmentation functions. More than 20 types of augmentationincluding Flip, Transpose, Crop, Rotate, Blur, Elastic aresupported. c) DL models:
Our platform support all the 2D and3D tasks of Detection, Segmentation, and Classification. Theclassification node provides VGG, Resnet, Densenet, Resnext,Inception, and Xception. The object detection node includesFaster R-CNN, YOLO-v2, SSD, Mask RCNN, and RetinaNet.The segmentation network includes U-net, Mask RCNN, FCN,and its variants. Table I lists the network architectures currentlysupported in our platform. More state of the art models willbe added in regularly updating. d) Training:
In the training class, users can choose toupload the pretrain weights and then train or finetune thenetwork. All the hyper-parameters including the total epochs,batch size, learning rate, learning rate scheduler, optimizercan be manually selected. We also provide hyper-parametersearching algorithms including random search, first in firstout (FIFO), and hyperband. These algorithms are implementedwith Auto-Gluon library [23]. During training, there will be agraph shows information about GPU usage, estimated trainingtime, current iteration steps, training and validation loss,accuracy, AUC scores. This will help users to find potential
Shape Elongation GLCM AutocorrelationFlatness ClusterProminenceLeastAxisLength ClusterShadeMajorAxisLength ClusterTendencyMaximum2DDiameterColumn ContrastMaximum2DDiameterRow CorrelationMaximum2DDiameterSlice DifferenceAverageMaximum3DDiameter DifferenceEntropyMeshVolume DifferenceVarianceMinorAxisLength IdSphericity IdmSurfaceArea IdmnSurfaceVolumeRatio IdnVoxelVolume Imc1Firstorder 10Percentile Imc290Percentile InverseVarianceEnergy JointAverageEntropy JointEnergyInterquartileRange JointEntropyKurtosis MCCMaximum MaximumProbabilityMean SumAverageMeanAbsoluteDeviation SumEntropyMedian SumSquaresMinimumRangeRobustMeanAbsoluteDeviationRootMeanSquaredSkewnessTotalEnergyUniformityVarianceGLDM DependenceEntropy GLRLM GrayLevelNonUniformityDependenceNonUniformity GrayLevelNonUniformityNormalizedDependenceNonUniformityNormalized GrayLevelVarianceDependenceVariance HighGrayLevelRunEmphasisGrayLevelNonUniformity LongRunEmphasisGrayLevelVariance LongRunHighGrayLevelEmphasisHighGrayLevelEmphasis LongRunLowGrayLevelEmphasisLargeDependenceEmphasis LowGrayLevelRunEmphasisLargeDependenceHighGrayLevelEmphasis RunEntropyLargeDependenceLowGrayLevelEmphasis RunLengthNonUniformityLowGrayLevelEmphasis RunLengthNonUniformityNormalizedSmallDependenceEmphasis RunPercentageSmallDependenceHighGrayLevelEmphasis RunVarianceSmallDependenceLowGrayLevelEmphasis ShortRunEmphasisShortRunHighGrayLevelEmphasisShortRunLowGrayLevelEmphasisGLSZM GrayLevelNonUniformity NGTDM GrayLevelNonUniformityGrayLevelNonUniformityNormalized GrayLevelNonUniformityNormalizedGrayLevelVariance GrayLevelVarianceHighGrayLevelZoneEmphasis HighGrayLevelZoneEmphasisLargeAreaEmphasis LargeAreaEmphasisLargeAreaHighGrayLevelEmphasis LargeAreaHighGrayLevelEmphasisLargeAreaLowGrayLevelEmphasis LargeAreaLowGrayLevelEmphasisLowGrayLevelZoneEmphasis LowGrayLevelZoneEmphasisSizeZoneNonUniformity SizeZoneNonUniformitySizeZoneNonUniformityNormalized SizeZoneNonUniformityNormalizedSmallAreaEmphasis SmallAreaEmphasisSmallAreaHighGrayLevelEmphasis SmallAreaHighGrayLevelEmphasisSmallAreaLowGrayLevelEmphasis SmallAreaLowGrayLevelEmphasisZoneEntropy ZoneEntropyZonePercentage ZonePercentageZoneVariance ZoneVarianceTABLE IIR
ADIOMIC F EATURES P ROVIDED IN D ARWIN P LATFORM errors in an early stage. Same as the radiomics module, usersare able to view the ROC curve, AUC score, confusion matrixafter training is completed. e) Visualization:
In deep learning, a commonly usedway for visualization is class activation mapping (CAM). Inthis node, users need to upload the weights, select the targetimages, and specify the layer to visualize. The generatedheatmap will help physicians understand how the networkmakes a decision, and could carry some potential underlyingmessages with medical meanings. f) Ensemble:
Ensembling is a widely used trick in deeplearning competitions. Ensemble a group of models usuallygives better performance than any single one. Here we presenttwo kinds of ensemble methods: voting and averaging. Ra-diomics models are also supported in this ensemble node.
B. Graph Building
There are three steps to build a graph, selecting the nodes,dragging these nodes to the canvas, and connecting them bymouse. Once the graph is built, the users only need to click the’Run’ button. The graph will be sent to the server and the dataflow will pass through each node sequentially until all nodeshave finished their computations. After that, one can click eachnode to check the output of it. The connection between allnodes is flexible. For instance, one can feed the radiomicsfeature to preprocessing nodes, feature selection nodes, andmachine learning nodes directly. To compare different featureselection methods simultaneously, s/he could just feed theradiomics features to several feature selection nodes parallellyspecified by different functions. The process is similar formachine learning nodes when comparing different models.Additionally, visualization nodes can receive a set of featuresand corresponding values which means it can follow featurenodes, preprocessing nodes, and feature selection nodes. Fig. 2shows the completed computational graph for a radiomicexperiment.
C. Model Retrieval
Once an experiment completed, the computational graphand corresponding model weights will be saved in the modeldatabase. Users can retrieve the history results and comparethe performance. For each history record, users are allowed tospecify a new test set for the model test.V. E
XPERIMENTS
A. Radiomic Performance Evaluation
To verify the performance of the radiomic module in theplatform, we carried out a demo project of identifying an ROIis on the left or the right lung. We randomly chose 19 imagesfrom LIDC-IDRI [24] and manually annotated 424 3D ROIs.The experiment runs on a server with 12 cores Intel i7-8700KCPU @ 3.70GHz. The average time for extracting features ofone ROI is 1.39 seconds. We use 339 ROIs for training, 85ROIs for validation.We constructed a computational graph in Fig.5. Forty fea-tures are selected from logistic regression, and then feed to
Fig. 5. Computational Graph for Lung ROI Classification. the SVM classifier. The running time of this whole pipelineis 234.25 seconds. Classification results are shown in Fig.6.It achieves a high performance with over 0.97 AUC and 0.99average precision (AP) on the validation set.
Fig. 6. Lung ROI Classification Results.
B. Deep Learning Performance Evaluation
We collected 10,701 lung CT scans from 10 hospitalsand constructed a model for lung nodules detection. Thecomputational graph is shown in Fig. 7. We randomly chose260 scans for validation, the rest for training. A 3D RetinaNet
Fig. 7. Computational Graph for Lung Nodules Detection. model trained from scratch for this experiment. The trainingpipeline runs 8 hours on a sever with 8 TITAN RTX.Fig.8 shows the Free-Response ROC (FROC) result. Themodel achieves a high performance with sensitivity over 0.95at 8 false positive per scan on the validation set.
Fig. 8. Lung Nodules Detection Results.
C. Case Study: Identifying Recurrent Glioma from Radiation-induced Temporal Lobe Necrosis in Contrast MRI
Doctors collected examinations of 83 patients for this study.All the patients have tumor-shape areas in their CE-T1WIafter 2 weeks of operation. We balanced split 70% of theexaminations for training, the rest for test.The computational graph is shown in Fig.9. The input 1223dimension features are reduced to 100 by ANOVA F-value.Doctors further selected 4 features based on the weights oflogistic regression and SVM.
Fig. 9. The Computational Graph for Glioma and Necrosis Classification.
Fig.10 shows the 4 features and the corresponding featureimportance in SVM. Tumors usually have a higher level ofheterogeneity than necrosis, thus an ROI with higher entropyis more likely to be a glioma. Maximal correlation coefficient(MCC) is a measure of the complexity of the texture. Skewnessdescribes the brightness distribution of the ROI. Strength isan indication of the image primitives. The 4 features suggestthe images of temporal lobe necrosis have a higher level ofunbalance in distribution, and have more complex textures.This is consistent with the histological features of the twodiseases. When the recurrent glioma occurs, tumor cells growrapidly and blood vessels are forming [25]. In radiation injury,multiple factors including telangiectasias, thrombosis, fibrinoidnecrosis of vessel walls, hyaline degeneration, and hemorrhageare performing, which led the images with more complex con-tents [26]. The AUC scores for the four features JointEntropy,Strength, MCC, Skewness in glioma classification are 0.78,0.74, 0.74, 0.72 respectively.
Fig. 10. Features Selected in Glioma and Necrosis Classification.
D. Case Study: Predict the Invasibility of Adenocarcinoma inPure Ground-Glass Nodules.
In this project, doctors studied the invasibility of adeno-carcinoma in pure ground-glass nodules (pGGNs). Doctorscollected the CT examinations of 136 patients, uploaded themto our system, and manually segmented ROIs. Our featureextractor automatically extracted 1223 features.
Fig. 11. The Computational Graph for pGGNs Prediction
Doctors divided the data into a training set (95 patients),and a test set (41 patients). They constructed a computationalgraph (Fig. 11) in the laboratory of our Darwin Research Plat-form. The 1223 features are normalized into 0-1 for prepos-sessing. Doctors used Select-K-Best and Recursive-Feature-Elimination using LASSO for feature selection and ultimatelygot 6 features. The SVM model is used for classification.The platform automatically generated the coefficients andloss paths in the feature selection period. It also presentsthe coefficients of the final selected features. The heatmapof the selected features is shown in Fig. 12. Table III andFig. 13 show the model performance on test set. It achievesan accuracy of over 90%, which outperform the experienceddoctors.
Accuracy Sensitivity SpecificityRadiologist 1 (10-year experience) 75.61% 50.00% 86.21%Radiologist 2 (20-year experience) 80.49% 75.00% 82.76%SVM prediction 90.24% 91.67% 89.66%TABLE IIIP
REFORMANCE OF THE
SVM
MODEL ON TEST SET
VI. C
ONCLUSION
In this paper, we present the DARWIN research platform.Radiologists could use the platform to analyze medical imag-ing data. The DARWIN research platform streamlines the col-lection, annotation, data preprocessing, model training, modelselection, and analysis. The platform provides various choicesfor users with a user-friendly GUI. Physicians could conducttheir experiments just by clicking the mouse and draggingthe related icons. The platform provides ways to visualize thehigh-dimension data, allowing users to comprehend the learn-ing period straightforwardly. The generated experiment reportsalso facilitate doctors with their research papers, enabling an
Fig. 12. Heatmap of the Selected Features in pGGNs predictionFig. 13. ROC curve of the SVM model on test set effortless way for radiologists to carry out machine learningexperiments. R
EFERENCES[1] S. M. McKinney, M. Sieniek, V. Godbole, J. Godwin, N. Antropova,H. Ashrafian, T. Back, M. Chesus, G. C. Corrado, A. Darzi et al. ,“International evaluation of an ai system for breast cancer screening,”
Nature , vol. 577, no. 7788, pp. 89–94, 2020.[2] A. Vial, D. Stirling, M. Field, M. Ros, C. Ritz, M. Carolan, L. Holloway,and A. A. Miller, “The role of deep learning and radiomic feature ex-traction in cancer-specific predictive modelling: a review,”
TranslationalCancer Research , vol. 7, no. 3, pp. 803–816, 2018.[3] A. Wibmer, H. Hricak, T. Gondo, K. Matsumoto, H. Veeraraghavan,D. Fehr, J. Zheng, D. Goldman, C. Moskowitz, S. W. Fine et al. ,“Haralick texture analysis of prostate mri: utility for differentiatingnon-cancerous prostate from prostate cancer and differentiating prostatecancers with different gleason scores,”
European radiology , vol. 25,no. 10, pp. 2840–2850, 2015. [4] A. Dekker, S. Vinod, L. Holloway, C. Oberije, A. George, G. Goozee,G. P. Delaney, P. Lambin, and D. Thwaites, “Rapid learning in practice:A lung cancer survival decision support system in routine patient caredata,”
Radiotherapy and Oncology , vol. 113, no. 1, pp. 47–53, 2014.[5] C. Parmar, P. Grossmann, D. Rietveld, M. M. Rietbergen, P. Lambin,and H. J. Aerts, “Radiomic machine-learning classifiers for prognosticbiomarkers of head and neck cancer,”
Frontiers in oncology , vol. 5, p.272, 2015.[6] S. H. Hawkins, J. N. Korecki, Y. Balagurunathan, Y. Gu, V. Kumar,S. Basu, L. O. Hall, D. B. Goldgof, R. A. Gatenby, and R. J. Gillies,“Predicting outcomes of nonsmall cell lung cancer using ct imagefeatures,”
IEEE access , vol. 2, pp. 1418–1426, 2014.[7] C. Parmar, P. Grossmann, J. Bussink, P. Lambin, and H. J. Aerts, “Ma-chine learning methods for quantitative radiomic biomarkers,”
Scientificreports , vol. 5, p. 13087, 2015.[8] W. Shen, M. Zhou, F. Yang, C. Yang, and J. Tian, “Multi-scale convo-lutional neural networks for lung nodule classification,” in
InternationalConference on Information Processing in Medical Imaging . Springer,2015, pp. 588–599.[9] S. Hussein, K. Cao, Q. Song, and U. Bagci, “Risk stratification oflung nodules using 3d cnn-based multi-task learning,” in
Internationalconference on information processing in medical imaging . Springer,2017, pp. 249–260.[10] K. Yan, M. Bagheri, and R. M. Summers, “3d context enhanced region-based convolutional neural network for end-to-end lesion detection,” in
International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2018, pp. 511–519.[11] K. Kamnitsas, C. Ledig, V. F. Newcombe, J. P. Simpson, A. D. Kane,D. K. Menon, D. Rueckert, and B. Glocker, “Efficient multi-scale 3dcnn with fully connected crf for accurate brain lesion segmentation,”
Medical image analysis , vol. 36, pp. 61–78, 2017.[12] P. F. Christ, M. E. A. Elshaer, F. Ettlinger, S. Tatavarty, M. Bickel,P. Bilic, M. Rempfler, M. Armbruster, F. Hofmann, M. DAnastasi et al. ,“Automatic liver and lesion segmentation in ct using cascaded fullyconvolutional neural networks and 3d conditional random fields,” in
International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2016, pp. 415–423.[13] G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, and A. V.Dalca, “An unsupervised learning model for deformable medical imageregistration,” in
Proceedings of the IEEE conference on computer visionand pattern recognition , 2018, pp. 9252–9260.[14] Y. D. Cid, J. Castelli, R. Schaer, N. Scher, A. Pomoni, J. O. Prior, andA. Depeursinge, “Quantimage: an online tool for high-throughput 3dradiomics feature extraction in pet-ct,” in
Biomedical Texture Analysis .Elsevier, 2017, pp. 349–377.[15] L. Zhang, D. Fried, X. Fave, L. Hunter, J. Yang, and L. E. Court, “Ibex:An open infrastructure software platform to facilitate collaborative workin radiomics,”
Medical Physics , vol. 42, no. 3, pp. 1341–1353, 2015.[16] C. Nioche, F. Orlhac, S. Boughdad, S. Reuze, J. Goyaouti, C. Robert,C. Pellotbarakat, M. Soussan, F. Frouin, and I. Buvat, “Lifex: Afreeware for radiomic feature calculation in multimodality imaging toaccelerate advances in the characterization of tumor heterogeneity,”
Cancer Research , vol. 78, no. 16, pp. 4786–4789, 2018.[17] R. Yuan, S. Shi, J. Chen, and G. Cheng, “Radiomics in rayplus: a web-based tool for texture analysis in medical images,”
Journal of DigitalImaging , vol. 32, no. 2, pp. 269–275, 2019.[18] J. J. Van Griethuysen, A. Fedorov, C. Parmar, A. Hosny, N. Aucoin,V. Narayan, R. G. Beets-Tan, J.-C. Fillion-Robin, S. Pieper, and H. J.Aerts, “Computational radiomics system to decode the radiographicphenotype,”
Cancer research , vol. 77, no. 21, pp. e104–e107, 2017.[19] R. Adams and L. Bischof, “Seeded region growing,”
IEEE Transactionson pattern analysis and machine intelligence , vol. 16, no. 6, pp. 641–647, 1994.[20] R. G. Lomax,
Statistical concepts: A second course . Lawrence ErlbaumAssociates Publishers, 2007.[21] L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,”
Journalof machine learning research , vol. 9, no. Nov, pp. 2579–2605, 2008.[22] A. V. Buslaev, A. Parinov, E. Khvedchenya, V. Iglovikov, and A. A.Kalinin, “Albumentations: Fast and flexible image augmentations,”
Information-an International Interdisciplinary Journal , vol. 11, no. 2,p. 125, 2020.[23] N. Erickson, J. Mueller, A. Shirkov, H. Zhang, P. Larroy, M. Li, andA. Smola, “Autogluon-tabular: Robust and accurate automl for structureddata,” arXiv preprint arXiv:2003.06505 , 2020.[24] S. G. Armato III, G. McLennan, L. Bidaut, M. F. McNitt-Gray, C. R.Meyer, A. P. Reeves, B. Zhao, D. R. Aberle, C. I. Henschke, E. A.Hoffman et al. , “The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database oflung nodules on ct scans,”
Medical physics , vol. 38, no. 2, pp. 915–931,2011.[25] A. Al Sayyari, R. Buckley, C. McHenery, K. Pannek, A. Coulthard, andS. Rose, “Distinguishing recurrent primary brain tumor from radiationinjury: a preliminary study using a susceptibility-weighted mr imaging-guided apparent diffusion coefficient analysis strategy,”
American jour-nal of neuroradiology , vol. 31, no. 6, pp. 1049–1054, 2010.[26] B. C. Oh, P. G. Pagnini, M. Y. Wang, C. Y. Liu, P. E. Kim, C. Yu,and M. L. Apuzzo, “Stereotactic radiosurgery: Adjacent tissue injuryand response afterhigh-dose single fraction radiation: Part ihistology,imaging, and molecularevents,”