Predicting Rate of Cognitive Decline at Baseline Using a Deep Neural Network with Multidata Analysis
Sema Candemir, Xuan V. Nguyen, Luciano M. Prevedello, Matthew T. Bigelow, Richard D.White, Barbaros S. Erdal
PPredicting Rate of Cognitive Decline at BaselineUsing a Deep Neural Network with Multidata Analysis
This study has been accepted and published in the SPIE Journal of Medical Imaging. https: // doi.org / / Sema Candemir, Xuan V. Nguyen, Luciano M. Prevedello, Matthew T. Bigelow, Richard D.White, Barbaros S. Erdal,for the Alzheimer’s Disease Neuroimaging Initiative Laboratory for Augmented Intelligence in Imaging of the Department of Radiology, The Ohio State University College of Medicine
Abstract
Purpose:
This study investigates whether a machine-learning-based system can predict the rate of cognitive decline in mildlycognitively impaired patients by processing only the clinical and imaging data collected at the initial visit.
Approach:
We built a predictive model based on a supervised hybrid neural network utilizing a 3-Dimensional ConvolutionalNeural Network to perform volume analysis of Magnetic Resonance Imaging and integration of non-imaging clinical data at thefully connected layer of the architecture. The experiments are conducted on the Alzheimer’s Disease Neuroimaging Initiativedataset.
Results:
Experimental results confirm that there is a correlation between cognitive decline and the data obtained at the first visit.The system achieved an area under the receiver operator curve (AUC) of 0 .
70 for cognitive decline class prediction.
Conclusion:
To our knowledge, this is the first study that predicts “slowly deteriorating / stable” or “rapidly deteriorating” classesby processing routinely collected baseline clinical and demographic data (Baseline MRI, Baseline MMSE, Scalar Volumetric data,Age, Gender, Education, Ethnicity, and Race). The training data is built based on MMSE-rate values. Unlike the studies in theliterature that focus on predicting Mild Cognitive Impairment-to-Alzheimer‘s disease conversion and disease classification, weapproach the problem as an early prediction of cognitive decline rate in MCI patients. Keywords:
Computer-aided detection / diagnosis, Alzheimer’s Disease in the early stages, Cognitive decline, Mild cognitiveimpairment, Baseline Visit
1. Introduction
Mild Cognitive Impairment (MCI) is an intermediate stagebetween Cognitively Normal (CN) and Alzheimer’s Disease(AD) [1]. The patients in the MCI phase have a varied prog-nosis such that the cognitive functions of some MCI patientsdeteriorate, while others remain stable or improve [2][3]. Al-though there has not been any successful treatment to reversecognitive decline, to date, therapy to decelerate its progressionis likely to be most beneficial if it is applied early [4][5]. In thisstudy, we investigate whether a machine learning-based systemcan predict the “rate of cognitive decline” in patients with di-agnosed MCI by processing only the clinical and imaging dataobtained at the initial visit. Data used in preparation of this article were obtained from the Alzheimer’sDisease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). Assuch, the investigators within the ADNI contributed to the design and imple-mentation of ADNI and / or provided data but did not participate in analysisor writing of this report. A complete listing of ADNI investigators can befound at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf Prior studies have reported on biomarkers and the predic-tion of MCI-to-AD conversion [6][7][8][4][9]. Unlike earlierstudies, we investigate the feasibility of predicting the “rate ofcognitive decline” in MCI patients at the first visit by process-ing only the baseline MRI and routinely collected clinical data.We built a deep-learning-based predictive model that integratesimaging and non-imaging demographic and clinical data in thesame neural network architecture. The system consists of 3main inputs: 1) MRI brain images, 2) scalar volumetric fea-tures, and 3) demographic and clinical data. MRI brain scansare provided to the network as sequential DICOM images andprocessed through a 3-Dimensional Convolutional Neural Net-work (3D-CNN). The scalar volumetric features extracted us-ing FreeSurfer [10] represent selected brain substructures andincluded total intracranial volume, whole-brain volume, and re-gional volumes of the hippocampus, entorhinal cortex, fusiformgyrus, and medial temporal lobe; this scalar data is integratedinto the system at the fully connected layer of the architecture.The demographic and clinical information included in the neu-ral network architecture are the ones that are routinely collectedat the initial clinical visit, and include age, gender, years of ed-
Preprint submitted to – October 7, 2020 a r X i v : . [ q - b i o . Q M ] O c t cation, ethnicity, race, and baseline mini-mental score exam-ination score. The proposed predictive model is illustrated inFigure 2.We supervised the predictive model with the change in“Mini-Mental State Examination (MMSE) scores [11][12]”with the MCI subjects grouped clinically according to (i) slowcognitive decline, and (ii) fast cognitive decline. The resultingmodel processes the clinical data obtained at the baseline visitand predicts the patient’s cognitive condition as either “slowlydeteriorating / stable” or “rapidly deteriorating”. The analysisis performed on publicly available Alzheimer’s Disease Neu-roimaging Initiative (ADNI) dataset (c.f., Section 2.1).
2. Materials and Methods
The data used in this study were obtained from theADNI [13], which is an ongoing multi-center study. Theprimary goal of ADNI has been to test whether serial MRI,positron emission tomography (PET), other biological markers,and clinical and neuropsychological assessment can be com-bined to measure the progression of MCI and early AD. Thesubjects in the dataset were diagnosed as AD, MCI, SubjectiveMemory Concern, or Cognitively Normal (CN) based on Mini-Mental State Examination (MMSE) scores. The enrolled sub-jects received multiple longitudinal follow-up visits over sev-eral years, as specified by the ADNI protocol. For our research,we utilize data on ADNI patients who were clinically diagnosedas MCI at their baseline visits. A total of 569 subjects were in-cluded. The demographics and clinical characteristics of thesubjects are summarized in Table 1.
Label MCINumber of Patients 569Age - mean [range] 76 [55 - 92]Gender [Female - Male] F:241 M:328Education - mean [range] 16 [6 - 20]Ethnicity Not Hispanic / HispanicRace White - Black - AsianBaseline MMSE score - mean [range] 28 [23 - 30]Table 1: Characteristics of the study subjects. MMSE- Mini-Mental State Ex-amination. MCI - Mild Cognitive Impairment.
The MMSE, which is a 30-point test, is a cognitive assess-ment tool [11][12], and we used the rate of decline in MMSEscores to supervise the system. Changes in MMSE scores infollow-up visits demonstrate the patient’s condition in terms ofcognitive capabilities. A decrease in MMSE score reflects de-terioration in cognitive capabilities; if a patient’s cognitive ca-pability is stable, the MMSE scores remain relatively stable.We model the change in MMSE scores by fitting a line to thescores obtained at follow-up visits. The slope of the line indi-cates the rate of cognitive loss. A patient who has faster cogni-tive deterioration would have a higher absolute value of slope. A slope close to zero indicates that the cognitive decline is sta-ble. In this document, the “Rate of Cognitive Decline” term willrefer to the slope of the decline. The predictive model is binary.Therefore, the rate of cognitive decline is converted to binaryvariables using a threshold of -0.05 points / month, such that pro-gressive rapidly deteriorating level of cognition is defined as arate of decrease exceeding 0.6 points / year. This threshold ap-proximates the mean and the median rate change for this cohort.The rate of cognitive decline distribution of the study subjectsis shown in Figure 1. Figure 1: The rate of cognitive decline distribution of the study subjects
The predictive model learns the mapping function from inputdata to the target output. Let V be the imaging sequence, D bethe corresponding clinical data, y be the target class, and f ( . )represent the mapping function between input data and outputlabels. The model can be formulated as y i = f ( V i , D i ) (1)for each subject i in N, where N is the number of patients withMCI in the training data. Clinical data include age, gender,baseline MMSE score, education, ethnicity, and race. We alsouse brain volumes as supporting scalar features, which are com-puted with an open-source library (FreeSurfer) that analysesand visualizes structural and functional neuroimaging data [10].Specifically used scalar features are whole-brain volume and re-gional volumes of the hippocampus, entorhinal cortex, fusiformgyrus, and medial temporal lobe. The brain volumes of eachsubject are available in the ADNI [13]. We pose the problem asa supervised classification task, with training subjects classifiedinto two groups based on the rate of MMSE (c.f., Figure 1). Theoutput variable y ∈ (0 ,
1) denotes the target classes, 0 represents“slowly deteriorating / stable” class, and 1 represents “rapidlydeteriorating” class. The proposed system is illustrated in Fig-ure 2.2 igure 2: An illustration of the hybrid prediction system. MPRAGE: Magnetization Prepared Rapid Gradient Echo. MRI: Magnetic Resonance Imaging. ReLU:Rectified Linear Unit. Batch Norm: Batch Normalization. We apply pre-processing techniques to each MRI volume V and corresponding clinical data D before the training. The MRIsequences are skull-stripped, which includes removal of non-cerebral tissue (calvaria, scalp, and dura) [14]. The skull-stripalgorithm, which is based on U-Net architecture [15] trained onskull-stripping datasets [16], reduces the processing size of vol-umes, thereby increasing computational speed during the train-ing. After the skull-strip, we have applied MRI scale standard-ization [17] to mitigate the intensity di ff erences between theMRI sequences.The neural network architectures require the inputs to bescaled in a consistent way for a stable and faster convergence.Therefore, we normalize the images, scalar volumetric features,and demographic and clinical data into the range between 0 and1. The scalar regional volume features are divided by each sub-ject’s whole-brain volume size for normalization. The demo-graphic and clinical data contains categorical values (e.g., gen-der and ethnicity) that are converted into numeric data. Therange for the numeric demographic data is between 0 and 1 toensure numerical stability. The deep learning algorithm is based on a supervised neuralnetwork that has a hybrid architecture with two main compo-nents: (i) a 3D Convolutional Neural Network (3D-CNN) thatlearns the brain morphology and patterns, and (ii) integration ofscalar volumetric features and non-imaging data (demographicand clinical information) at the fully connected layer.
The 3D-CNN processes MRI scans models the patterns andstructures in brain volume. Earlier layers of the model capturethe low-level features of brain details, while higher-level layerslearn abstract features. The layout of the 3D-CNN architec-ture is employed from [14] that is proposed for MRI analysisfor AD / CN classification. The architecture consists of threebatches of convolutional layers with kernels of 3 × × , 3 , and 4 sizes are usedfor feature reduction and spatial invariance. The architectureuses ReLU (Rectified Linear Unit) activation that introducesnonlinearity to the system [19]. The output of the deepest con-volutional layer is flattened and fed to the fully connected layer.The architecture parameters are listed in Table 2. The clinical and demographic information presumably con-tains additional information that would help the classificationdecision. To incorporate the non-imaging data for assessmentof its impact, we have changed the standard CNN architecture.The convolutional part of CNN is the feature extraction compo-nent of the architecture; the fully connected layer part is theclassifier component. The output of the final pooling layer,which holds the imaging features, is flattened and fed into thefully connected layer. The flattened imaging features and non-3maging features create a concatenated vector as an input tothe dense layer. The rest part of the architecture is the classi-fier component of the hybrid system, trained with this vector toform the final prediction model. The concatenated dense layeris then followed by a dropout layer [20] in which the systemtemporarily ignores randomly selected neurons during the train-ing to prevent the system from memorizing the training datawith the intent to decrease overfitting. The final layer is anotherdense layer with a softmax activation function with two nodesthat provide probabilities for “slowly deteriorating / stable” classand “rapidly deteriorating” class. The architecture parametersare listed in Table 2. The voxel-based CNNs are prone to over-fitting due to high-dimensional data, large number of parameters, and relativelysmall number of cases to optimally train the system [21, 14, 22].To address the relatively low number of patients, we utilizedaugmentation strategies. We flipped MRI volumes such that leftand right hemispheres are reversed [14] and randomly tilted atless than 5 ◦ . We have also employed the regularization tech-niques of dropout [20] and weight decays [23] in order to in-crease the generalization capacity of the model. The parametersof dropout and weight decays are listed in able 3.
3. Experiments
The dataset used in the study consists of 569 subjects withMPRAGE (MRI) scans and corresponding clinical data (c.f.,Section 2.1). We perform 5-fold cross-validation to reduce theperformance di ff erence due to relatively small size datasets andprovide more robust generalization performance. At each fold,60% of the dataset is used to train the model, 20% is used formodel validation, and 20% of the dataset is used to test themodel.The training parameters are listed in Table 3. We train themodel using Adam optimizer [24], which provides faster con-vergence due to the velocity and acceleration components. As atraining strategy, we monitor the model performance and usetwo early stopping callbacks to stop the training before themodel begins to overfit [25]. We set a large epoch value (c.f.,Table 3, max epoch 400) as an upper bound iteration. The num-ber of training iteration is decided automatically based on themodel performance on the validation and training set. If the val-idation loss has started to increase during the training process,the system triggers the early stopping callback. If the valida-tion loss continues to increase for another 20 iterations, thenthe system stops the training. The continuous increase in vali-dation loss is an indication of overfitting. The second callbackis monitoring the training accuracy. If the training accuracyreaches the maximum value, the early stopping callback stopsthe training due to an indication of no further improvement inthe model. The weights are randomly initialized from scratch. The model is developed in Python (version 3.6.8) usingTensorflow Keras API (version 2 . . − t f ) and trained on anNvidia Quadro GV100 system with 32GB graphics cards withCUDA / CuDNN v9 dependencies for GPU acceleration.
We built 3 models: (i) an imaging model based on a 3D-CNN that processes brain MRI, (ii) a hybrid model that com-bines the 3D-CNN component with brain-volume scalar dataand demographic and clinical information, and (iii) a modelthat processes brain-volume scalar data and demographic andclinical information. We assess the models’ prediction perfor-mance in terms of accurately classifying the cognitive declineon a test dataset at each test fold and average the evaluation met-ric scores across all the models. The performance metrics usedin the study are Sensitivity, Specificity, Accuracy, PPV, NPV,and AUC. Table 4 lists the performance metrics.
The correlation between the morphological changes inthe brain (e.g., parenchymal volume loss) and AD isknown [26][27]. Based on a prior study [28], (i) MCI sub-jects have medium atrophy of hippocampus; (ii) the brain mor-phology in non-converters is similar to brain morphology inCN, and converters are more similar to AD, and (iii) convert-ers have more severe deterioration of neuropathology than non-converters. Due to the correlation between the pathologicalchanges in brain morphology and the AD stages, we first mea-sured how much we could predict the pace of the cognitive de-cline of patients by processing only the baseline MRI scansthrough a 3D-CNN. The system achieved 0 .
67 AUC for pre-dicting the cognitive-decline class by processing only baselineMRI sequences. The Receiver Operator Characteristic (ROC)curve for this experiment is shown in Figure 3.(a).
The hybrid model processes the MRI sequences, brain vol-ume scalar data, and demographic information (age, gender,years of education, ethnicity, and race). Table 4 lists the per-formance scores obtained with the proposed system in terms ofmean and standard deviation across the cross-validated folds.The system achieved an accuracy of 63 . . . . .
67. Adding thebrain volume and demographic information as scalar values tothe system increased the system performance from 0 .
67 AUCto 0 .
70 AUC as shown in Figure 3.(b).
The voxel-based convolutional neural networks are prone toover-fitting due to high dimensional data, large number of pa-rameters, but relatively low number of subject to optimally trainthe system [21, 14, 22]. Although we utilize several regulariza-tion techniques, we still observed over-fitting due to the 3D-CNN module of the hybrid system. In this experiment, we re-move the 3D-CNN module of the hybrid model and run the4 ayer(Type) Input Size (output shape) × ×
83 0Conv3D 130 × ×
83 32 – 896Conv3D 130 × ×
83 32 – 27680Batch Norm 130 × ×
83 32 – 128Max Pooling 3D 65 × ×
41 32 2 0Conv3D 65 × ×
41 64 – 55360Conv3D 65 × ×
41 64 – 110656Batch Norm 65 × ×
41 64 – 256Max Pooling 3D 21 × ×
13 64 3 0Conv3D 21 × ×
13 128 – 221312Conv3D 21 × ×
13 128 – 442496Batch Norm 21 × ×
13 128 – 512Max Pooling 3D 5 × × Input (Metadata)
Dense 512 3932672 12
Concatenate
524 0DropoutDense 256 134400DropoutDense (Out) 2 514Table 2: The architecture parameters of the proposed model. Each row represents a layer, and the input of a particular layer is the output of the previous layer. Thereare two input layers: (1) processing MRI sequences, (2) processing meta-data. The meta-data input layer is added to the architecture at the Dense layer through aConcatenate function. (a) (b)
Figure 3: The plots depict the system performance for predicting cognitive-decline class. (a) The predictive model processed only MRI sequences with 3D-CNN;average AUC is 0.67. (b) The hybrid predictive model is based on MRI sequences with 3D-CNN, brain-volume scalar data and non-imaging clinical data; theaverage AUC = arameter ValueProcessing dimension of each MRI volume 116 × ×
83 voxelsOptimizer Adam [24]Learning Rate 0 . β β (cid:15) − Loss Function categorical cross-entropyBatch Size 16Drop-out keep rate 0.5 L ffi cient 0.5 L ffi cient 1Early stopping max epoch 400Early stopping patience epoch 20Table 3: Implementation Details - Parameters experiments only using brain − volume scalar data with non-imaging clinical data. The system achieved 0 .
70 average AUCfor cognitive decline class prediction as shown in Figure 4.(a).
The literature has several techniques to combine data fromdi ff erent resources. We summarized these studies in Sec-tion 3.4. We utilized CNN-based architecture as a classifierand combined the imaging features with non-imaging data ata dense layer in a straightforward way, as in [14][29]. Notethat imaging features or non-imaging features should not dom-inate the training. To our knowledge, there is not any CNN-based study that adjusts the e ff ects of modules on the predic-tion results. External weights can be used to adjust the contri-bution of one module over the other modules. However, theseweights are additional hyper-parameters of the system and canbe decided on train / validation subsets. In order to observe howdi ff erent weights a ff ect the final decision, we have conductedan additional experiment. We have multiplied scalar input val-ues (scalar brain volumes and demographic data) with externalweight (coe ffi cient value) and kept the MRI imaging weightsthe same. The external weight adjusts the contribution of thenodes to the classifier decision. The coe ffi cient set is 0.5, 1,and 1.5. Figure 4.(b) shows the performance of the model withdi ff erent coe ffi cient values. The correlation between themorphological changes in the brain (e.g., parenchymal volumeloss) and AD has been known for years [26][27]. The literaturehas several studies with quantitative analysis of brain MRI toassess AD (e.g., classification of AD vs. CN) [30] [14]. Thesestudies measured the volumes, cortical thickness, or shape ofvarious structures such as the hippocampus [28] or the wholebrain [14] to assess the disease and the severity of the diseaseas a percentage of volume. For example, in [14], a 3D-CNN-based framework is proposed to learn the imaging characteris-tics of AD and CN through convolutional layers. The model isfurther modified to diagnose MCI, the prodromal stage of AD.In our study, instead of anatomy-disease correlation, we focus on anatomy-function correlation. A comprehensive review ofAD detection / classification can be found in [31]. MCI-to-AD Conversion.
Many researchers have attemptedto predict the conversion of MCI to AD using the correla-tion between the morphological changes in the brain and dis-ease progression [28]. The volumetric analysis of the brain(especially the hippocampus and entorhinal cortex) producessatisfactory results in predicting conversion to AD [32] [33].One of the most commonly employed classifiers is the Sup-port Vector Machine (SVM) [8] [34]. Recent studies utilizedeep-learning-based approaches using neural network classi-fiers [28][35]; they show that predicting progressive MCI ordetecting MCI patients who later progress to AD is still a goalof ongoing research [4] [9].
Hybrid Models.
The clinical and demographic informationcontains additional data that contributes to the algorithm deci-sion. To our knowledge, the algorithms that incorporate clin-ical data into MRI data results are limited [36]. In [14], ageand gender information are concatenated with imaging fea-tures in a 3D-CNN architecture through additional nodes atthe fully connected layer. In [28], a CNN was trained withlocal patches extracted from the hippocampus and combinedwith FreeSurfer brain data. The algorithm extracted imag-ing features through a CNN architecture and processed imag-ing features and FreeSurfer brain data using principal compo-nent analysis following by the Lasso regression algorithm. Theprocessed features were provided as input to a NN algorithmthat combined these features. In [36], the authors proposed amulti-modal fusion model to classify MCI and CN cases. Thestudy employed two Multi-Layer Perceptron (MLP) architec-tures to train non-imaging data and 2D-CNN to train the imag-ing data. The predictive model processed the test scores ofmini-mental state examinations, the Wechsler memory scalefor logical memory, and MRI sequences. The predictions fromeach NN block were then combined using majority voting. An-other interesting study that combined baseline MRI with base-line cognitive test scores was proposed in [8]. The cognitivescores used in the study were Rey’s Auditory Verbal LearningTest, Alzheimer’s Disease Assessment Scale cognitive subtest,MiniMental State Examination, Clinical Dementia Rating Sumof Boxes, and Functional Activities Questionnaire. The MRI,age, and cognitive measurements were integrated as input fea-tures to a random forest classifier. Another hybrid method wasproposed in [37] that combined MRI and FDG-PET images atmultiple scales within a NN framework. Six independent deepNNs processed di ff erent scales of image sequences. AnotherNN fused the features extracted from these first 6 DNN. Thealgorithm was proposed to classify AD and NC cases. One ofthe most recent studies is [35] that combined MRI sequences,demographic, neuropsychological, and APOe4 genetic data topredict MCI patients who have a likelihood of developing ADwithin 3 years. The study combined the imaging data withnon-imaging data at the fully connected layer of their proposeddeep-learning-based architecture. We list the recent studies thatcombined di ff erent sources of information in Table 5. Severalother detailed comparison tables can be found in [31][35] andin [8].6 etric th = = = = = = = = ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± Table 4: The hybrid model prediction performance. Training: 60%, Validation: 20%, Test: 20% of ADNI baseline set. (FN = False Negatives, FP = False Positives,NPV = Negative Predictive Value, PPV = Positive Predictive Value, TN = True Negatives, TP = True) (a) (b)
Figure 4: (a) The predictive model processes only scalar data (brain-volume and non-imaging clinical data); average AUC is 0.70. (ROC: Receiver OperatorCharacteristic) (AUC: Area Under the Curve). (b) An illustration of the hybrid prediction system. MPRAGE: Magnetization Prepared Rapid Gradient Echo. MRI:Magnetic Resonance Imaging. ReLU: Rectified Linear Unit. Batch Norm: Batch Normalization.Study Method Data Source Brain Region Objective PerformanceThis Study 3D-CNN Baseline MRI + Whole Brain Predicting AUC: 0 . + Fast Decliners Acc: 63 . + Sens:60 . . et al. [4] Recurrent NN Baseline MRI + Regional MCI-to-AD AUC: 0 . + Hippocampus Conversion Acc: 81%Long. CSF biomarkers + Sens:84%Long. Cognitive performance + Spec:80%Lin et al. [28] 2.5D CNN Baseline MRI + Regional MCI-to-AD AUC: 0 . . + Lasso + NN Sens:84 . . et al. [37] Multi-modal MRI + FDG-PET Whole Brain Stable MCI vs Acc: 82 . et al. [14] Multi-modal MRI + Whole Brain AD-NC-MCI Acc: 94 . + Gender Classification Sens:94%Sens:91%Moradi et al. [8] Random Forest Baseline MRI + Age Whole Brain MCI-to-AD AUC: 0 . + Conversion Acc: 82%(RAVLT + ADAS-cog) Sens: 87%(MMSE + CRD-SB + FAQ) Spec:74%Spasov et al. [35] 3D-CNN MRI + Whole Brain MCI-to-AD AUC: 0 . + Conversion Acc: 86%neuropsychological data + Sens: 87.5%APOe4 genetic data Spec:85%Table 5: ML: Machine Learning, 2D: 2-Dimension, 3D: 3 Dimension, NN: Neural Network, CNN: Convolutional Neural Network, R-CNN: Recurrent ConvolutionalNeural Network, Acc: Accuracy, Sens: Sensitivity, Spec: Specificity, AUC: Area Under Curve. CSF: Cerebrospinal fluid, Long: Longitudinal, RAVLT: Rey’sAuditory Verbal Learning Test, ADAS-cog: Alzheimer’s Disease Assessment Scale cognitive subtest, MMSE: MiniMental State Examination, CDR-SB: ClinicalDementia Rating Sum of Boxes, FAQ: Functional Activities Questionnaire. his study. Unlike prior studies, we investigate the feasibil-ity of predicting the “rate of cognitive decline” in MCI patientsat the first visit by processing only the baseline MRI and rou-tinely collected clinical data. The training data is separatedinto two classes based on MMSE-rate values. We train ourmodel with “slowly deteriorating / stable” or “rapidly deteriorat-ing” classes formed based on MMSE-rate values. Therefore,we do not predict patients that convert to AD. However, someMCI cases deteriorated faster than the others. We investigatethe prediction performance of our multi-modality architectureto predict the rapidly deteriorating cases based on informationavailable at only the baseline visit. The proposed hybrid archi-tecture jointly learns brain patterns and morphology from MRIsequences and additional information from the demographicdata. To our knowledge, this is the first research study thatinvestigates the feasibility of predicting the rate of cognitivedecline by processing routine data collected at the first visit.We follow the same concatenation approach as in [14] whichis proposed for disease detection. Another similar study pro-posed in [28] trained a convolutional neural network with localpatches extracted from the hippocampus and combined the ex-tracted information with FreeSurfer brain data. Our results aresimilar in that combining CNN features with scalar brain datafeatures obtained with FreeSurfer increases the prediction per-formance. However, our study has di ff erences, since our model(i) does not predict the MCI-to-AD conversion probability butinstead predicts the rate of cognition deterioration in MCI pa-tients by utilizing only the first-visit data; (ii) identifies patternswithin the whole brain MRI instead of only the hippocampus,and (iii) uses limited FreeSurfer brain data (6 additional vol-ume elements) compared with the brain data used in [28] (325additional data).Although, we roughly compare our study with MC-to-ADconversion studies, note that there are di ff erences on approach-ing the problem. To our knowledge, this is the first study thatpredicts “slowly deteriorating / stable” or “rapidly deteriorating”classes by processing routinely collected baseline clinical anddemographic data (Baseline MRI, Baseline MMSE, Scalar Vol-umetric data, Age, Gender, Education, Ethnicity, Race). Thetraining data is built based on MMSE-rate values. Therefore,how our study approach predicting progressive MCI is di ff erentthan the previous studies. Also note that our method uses onlybaseline data, not the data that is not routinely asking duringvisits (e.g., APOe4 genetic data) or longitudinal data.
4. Conclusions and Discussion
In this study, we investigate whether a machine learning-based system can predict cognitive decline in MCI patients atthe initial visit by processing routinely collected clinical data.Unlike other studies that focus on predicting MCI-to-AD con-version or AD / CN / MCI classification, we approach the prob-lem as an early prediction of cognitive decline rate in MCI pa-tients . The ability to identify an individual’s cognitive declinerate potentially helps the clinician to develop early preventivetreatment strategies. We observed the performances of 3 models for the predic-tion of cognitive-decline class. Our results confirm that thereis a correlation between the cognitive decline and the clinicaldata obtained at the first visit. The imaging model achieved0 .
67 AUC. By adding brain volume and demographic informa-tion as scalar values to the system, the performance increasedto 0 .
70 AUC. Processing brain volumes (from FreeSurfer braindata) and demographic information as scalar values providesimilar results as the hybrid module performance. Even thoughpatient’s cognitive condition is mostly decided based on non-imaging clinical data (e.g., MMSE score, patient age) at theclinical visit, and MRI scans are generally collected to excludeother brain pathology, our results show that the structural MRIprovides useful information related to the patient‘s cognitivecondition and may further contribute to the clinical evaluationand follow-up of patients with MCI. We have conducted experi-ments on Alzheimer’s Disease Neuroimaging Initiative (ADNI)dataset (c.f., Section 2.1) due to the availability of longitudinalMMSE scores and baseline clinical data. To our knowledge,there is not any available dataset that has a rich source of infor-mation regarding the Alzheimer’s Disease and its progression.However, the system needs to be further investigated and vali-dated on an independent dataset.Our system performance is lower compared to the publishedstudies that investigate MCI-to-AD conversion or AD / CN clas-sification by modeling disease progression by processing lon-gitudinal data obtained at several visits or by processing ad-ditional data that is not routinely obtained during visits (e.g.,APOe4 genetic data). Note that predicting cognitive decline ismore challenging than AD / CN classification due to the subtlenature of pathological changes [28]. Moreover, our system pro-cessed only data that is routinely collected at the first visit, andthus makes predictions based on much less information com-pared to studies that incorporate follow-up data through time-sequence analysis.The clinical and demographic information contains addi-tional data that contributes to the algorithm decision. In thisstudy, we utilized CNN-based architecture as a classifier andcombined the imaging features with non-imaging data at adense layer in a straightforward way. To our knowledge, thereis not any comprehensive study that investigates the best merg-ing methods of di ff erent sources of information in CNN-basedarchitecture, and it is an open research area. Acknowledgments
This research is supported by the Department of Radiologyof The Ohio State University College of Medicine. In addition,the project is partially supported by a donation from the EdwardJ. DeBartolo, Jr. Family (Funding), Master Research Agree-ment with Siemens Healthineers (Technical Support), and Mas-ter Research Agreement with NVIDIA Corporation (TechnicalSupport).Data collection and sharing for this project was funded bythe Alzheimer’s Disease Neuroimaging Initiative (ADNI) (Na-tional Institutes of Health Grant U01 AG024904) and DOD8DNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Ag-ing, the National Institute of Biomedical Imaging and Bioengi-neering, and through generous contributions from the follow-ing: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Dis-covery Foundation; Araclon Biotech; BioClinica, Inc.; Bio-gen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate;Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Com-pany; EuroImmun; F. Ho ff mann-La Roche Ltd and its a ffi References [1] W. R. Markesbery, Neuropathologic alterations in mild cognitive impair-ment: a review, Journal of Alzheimer’s Disease 19 (1) (2010) 221–228.[2] R. C. Petersen, G. E. Smith, S. C. Waring, R. J. Ivnik, E. G. Tangalos,E. Kokmen, Mild cognitive impairment: clinical characterization and out-come, Archives of neurology 56 (3) (1999) 303–308.[3] T. Qarni, A. Salardini, A multifactor approach to mild cognitive impair-ment, in: Seminars in neurology, Vol. 39, Thieme Medical Publishers,2019, pp. 179–187.[4] G. Lee, K. Nho, B. Kang, K.-A. Sohn, D. Kim, Predicting Alzheimer’sdisease progression using multi-modal deep learning approach, ScientificReports 9 (1) (2019) 1952.[5] J. L. Cummings, R. Doody, C. Clark, Disease-modifying therapies forAlzheimer’s disease: challenges to early intervention, Neurology 69 (16)(2007) 1622–1634.[6] H.-I. Suk, D. Shen, Deep learning-based feature representation forAD / MCI classification, in: International Conference on Medical Im-age Computing and Computer-Assisted Intervention, Springer, 2013, pp.583–590.[7] S. F. Eskildsen, P. Coup´e, V. S. Fonov, J. C. Pruessner, D. L. Collins,Alzheimer’s Disease Neuroimaging Initiative, et al., Structural imagingbiomarkers of Alzheimer’s disease: predicting disease progression, Neu-robiology of aging 36 (2015) S23–S31.[8] E. Moradi, A. Pepe, C. Gaser, H. Huttunen, J. Tohka, Alzheimer’s DiseaseNeuroimaging Initiative, et al., Machine learning framework for earlyMRI-based Alzheimer’s conversion prediction in MCI subjects, Neuroim-age 104 (2015) 398–412.[9] M. T´abuas-Pereira, I. Baldeiras, D. Duro, B. Santiago, M. Ribeiro,M. Leit˜ao, C. Oliveira, I. Santana, Prognosis of early-onset vs. late-onset mild cognitive impairment: Comparison of conversion rates and its pre-dictors, Geriatrics 1 (2) (2016) 11.[10] Freesurfer, https://surfer.nmr.mgh.harvard.edu// , [Online; ac-cessed 30 November 2019].[11] I. Arevalo-Rodriguez, N. Smailagic, M. R. i Figuls, A. Ciapponi,E. Sanchez-Perez, A. Giannakou, O. L. Pedraza, X. B. Cosp, S. Cullum,Mini-Mental State Examination (MMSE) for the detection of Alzheimer’sdisease and other dementias in people with mild cognitive impairment(MCI), Cochrane Database of Systematic Reviews (3) (2015).[12] L. E. Harrell, D. Marson, A. Chatterjee, J. A. Parrish, The severe mini-mental state examination: a new neuropsychologic instrument for thebedside assessment of severely impaired patients with Alzheimer disease,Alzheimer Disease & Associated Disorders 14 (3) (2000) 168–175.[13] Alzheimer‘s disease neuroimaging initiative, http://adni.loni.usc.edu// , [Online; accessed 24 November 2018].[14] S. Esmaeilzadeh, D. I. Belivanis, K. M. Pohl, E. Adeli, End-to-endAlzheimer’s disease diagnosis and biomarker identification, in: Inter-national Workshop on Machine Learning in Medical Imaging, Springer,2018, pp. 337–345.[15] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks forbiomedical image segmentation, in: International Conference on Medicalimage computing and computer-assisted intervention, Springer, 2015, pp.234–241.[16] Deepbrain - skull strip algorithm, https://github.com/iitzco/deepbrain// , [Online; accessed 10 October 2019].[17] L. G. Ny´ul, J. K. Udupa, X. Zhang, New variants of a method of mri scalestandardization, IEEE transactions on medical imaging 19 (2) (2000)143–150.[18] S. Io ff e, C. Szegedy, Batch normalization: Accelerating deep net-work training by reducing internal covariate shift, arXiv preprintarXiv:1502.03167 (2015).[19] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015)436.[20] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov,Dropout: a simple way to prevent neural networks from overfitting, TheJournal of Machine Learning Research 15 (1) (2014) 1929–1958.[21] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoo-rian, J. A. Van Der Laak, B. Van Ginneken, C. I. S´anchez, A surveyon deep learning in medical image analysis, Medical image analysis 42(2017) 60–88.[22] M. R. Arbabshirani, S. Plis, J. Sui, V. D. Calhoun, Single subject predic-tion of brain disorders in neuroimaging: promises and pitfalls, Neuroim-age 145 (2017) 137–165.[23] A. Krogh, J. A. Hertz, A simple weight decay can improve generalization,in: Advances in neural information processing systems, 1992, pp. 950–957.[24] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXivpreprint arXiv:1412.6980 (2014).[25] I. Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT press, 2016.[26] L. A. van de Pol, A. Hensel, W. M. van der Flier, P. J. Visser, Y. A.Pijnenburg, F. Barkhof, H. J. Gertz, P. Scheltens, Hippocampal atrophyon MRI in frontotemporal lobar degeneration and Alzheimer’s disease,Journal of Neurology, Neurosurgery & Psychiatry 77 (4) (2006) 439–442.[27] G. Karas, P. Scheltens, S. Rombouts, P. Visser, R. Van Schijndel, N. Fox,F. Barkhof, Global and local gray matter loss in mild cognitive impair-ment and Alzheimer’s disease, Neuroimage 23 (2) (2004) 708–716.[28] W. Lin, T. Tong, Q. Gao, D. Guo, X. Du, Y. Yang, G. Guo, M. Xiao,M. Du, X. Qu, et al., Convolutional neural networks-based MRI imageanalysis for the Alzheimer’s disease prediction from mild cognitive im-pairment, Frontiers in neuroscience 12 (2018).[29] S. I. Dimitriadis, D. Liparas, M. N. Tsolaki, A. D. N. Initiative, et al.,Random forest feature selection, fusion and ensemble strategy: Combin-ing multiple morphological mri measures to discriminate among healhyelderly, mci, cmci and alzheimer’s disease patients: From the alzheimer’sdisease neuroimaging initiative (adni) database, Journal of neurosciencemethods 302 (2018) 14–23.[30] X. Long, L. Chen, C. Jiang, L. Zhang, A. D. N. Initiative, et al., Predic-tion and classification of alzheimer disease based on quantification of mrideformation, PloS one 12 (3) (2017).[31] S. Vieira, W. H. Pinaya, A. Mechelli, Using deep learning to investi-gate the neuroimaging correlates of psychiatric and neurological disor- ers: Methods and applications, Neuroscience & Biobehavioral Reviews74 (2017) 58–75.[32] G. B. Frisoni, N. C. Fox, C. R. Jack, P. Scheltens, P. M. Thompson, Theclinical use of structural mri in alzheimer disease, Nature Reviews Neu-rology 6 (2) (2010) 67–77.[33] L. K. McEvoy, D. Holland, D. J. Hagler Jr, C. Fennema-Notestine, J. B.Brewer, A. M. Dale, Mild cognitive impairment: baseline and longitudi-nal structural mr imaging measures improve predictive prognosis, Radi-ology 259 (3) (2011) 834–843.[34] S. Rathore, M. Habes, M. A. Iftikhar, A. Shacklett, C. Davatzikos, A re-view on neuroimaging-based classification studies and associated featureextraction methods for alzheimer’s disease and its prodromal stages, Neu-roImage 155 (2017) 530–548.[35] S. Spasov, L. Passamonti, A. Duggento, P. Li`o, N. Toschi, A. D. N. Initia-tive, et al., A parameter-e ffi cient deep learning approach to predict conver-sion from mild cognitive impairment to alzheimer’s disease, Neuroimage189 (2019) 276–287.[36] S. Qiu, G. H. Chang, M. Panagia, D. M. Gopal, R. Au, V. B. Kolacha-lama, Fusion of deep learning models of mri scans, mini–mental stateexamination, and logical memory test enhances diagnosis of mild cog-nitive impairment, Alzheimer’s & Dementia: Diagnosis, Assessment &Disease Monitoring 10 (2018) 737–749.[37] D. Lu, K. Popuri, G. W. Ding, R. Balachandar, M. F. Beg, Multimodal andmultiscale deep neural networks for the early diagnosis of alzheimer’sdisease using structural mr and fdg-pet images, Scientific reports 8 (1)(2018) 1–13.cient deep learning approach to predict conver-sion from mild cognitive impairment to alzheimer’s disease, Neuroimage189 (2019) 276–287.[36] S. Qiu, G. H. Chang, M. Panagia, D. M. Gopal, R. Au, V. B. Kolacha-lama, Fusion of deep learning models of mri scans, mini–mental stateexamination, and logical memory test enhances diagnosis of mild cog-nitive impairment, Alzheimer’s & Dementia: Diagnosis, Assessment &Disease Monitoring 10 (2018) 737–749.[37] D. Lu, K. Popuri, G. W. Ding, R. Balachandar, M. F. Beg, Multimodal andmultiscale deep neural networks for the early diagnosis of alzheimer’sdisease using structural mr and fdg-pet images, Scientific reports 8 (1)(2018) 1–13.