Functional connectome fingerprinting: Identifying individuals and predicting cognitive function via deep learning
Biao Cai, Gemeng Zhang, Aiying Zhang, Li Xiao, Wenxing Hu, Julia M. Stephen, Tony W. Wilson, Vince D. Calhoun, Yu-Ping Wang
11 Functional connectome fingerprinting: Identifyingindividuals and predicting cognitive function viadeep learning
Biao Cai a , Gemeng Zhang a , Aiying Zhang a , Li Xiao a , Wenxing Hu a , Julia M. Stephen b , Tony W. Wilson c Vince D. Calhoun d and Yu-Ping Wang aa Biomedical Engineering Department, Tulane University, New Orleans, Louisiana, USA b The Mind Research Network, Albuquerque, New Mexico, USA c Department of Neurological Sciences, University of Nebraska Medical Center (UNMC), Omaha, NE, USA d Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) (Georgia StateUniversity, Georgia Institute of Technology, Emory University), Atlanta, GA 30030
Abstract
The dynamic characteristics of functional network con-nectivity have been widely acknowledged and studied. Bothshared and unique information has been show to be presentin the connectomes. However, very little has been knownabout whether and how this common pattern can predict theindividual variability of the brain, i.e. ”brain fingerprinting”,which attempts to reliably identify a particular individualfrom a pool of subjects. In this paper, we propose toenhance the individual uniqueness based on an autoencodernetwork. More specifically, we rely on the hypothesis thatthe common neural activities shared across individuals maylessen the individual discrimination. By reducing contri-butions from shared activities, inter-subject variability canbe enhanced. Results show that that refined connectomesutilizing an autoencoder with sparse dictionary learning cansuccessfully distinguish one individual from the remainingparticipants with reasonably high accuracy (up to . for the rest-rest pair). Furthermore, high-level cognitivebehavior (e.g., fluid intelligence, executive function, andlanguage comprehension) can also be better predicted usingthe refined functional connectivity profiles. As expected,high-order association cortices contributed more to bothindividual discrimination and behavior prediction. Theproposed approach provides a promising way to enhanceand leverage the individualized characteristics of brainnetworks. Index Terms —Functional connectivity, common connectivitypatterns, autoencoder network, refined connectomes, individualidentification, high-level cognition prediction.
I. I
NTRODUCTION
Functional magnetic resonance imaging (fMRI) allows fornon-invasive interrogation of brain functions based on theblood-oxygenation-level-dependent-signal (BOLD signal) [1]–[3]. Intriguingly, a functional connectome based on functionalconnectivity (FC) extracted from the fMRI time series providesa promising tool to investigate individual differences in humancognitive and behavioral performance from a network per-spective. Recently, numerous studies have reported individualvariability in functional connectivity. For instance, Airan et al.provided an in-depth study to evaluate the degree of influencethat standard fMRI acquisition and analysis schemes have on individual subject variability [4]. This variability is assumed tobe associated with both genetic and environmental factors, andthereby neural development. Meanwhile, such variability alsomay partially affect individual cognition and behavior [5], [6].More importantly, the functional connectome of the humanbrain constitutes individualized patterns that enables us toidentify one from a pool of individuals just like a fingerprint[7]. Specifically, Finn et al. demonstrated that such connectivityprofiles could be used to distinguish individuals among adultparticipants across rest/task modalities. In their work, theyshowed that the discriminative subnetworks of individualscontributed most to the prediction of fluid intelligence score [7].Kaufmann et al. reported that the functional profile developedinto a stable, individual wiring pattern during adolescence,and they demonstrated that reduced mental health induceda delay and an overall reduction of such wiring [8]. Thestudies mentioned above used a standard procedure to extractthe functional profiles, but overlooked the influence fromthe group-wise contribution. Inspired by this, we refined themeasures of capturing individual connectomes by increasinginter-subject variability across FC values in a population witha goal of improving the power of predicting individualitywith FC fingerprints [9]. Recently, the limitation of staticconnectivity has been widely realized, and the concept ofdynamic connectivity has emerged to emphasize the time-varying characteristics of the FC [10]–[17]. Incorporatingthe information from the time-varying FC, Liu et al. studiedwhether and how the dynamic properties of the chronnectomeacted as a fingerprint of the brain to identify individuals[18]. Mounting evidence indicates that during the dynamicFC analysis, a state (termed stable state) that resembles thestatic FC reoccurs more frequently than others [11], [12]. Thesestable states are similar across subjects and share the basicconfiguration with all dynamic patterns. Hence, we hypothesizethat the common neural activities, which do not have theindividual-specific characteristics in the connectomes, can berepresented by the static FC. They may impede the revealing ofindividualized characterization. To our knowledge, this factorhas not yet been considered when performing the individualized a r X i v : . [ q - b i o . N C ] J un pattern analysis. Thus, our motivation is that, by reducing thecontribution from the common neural activity, we increasesensitivity to individual variability in FC.To this end, we employ a dimensionality reduction techniqueto extract the basic configuration. More specifically, we projectfMRI data onto its underlying subspace with a commonstructure. A simple and commonly used method is principalcomponent analysis (PCA), which finds the direction of thegreatest variance in the dataset and represents each datapoint by its coordinates along each of these directions [19].However, PCA cannot extract nonlinear structures modeledby higher than second-order statistics. Various methods havebeen proposed for nonlinear dimension analysis, such as theauto-associative networks, generalized PCA and kernel PCA[20]–[23]. The more recently proposed autoencoders (AEs)[24] belong to a family of nonlinear dimensionality reductionmethods using neural networks. Through multi-layer neuralnetworks, the autoencoder and its extensions demonstratepowerful performance to learn key features from data [25]–[27]. For real-world datasets, successful applications include[24], [28], [29]. Thus, in this work, we employ autoencoder toestimate the common neural activity from rest/task fMRI data.The remainder of this paper is organized as follows. InSection 2, we first describe the dataset used in this work. Then,we introduce our proposed framework for estimating FCs stepby step, followed by a series of experiments conducted. Inparticular, we analyze whether refined connectomes extractedby our proposed method can better distinguish each individualfrom a pool of participants, and predict high-level cognitivebehaviors. The corresponding results are illustrated in Section 3.Some discussions and concluding remarks are given in Section4 and 5. II. M ATERIALS AND METHODS
A. Data acquisition
We used the publicly available S1200 Data Release of theHuman Connectome Project (HCP) [30]. The S1200 releasecontains behavioral and 3T MR imaging data from 1206healthy young adult participants collected from August 2012 toOctober 2015. 889 subjects have complete data for all four 3TMRI modalities in the HCP protocol: structural images (T1wand T2w), resting-state fMRI (rsfMRI), task fMRI (tfMRI),and high angular resolution diffusion imaging (dMRI). Awritten informed consent was obtained for each subject. AllHCP subjects were scanned on a customized Siemens 3T”Connectome Skyra” housed at Washington University in St.Louis, using a standard 32-channel Siemens receiver head coiland a ”body” transmission coil designed by Siemens specificallyfor the smaller space available using the special gradients of theWU-Minn and MGH-UCLA Connectome scanners. To addresshead motion, dynamic head position information was acquiredusing an optical motion tracking camera system (Moire PhaseTracker Kineticor). fMRI was acquired using a whole-brainmultiband gradient-echo (GE) echoplanar (EPI) sequence withthe following parameters: TR/TE = 720/33.1ms, flip angle =90 ◦ , FOV = 208 × ×
90 (RO × PE),multiband factor = 8, echo spacing = 0.58ms, slice thickness = 2mm. The resulting normal voxel size was 2.0 × × B. Data preprocessing
Our study used the fMRI dataset from HCP with the minimalpreprocessing pipeline, which included gradient distortioncorrection, head motion correction, image distortion correction,spatial normalization to standard Montreal Neurological Insti-tute (MNI) and intensity normalization [32]. Further, we appliedthe standard preprocessing procedures to reduce biophysicaland other noise sources in the minimally processed fMRI data.These procedures contained the removal of linear componentsrelated to the 12 motion parameters (original motion parametersand their first-order derivatives), removing linear trend andperforming band-pass filtering (0.01-0.1Hz). Notably, Finn etal. has reported that the smoothing level had essentially noeffect on identification accuracy [7]. Thus, we investigated theanalysis based on the data without applying spatial smoothing.To facilitate the understanding of behaviors associated withdifferent brain regions, we applied a 268-node functional atlasprovided by Finn et al. [7], which was defined using a group-wise spectral clustering algorithm [33]. More specifically, weextracted the time series of each node by averaging the timecourses of all voxels that belonged to that node. Then, weassigned these nodes into 8 functional networks, includingmedial frontal (Med F), frontoparietal (FP), default mode(DMN), subcortical-cerebellum (Sub-Cer), motor (Mt), visualI (Vis I), visual II (Vis II) and visual association (Vis Assn)regions. Axial, sagittal and coronal views of these functionalnetworks were displayed in Fig.1.
C. Autoencoder network construction
An autoencoder was used to extract common neural activityfrom the BOLD time series. For each participant of a modality,BOLD time courses with p ROIs and n t time points ( p , n t ∈ N )are available. These signals are marked as the original timeseries and set as inputs to the AE network. The AE networkcan be defined as: Fig. 1. Axial, sagittal and coronal views (from left to right) of 8 functionalnetworks provided by Finn et al. 1. Med F: Medial Frontal; 2. FP: Fron-toparietal; 3. DMN: Default Mode; 4. Sur-Cer: Subcortical-cerebellum; 5. Mt:Motor; 6. Vis I: Visual I; 7. Vis II: Visual II; 8. Vis Assn: Visual Association. y = f θ ( x ) = s ( W x + b ) , (1) z = g θ (cid:48) ( y ) = s ( W (cid:48) y + b (cid:48) ) , (2)where the deterministic mapping f θ is called the encoder,which transforms an input vector x ∈ R d into the hiddenrepresentation y ∈ R d . Its parameter set is θ = { W, b } , where W is a d (cid:48) × d weight matrix with d (cid:48) ≤ d and b is an offsetvector of dimensionality d (cid:48) .The resulting hidden representation y is then mapped back toa reconstructed d -dimensional vector z in the input space,i.e., z = g θ (cid:48) ( y ) . This mapping g θ (cid:48) is called the decoder. s and s are activation functions for encoding and decoding layersrespectively. Here, we used rectified linear units (ReLUs) in allencoder/decoder pairs, except for s of the first pair (becausethe time series have both the positive and negative values)[34]. Training was performed by minimizing the least-square (cid:107) x − z (cid:107) . After training of one layer, we used its output f θ ( x ) as the input to train the next layer. In order to avoid datasetspecific tuning as much as possible, we set the parametersin the DNNs (deep neural networks) training as the default(following the recommendation in [35]). More importantly,inspired by the work provided by van der Maaten [35], weset layers’ dimensions in AE as d -500-500-2000-10-2000-500-500- d for all fMRI modalities, where d is the number of timepoints (i.e., d = n t ) varying with respect to different fMRImodalities. All layers are fully connected. Then, we calculatedthe residual time courses, which are the differences betweenthe original time series and reconstructed ones generated bythe AE network. Next, the residual time series were set asthe inputs to the subsequent sparse dictionary learning (SDL)model. An illustration of the workflow is displayed in Fig.2(a). D. Increased individual identifiability employing the sparsedictionary learning model
In our previous study, we indicated that individual connectiv-ity analysis benefits from group-wise inferences and the refinedconnectomes are indeed desirable for brain mapping [9]. Thus,to further improve the inter-subject variability across FCs, we
Fig. 2. An illustration of the workflow to refine the brain connectivity. (a)Extraction and reduction of the effect of common neural activity using theautoencoder network. The dimensions of the AE network are set to be d -500-500-2000-10-2000-500-500- d for each participant. The difference between theoriginal and reconstructed time series (residual time series) is set as the inputto a sparse dictionary learning model (SDL). (b) Decomposition of the FC intoboth the group-wise and subject-wise patterns using the SDL model. Note theassumption that the subject-specific FC may carry most of the identificationinformation for a participant, which is tested and implemented here. implemented the same pipeline to reduce group-wise contri-bution. Assume that we have n ∈ N subjects. For the residualtime courses ( p ROIs and n t time points, p, n t ∈ N ), we firstcalculate a correlation matrix, C i ∈ R p × p ( i ∈ , , · · · , n ) , foreach subject. C i ( b , b ) is the Pearson correlation between ROIs b and b across the entire residual time series. In considerationof the symmetry of the correlation matrix, we discard theupper triangular part of C i . This leads to the edge weightvector e i = vec ( C i ) ∈ R p ( p − / for each subject. Next, weconcatenate edge weight vectors from all subjects to formall the subject matrices Y = [ e , e , . . . , e n ] with the size of m × n , where m = p ( p − / , n = 1 , , . . . , N . Identifyingthe sparse representation of the functional connectivity acrosssubjects ( Y ) can be modeled as an SDL problem. By solvingthe following formulation, we can approximate the given data Y : min D,X (cid:107) Y − DX (cid:107) F subject to (cid:107) x i (cid:107) ≤ L, i = 1 , , . . . , n, (3)where L is a non-negative model parameter to controlthe sparsity level of representations. D ∈ R m × K denotesthe dictionaries, and K is the size of dictionaries. X =[ x , x , . . . , x n ] ∈ R K × n is the representation matrix and (cid:107)·(cid:107) , (cid:107) · (cid:107) F denote the l and Frobenius norms, respectively. Moredetails about the SDL model can be found in our previouswork [9].Since we want to improve the inter-subject variability, group-wise contributions can be excluded from each correlation matrix C i to obtain a new refined functional connectome (cid:98) C i . Therefined functional pattern is defined as follows: (cid:98) C i = C i − mat ( Dx i ) (4)where mat ( Dx i ) ∈ R p × p is the correlation matrix recon-structed from the lower triangular information Dx i . Theframework for the SDL model is illustrated in Fig.2(b).Note that during analysis, the SDL model is performed oneach subject of ROI network from different fMRI modalitiesindividually. E. Individual identifiability analysis
To explore the use of functional connectomes as fingerprintsusing our pipeline, we investigated individual identificationability proposed by Finn et al. [7]. Identification is performedacross pairs of scans consisting of one target and one sessionfrom the HCP database, with the requirement that the targetand database sessions are from different days to avoid theinterference as much as possible. For the target session witha given subject (e.g., resting-state1, R1), we would like toidentify that the connectivity pattern from the session in thedatabase (e.g., language, Lg) belongs to the same subject.More specifically, for each participant, we first compare thecorrelation matrix of this subject from session 1 to each of thematrices of all the participants from session 2 ( s → s ). Foreach comparison, the similarity scores between the connectivitypatterns from session 1 and session 2 are simply estimatedusing the Pearson correlation coefficient. Then, we assign thisparticipant the same label with the subject in session 2 whohas the maximal similarity score with this participant. If theFCs with same label are indeed from the same participant, theidentification accuracy is considered to be . Otherwise, itis designated as . By calculating the proportion of subjectswith the correct identification, we determine the identificationaccuracy of all the participants. Finally, the session 1 andsession 2 are reversed, and the procedures discussed above arerepeated ( s → s ). Because we have three fMRI modalities(R1, Wm, Mt) for one day and three modalities of fMRI (R2,Lg, Em) for another day, this results in 9 possible combinationsfor s → s (likewise, 9 possible combinations for s → s ).After obtaining the identification accuracy for all the par-ticipants, we performed 10,000 nonparametric permutationtests (two-sided) to assess whether the observed accuracieswere significantly above chance. For each permutation, werandomize the identities of the subjects in both sessions,perform the identification procedures and record the accuracies.A significant level of p-value = 0.05 is used as the thresholdfor the , permutation tests.We then investigated the identification accuracy on the basisof each specific functional network to figure out which brainnetwork contributes more to the individual discriminability.These functional networks are defined in the section of datapreprocessing. During this process, a single network or acombination of networks are used to estimate the individualidentification. Note that, if we denote the set of nodes belongingto network j as V j = v jk , k = 1 , , . . . , K j , where K j is thetotal number of nodes in network j , only connections withinthe selected network are included. F. Fundamental neural activities contribution to individualidentification
To check the hypothesis that common neural activities mayweaken the individual variability, we compare the performanceof identifiability with and without the AE network processing.For this purpose, we exclude the SDL model in this experimentto avoid its influence. As a first pass, we calculate correlationsbetween connectivity matrices of all participants across 9possible combinations ( s → s ) under these two scenarios(with and without the AE network). For each correlation matrix,the row and column are symmetric. Thus, diagonal elementsare similarity scores from the matched subjects, while off-diagonal elements are the ones from the unmatched participants.By observing the difference between the mean values ofdiagonal and off-diagonal factors, the individual identifiabilitycan be evaluated. The larger the difference, the stronger thediscriminative power.We then estimate the identification accuracy for these twoscenarios, respectively. The procedures have already beenprovided in the section of individual identifiability analysis. Ifthe identification rates generated by the scheme with the AEnetwork are much higher than those without the AE network,we assume that using AE can therefore increase the subject-specific identifiability by reducing the effect of the commonneural activities. Finally, we reverse session 1 and session 2and repeat the above proceedings ( s → s ).Afterwards, we investigate the regions in the FC, to whichthe signals removed by the AE network belong. To filter out theinfluence induced by activities in the task runs, we restrict theanalysis to resting-state fMRI (R1 and R2). Group differencesbetween the analyses with and without AE are considered here.First, we calculate the correlation matrix for each subject. Next,we transform correlation matrices into the edge-weight vectors( e i ), and concatenate them into the data Y as mentioned inthe previous section. Finally, a two-sample t-test is applied tothe data Y with a significant level of q = 0 . to examine thegroup differences. G. Edgewise contribution to identification
To investigate which connections of the FC contribute moreto subject identification, we estimated the modified differentialpower (DP) provided by Liu et al. [18]. In this part, wealso restrict the study to R1 and R2 sessions. The modifieddifferential power is defined as follows: DP ( i, j ) = 1 − (cid:88) l P l ( i, j ) ,P l ( i, j ) = | φ lk ( i, j ) > φ ll ( i, j ) | + | φ kl ( i, j ) > φ ll ( i, j ) | N − , (5)where P l ( i, j ) is an empirical probability to quantify thedifferential power of an edge for the purpose of subjectidentification; l and k ( l (cid:54) = k ) represent the labels of twodifferent participants; i and j ( i (cid:54) = j ) denotes two differentnodes within the functional connectivity; n is the total numberof participants in the analysis ( n = 862 ). | φ lk ( i, j ) > φ ll ( i, j ) | indicates the probability that | φ lk | between two different subjects is higher than | φ ll | of the same participant. Given twosets of connectivity matrices [ X R l ( i, j )] , [ X R k ( i, j )] obtainedfrom the R1 and R2 sessions after z-score normalization,the corresponding edge-wise product vector φ lk ( i, j ) can becalculated as follows: φ lk ( i, j ) = X R l ( i, j ) ∗ X R k ( i, j ) , l, k = 1 , , . . . , N (6) | φ ll | can also be obtained in the same way. DP reflectseach edge’s ability to distinguish an individual subject. Fora given functional connectivity, a higher DP value means agreater contribution to individual identification. Furthermore,to investigate the network-dependent contribution to subject-specific identification, we also count the number of the highestDP values (top 1%) within or between functional networks. Inthis manner, we examine whether specific brain networks playa significant role in discriminating individuals. H. Prediction analysis for individual cognitive behavior
To determine whether our refined FC applying the AEnetwork could benefit individual cognitive prediction, wedepicted it from two aspects: regression and classificationanalysis for continuous and discrete targets, respectively. Here,we select items of high-level cognition from the HCP proto-col, including fluid intelligence (Penn Progressive Matrices,HCP: PMAT24 A CR, Mean ± SD: 17.04 ± ± SD: 102.54 ± ± SD: 102.05 ± ± SD:109.44 ±
1) Regression analysis:
To determine whether the refined FCprofiles can better predict the individual high-cognitive behaviorrelative to those without employing the AE network, we useleave-one-subject-out cross-validation (LOOCV) strategy toestimate the prediction accuracy [9]. For instance, to assessthe ability of refined FCs to describe fluid intelligence, ineach LOOCV fold, one participant is assigned as the testsample, and the remaining n − subjects are considered asthe training samples. First, we concatenate all the connectionswithin the FC profiles (i.e., 35778 connections) to generatea feature vector for each subject. Second, we investigate afeature selection step, which calculates the correlation betweeneach connection of FCs (Pearson correlation between twoROIs) and fluid intelligence scores on the training set. If thecorrelation is significant (p-value <
2) Classification analysis:
To further evaluate predictivepower of refined FCs, we performed classification analysesfor two subsets based on these high-cognitive scores. Moreprecisely, we first extract participants who are within eitherupper or lower δ -th percentile of the distribution of behavioralscores. That is, we retain subjects who have the highest orlowest δ % high-cognitive scores. Cases of δ ∈ { , , } are considered here. Next, the feature selection step discussedin the regression analysis section is applied (the significantlevel p-value = δ ,a relatively low number of subjects are left for each case (172subjects for δ = 10 , 344 subjects for δ = 20 and 516 subjectsfor δ = 30 ). Hence, we repeat the experiment times. Foreach run, we divide participants into a training set (75%) anda testing set (25%). A SVM function built in Matlab with aGaussian kernel is employed, and a grid search is applied tooptimize parameters within the SVM model (e.g., the radius ofthe Gaussian kernel, the weight of the soft margin cost function).To validate that reducing the common neural activities canbenefit individual discrimination, the results generated by theframework without the AE network are also provided.III. R ESULTS
A. Refined FC based individual identification
As a first pass, we evaluated the identification accuracyapplying the whole brain connectivity matrices (268 nodes,without prior network definitions) to validate that the refinedFCs can highlight the subject-specific variability. Identificationrates are described in Fig.3(a). Even with a large numberof participants (862 subjects), refined FCs worked well forindividual identification. The success rates were . and . based on a database-target rest1-rest2 and the reverserest2-rest1, respectively. Meanwhile, the identification ratesranged from . to . for rest-task pairs and . to . for task-task combinations. In relative to raw FCs ( subjects, accuracy between the two rest sessions, ± for other pairs in Finn et al.’s work [36]), refined FC profileslargely improved the performance of individual discrimination.Given that identification trials were not independent of eachother, we performed 10,000 nonparametric permutation tests(two-sided) to assess the significance level of these results.Across 10,000 iterations, the p-value for each pair of sessions isbelow 0.0001. It indicates that the success rates of identificationare significantly above chance.Next, we examined the identification accuracy based on eachfunctional network to explore which brain network contributesmore to the individual variability. These networks are defined by Finn et al. [7] and depicted in Fig.1. The medial frontalnetwork (network 1) and the frontoparietal network (network 2)achieve the highest success rates in individual discrimination,which comprises the higher-order association cortices in thefrontal, parietal and temporal lobes. In comparison with themedial frontal network, the frontoparietal network performsmuch better especially for the rest-task and task-task pairs.Furthermore, we also checked whether the combination ofnetworks 1 and 2 can provide better performance than eachindividual. As shown in Fig.3(b), in each scenario, the identifi-cation accuracies using the combination of networks 1 and 2are higher than implementing network 1 or 2 independently,and are pretty close to those generated through applying thewhole-brain nodes. For other networks, subcortical-cerebellumnetwork (network 4) and motor network also contribute tosubject-specific variability. B. Fundamental neural activities contribution to identification
To investigate the contribution of common neural activitiesto individual identifiability, we estimated the correlationsbetween connectivity matrices of all subjects across 9 possiblecombinations for both time courses with and without the AEnetwork processing. For each correlation matrix, the row andcolumn are symmetric by subject. Thus, diagonal elements arecorrelation coefficients from the matched subjects, while off-diagonal elements are those from the unmatched participants.The results for all 9 possible pairs are displayed in Fig.4(a).In comparison with raw cross-subject correlation coefficients,scores generated after the AE network significantly becomeweak for both diagonal and off-diagonal factors. However,applying the AE network improves the difference betweendiagonal and off-diagonal elements in the correlation matrix.It indicates that reducing the contribution from commonneural activities mentioned above indeed helps individualdiscrimination.Next, we repeated the identification experiments applyingconnectivity matrices from the two scenarios discussed above tofurther validate our hypothesis. By checking the identificationresults in Fig.4(b), we observe that identification accuracies forall 9 pairs have increased through reducing the common neuralcontribution. More specifically, for the pairs of rest-rest, theidentification rates were improved around . As to rest-taskcombinations, when using connectivity matrices of resting-state fMRI in the database, the ability of discrimination hassignificantly enhanced (R1-Lg : . , R1-Em : . , R2-Wm : . , R2-Mt : . ). By contrast, the rates applyingmatrices achieved from task-based fMRI in the database gainaround . Meanwhile, with the AE network preprocessing,the identification accuracies increase for every condition oftask-task combination, ranging from . (Em-Mt) to . (Lg-Wm). Thus, weakening the signals from common neuralactivities can significantly enhance inter-subject differences,and the pairs of rest-rest possess the strongest identificationpower.To explore which connections were removed by the AEnetworks, we tested the group average functional connectomesbefore and after applying the AE networks and examined the difference between them using a two-sample t-test. FromFig.4(c), we obtain that the strength of links in the connec-tomes reduces overall. However, the significant difference islargely related to the frontoparietal and subcortical-cerebellumnetworks. C. Evaluation edgewise contributions to identification
To determine which connections contribute more to subject-specific identification, we calculated the modified differentialpower (DP). The modified DP reflects each edge’s abilityto distinguish an individual from a pool of participants. Aconnection with high DP tends to have a similar value withinan individual across modalities, but possess a different degreeacross individuals regardless of modalities.By restricting the analysis to resting-state fMRI (rest1 andrest2), we estimated the modified DP for all edges in thebrain. We determined which connections were in the 99.9percentile across all of the links (Fig.5). We observe that themajority of edges in the 99.9 percentile of the edges are inthe frontal, parietal, and temporal lobes. Meanwhile, most ofthe nodes with high DP values involve in the frontoparietaland medial frontal networks. Some of them belong to thedefault mode network. By checking the results in Fig.5(b), weobtain that for the connections possessing high DP values inthe connectivity, . of them are used to link the medialfrontal and frontoparietal networks. In addition, areconnections linking these networks to others ( of them linkthe frontoparietal network to other networks). It indicates thatconnections related to high-order association cortices are themost discriminative of individuals. On the other hand, medialfrontal and frontoparietal networks play a significant role inindividual identification. D. Connectivity profiles predict high-level cognitive behaviors1) Regression analysis:
To test whether refined FC profilesbenefit the behavior prediction, we explored the predictionabilities across different high-level cognitive scores of connec-tivity profiles under the scenarios with and without the AEnetwork processing. Note that for these two conditions, theSDL model was included during the analysis. As demonstratedin the scatter plots in Fig.6, the predicted scores by theconnectivity profiles with AE network have higher correlationcoefficients with the observed scores relative to those withoutthe AE network. Besides, the range of predicted scores bythe refined FCs is much narrower across all the cognitivescores (especially for the language/vocabulary comprehension).To validate the prediction power of our proposed frameworkapplying the AE network, we performed 100 nonparametricpermutations for each score. The results illustrate that theprediction of each high-level cognitive behavior (correlationbetween observed and predicted scores) is above chance(fluid intelligence: p-value < < < < Fig. 3. Identification accuracy across session pairs and networks. (a) Identification rates from the whole brain are highlighted in color-coded matrices tocompare the accuracies across rest-rest, rest-task and task-task sessions, respectively. R1, Rest1; Wm, Working Memory task; Mt, motor task; R2, Rest2; Lg,language task; Em, emotion task. (b) Identification results based on all 9 sessions in the database and target combinations. Each row shares the same databasesession and each column shares the same target session. The color of the bar (grey or white) indicates which fMRI modality was used as the database, and theother one was served as the target. Graphs display the identification rate based on each network as well as the combination network 1 and 2 and the wholebrain(all).
Furthermore, by observing the selected features from refinedFC profiles, we find that different brain regions exhibit distinctcontributions to various high-level cognitive parameters. Morespecifically, frontal and parietal lobes contribute most to fluidintelligence prediction according to this study. In detail, nodeslocated in the frontal and parietal regions provide positiveconnections highly related to fluid intelligence. Moreover, someof the positive links are relevant to the right cerebellum, whilea large portion of negative links connects with the insularegion. When using the refined FC profiles to predict theperformance of cognitive flexibility, in comparison with positiveedges, negative connections play a more critical role. Most ofthese negative edges are linked with frontal and parietal lobes.Notably, the region of the motor strip is involved in both thepositive and negative connections regarding cognitive flexibility.Similarly, negative connections predominantly contribute toinhibition function as well. Among these negative links, themajority of them interact with the parietal lobe. Also, thenegative edges from the motor strip and subcortical regions haveequivalent influence on the prediction of inhibition function.In the prediction of language comprehension, through both thepositive and negative edges in the connectivity, the frontal andtemporal lobes closely connect with language comprehension.As a consequence, the data-driven framework applying therefined FC profiles can adequately enable us to find out whichbrain regions closely interact with high-level cognitive behavior.
2) Classification analysis:
To analyze the prediction powerof refined FCs further, we investigated whether we were ableto discriminate two subsets based on high-level cognitivebehavior. In the experiments, the results with and without theAE network processing were compared. In general, regardlessof the percentile values δ , applying refined connectomes leadsto satisfactory classification rates for all the cognitive scores ( > ). However, reducing the common neural activities withthe AE network has a different influence on the classificationof various cognitive parameters. More specifically, relative toonly implementing the SDL model, the proposed refined FCshave better performance in predicting fluid intelligence. Whilethese two frameworks offer similar classification rates with δ = 10 (mean values, with the AE network: . , without theAE network: . ), refined FC profiles possess more stableclassification accuracies as the difference of fluid intelligencebetween the two subsets decreases. Interestingly, functionalconnectomes generated by implementing the AE networkshowed an improvement in classifying cognitive flexibilityrelated subsets. Regarding those without the AE network, theaccuracy increases from . to . for δ = 10 , . to . for δ = 20 and . to . for δ = 30 . As tothe cognitive parameters of inhibition function and languagecomprehension, while the slight differences exist, whetheradopting the AE network or not will not affect the performanceof classification. IV. D ISCUSSION
Recently, the study and use of dynamic functional networkanalysis has drawn more attention. Some basic configurationsin brain connectivity appear across all time-varying states, andmay not contribute to subject-specific discrimination. In ourprevious work, we pointed out that refining the measures ofindividual connectomes can help improve the fingerprintingpower of individuality [9]. In this study, we applied theautoencoder network to extract the signals of common neuralactivities, which may hinder the identification of individualdifference. Therefore, we removed them from the raw timecourses when constructing connectivity network between ROIs.We then applied dictionary learning on all the data. Based on
Fig. 4. Evaluation of the influence of common neural activities on individual identification. (a) Analysis of identifiability matrices based on all 9 pairs of thecombination of a session in the database and the target ( s → s ). For the top line of each sub-figure, from left-right: identifiability matrix (i.e. Pearsoncorrelation coefficient between functional connectivity across subjects and modalities) of the original data; identifiability matrix of the AE residual data. Therow and column subject order of identifiability is symmetric. Hence, diagonal elements are correlation scores from the matched subjects, while off-diagonalcoefficients are from the unmatched participants. Mean correlation coefficients for both diagonal (match) and off-diagonal (unmatched) elements are alsocalculated (bottom, err bars indicates ± s.d.). ** means p-value < − for two-tailed t-test. Besides, the difference (mean value) between the diagonal andoff-diagonal elements for both the original and AE residual data are displayed. (b) Comparison of identification accuracies across all 9 pairs of databasesession and target session between using original time series (top) and AE residual data (bottom). Note that only the situation with whole-brain nodes (264nodes) is considered here. (c) Static functional network connectivity for the original data (left) and AE residual signals (middle) estimated by the Pearsoncorrelation. Meanwhile, group differences (right) between them is obtained by applying a two-sample t-test. These variations are visualized by plotting the logof p-value with the sign of t statistics, − sign ( t ) log ( p ) . Note that in this part, the results from the rest1 and rest2 are displayed individually. the pipeline, we generated the new refined FC connectomes[9]. We showed that the refined FC profiles can successfullydistinguish one individual from a pool of the population with ahigh identification accuracy. Moreover, the refined connectomescan significantly predict high-level cognitive behavior, includingfluid intelligence, cognitive flexibility, inhibition function, andlanguage comprehension. Notably, reducing the signals ofcommon neural activities benefited individual identification andcognition prediction, where frontal, parietal and temporal lobescontributed significantly. Collectively, our findings supportedour assumption that common neural activities may impede theindividual identifiability and its removal can help enhance theuniqueness of each individual.When testing the identification ability of refined FCs, wefound that regardless of database-target combinations, theconnectomes applying the proposed approach successfullydistinguished each individual from all the participants. Theaccuracy was from . to . for the rest-rest pairs.As to the rest-task and task-task combinations, the successrates ranged from . to . and from . to . ,respectively. The strong individual identification power of refined FCs from our results suggest that connectomes varyacross participants and are unique for each subject. This is inagreement with previous findings that the connectomes couldbe used as the fingerprint to identify an individual [7], [9].Compared to results obtained by raw FCs, reducing the effectof common neural activities and group factors can significantlyimprove the success rates of identification [36]. For the pairsof rest-rest fMRI, we gained accuracy improvement.Also, the success rates increased by around for othercombinations. The findings validate our assumption that somepatterns caused by common neural activities may impede theindividual variability, and thus reducing their impact helpedcapture unique characteristics of the brain.The contributions of functional networks to individual iden-tification was also examined. Although discrimination basedon the whole connectivity matrix performed best, the medialfrontal and frontoparietal networks achieved high accuracy. Thecombination of these two networks provided better performancethan only one of them alone. These networks are composed ofhigher-order association cortices (fontal, parietal and temporallobes), which have been proven to show the highest inter- Fig. 5. Edgewise contributions to individual identification. (a) Connections that possess the highest DP scores in individual connectivity profiles (top, circleplot). Axial, sagittal and coronal views of these links are also provided (bottom, from left to right). Note that connections with the highest 1% DP values areshown here. In the circle plots (top), the 268 nodes (the inner circle) are organized into a lobe scheme (the outer circle) roughly reflecting brain anatomy fromanterior (top of the circle) to posterior (bottom of the circle) and split into left and right hemisphere. Lines indicate edges or connections. (b) The percentageof connections within and between each pair of networks (8 functional networks defined in Fig.1) using the same data as (a). The color depth of the grid in thematrix indicates the fraction of DP edges for each pair of networks. subject variance [7], [37]. Relative to the medial frontal network,frontoparietal contributed more to the identification, especiallyfor rest-task and task-task pairs. It is consistent with thefunction of the frontoparietal system, which is particularlyactive in tasks requiring a high degree of cognitive control.Even for the rest-rest combination, the frontoparietal networkworked very well for identification. Thus, we believe that thefrontoparietal system plays a significant role in the brain’suniqueness regardless of whether the mind is at rest or not.Also, we detected that the subcortical-cerebellum and motornetwork positively correlated with individual differentiation. Itmatches with the conclusion that there was a gradual increaseof variability in primary regions of the visual and sensorimotorsystems specific to subcortical and cerebellum structures asthe brain developed [38].To further estimate the contribution of common neuralactivities to individual variability, we compared the correlationmatrices with and without AE network, and repeated theidentification experiments for these two cases. While applyingthe AE network weakened both diagonal and off-diagonalfactors of correlation matrices, reducing the signals fromcommon connectivity patterns increased the difference betweendiagonal and off-diagonal elements. Besides, without usingthe SDL model, reducing the common neural contributionincreased the identification rates across all 9 combinations.All these findings indicate that weakening the contribution ofcommon neural activities helps strengthen the subject-specificvariability and enhances the uniqueness of the human brain.Intriguingly, we observed that the time courses identifiedby the AE networks are mainly related to the frontoparietaland subcortical-cerebellum systems, which had a significantinfluence on the identification of individuals. In light of this, theinteraction between these networks with individual predictions will be conducted in future work.By examining the modified differential power of each edgein the connectivity map, we determined that most of theconnections with high DP values were related to the higher-order association (frontal, parietal, and temporal) lobes. Also, . of high DP edges were connected with the medialfrontal and frontoparietal systems, and . of them linkedthese networks to others. These findings further validate thefunction of the medial and frontoparietal networks in individualidentification.When exploring the prediction abilities across differentcognitive parameters with and without AE processing, weobserved that the predicted scores by applying AE processingpossessed higher correlation coefficients with the observedscores relative to those without AE processing. This suggeststhat the reduction of common functional patterns can improvethe power of cognitive behavior prediction. Next, we analyzedthe selected features that contributed more to the cognitionprediction and found that different brain regions had variouseffects on each cognitive measurement. We demonstrated largecontributions of the frontal and parietal lobes to individualfluid intelligence and execution function (e.g., flexibility andinhibition function). These findings are consistent with previousstudies [7], [18], [37]. Specifically, we detected that insula wasclosely associated with fluid intelligence. This agrees with thestatement that fluid intelligence has been correlated with adistributed network comprising regions of frontal, insula andparietal cortex [39]. Furthermore, we found that the motorstrip region was highly related to executive function. Thispoint is supported by the argument that motor and cognitiveprocesses are functionally related and most likely share asimilar evolutionary history. It is well established that multiplebrain regions integrate both motor and cognitive functions [40]. Fig. 6. Connectivity profiles predict cognitive behavior. Scatter plots display prediction results from a leave-one-subject-out cross-validation (LOOCV) analysiscomparing the predicted and the observed high-level cognitive scores. Both the connectomes with and without applying the AE network processing areconsidered. Note that under these two scenarios, the sparse dictionary learning (SDL) model is included. In the scatter plot, each dot represents one subject,and the area between dashed lines reflects confidence interval for the best-fit line, which is used to assess the predictive power of the model. R-values arethe correlation coefficients between the predicted and observed high-level cognitive scores. Furthermore, for each cognitive scores, edges retained from thefeature selection step (p-value < Fig. 7. Classification results between low and high cognitive groups based on high-level cognitive behavior. Through the feature selection step, significant edgesare retained (p-value = δ = 10 , δ = 20 and δ = 30 ). Black boxes provide the classification accuraciesapplying FC profiles with the AE network processing, while blue boxes represent the rates using those without the AE network. Note that in the analysis, theSDL model is involved in these two different scenarios. For language comprehension, we demonstrated that the frontaland temporal lobes closely interacted with it. Broca’ area (inthe frontal lobe) and Wernicke’s area (in the temporal lobe)are cortical areas that respond to human language. In sum,connectomes which are refined by our approach can improveour ability to characterize the relationship between brain regionsand cognitive behavior and help enhance our understanding ofthe human brain.To further analyze the predictive power of refined FCs, weinvestigated the classification analysis and obtained satisfyingclassification rates regardless of the percentile values δ . Byusing the AE network, more stable rates were obtained forfluid intelligence and cognitive flexibility with gained accuracyacross all the conditions. However, no improvement was madeby removing signals from neural activities for the languagecomprehension. Hence, we consider the basic configuration inthe connectivity map has a different contribution to variouscognitive behavior.Several issues need further consideration. First, in this work,we still implemented the static functional network connectivity.However, recent research has noted that dynamic functionalconnectivity could provide complementary individual informa-tion [18]. A combination of dynamic and static connectivity is apromising direction for analyzing individual variability. Second,several confounding factors might affect the performance ofour proposed approach, such as parcellation schemes, and headmotion. Follow-up studies are needed to further explore thesefactors. Third, we focus on group common and individualizedaspects of the connectome. More work is needed to more fullyunderstand these aspects of brain function.V. C ONCLUSION
In this work, we assumed that the common neural activitiesmight weaken the difference in brain connectivity across partic-ipants. By proposing our framework including the autoencoder network, we reduced these common patterns of connectivityto enhance the uniqueness of each individual. We observedthat refined FC profiles estimated by our proposed pipelinecan identify each individual with high accuracy (up to . for the rest-rest pair). Meanwhile, connectomes refined byour approach can also be used to predict high level cognitivebehavior (e.g., fluid intelligence). Hence, reducing the signalsof common neural activities indeed improved both individualidentification and prediction of cognitive function, wherefrontal, parietal, and temporal lobes contributed significantly.In summary, the findings in this study validate our hypothesis,and our proposed approach provides a promising way to studyindividualized brain networks.A CKNOWLEDGMENT
Data were provided in part by the Human ConnectomeProject, WU-Minn Consortium (principal investigators, D. VanEssen and K. Ugurbil; 1U54MH091657) funded by the 16 USNational Institutes of Health (NIH) institutes and centers thatsupport the NIH Blueprint for Neuroscience Research; and bythe McDonnell Center for Systems Neuroscience at WashingtonUniversity. The authors would like to thank the partial supportby NIH (R01 GM109068, R01 EB020407, R01 MH104680,R01 MH107354, R01 MH103220) and NSF (
EFERENCES[1] B. Biswal et al. , “Functional connectivity in the motor cortex of restinghuman brain using echo-planar mri,”
Magnetic resonance in medicine ,vol. 34, no. 4, pp. 537–541, 1995.[2] M. D. Greicius et al. , “Functional connectivity in the resting brain: anetwork analysis of the default mode hypothesis,”
Proceedings of theNational Academy of Sciences , vol. 100, no. 1, pp. 253–258, 2003.[3] J. S. Damoiseaux et al. , “Consistent resting-state networks across healthysubjects,”
Proceedings of the national academy of sciences , vol. 103,no. 37, pp. 13 848–13 853, 2006.[4] R. D. Airan et al. , “Factors affecting characterization and localization ofinterindividual differences in functional connectivity using mri,”
Humanbrain mapping , vol. 37, no. 5, pp. 1986–1997, 2016. [5] A. Baldassarre et al. , “Individual variability in functional connectivitypredicts performance of a perceptual task,” Proceedings of the NationalAcademy of Sciences , vol. 109, no. 9, pp. 3516–3521, 2012.[6] R. T. Gerraty et al. , “Transfer of learning relates to intrinsic connectivitybetween hippocampus, ventromedial prefrontal cortex, and large-scalenetworks,”
Journal of Neuroscience , vol. 34, no. 34, pp. 11 297–11 303,2014.[7] E. S. Finn et al. , “Functional connectome fingerprinting: identifyingindividuals using patterns of brain connectivity,”
Nature neuroscience ,vol. 18, no. 11, p. 1664, 2015.[8] T. Kaufmann et al. , “Delayed stabilization and individualization inconnectome development are related to psychiatric disorders,”
Natureneuroscience , vol. 20, no. 4, p. 513, 2017.[9] B. Cai et al. , “Refined measure of functional connectomes for improvedidentifiability and prediction,”
Human brain mapping , vol. 40, no. 16,pp. 4843–4858, 2019.[10] R. M. Hutchison et al. , “Resting-state networks show dynamic functionalconnectivity in awake humans and anesthetized macaques,”
Human brainmapping , vol. 34, no. 9, pp. 2154–2177, 2013.[11] R. M. Hutchison et al. , “Dynamic functional connectivity: promise, issues,and interpretations,”
Neuroimage , vol. 80, pp. 360–378, 2013.[12] E. A. Allen et al. , “Tracking whole-brain connectivity dynamics in theresting state,”
Cerebral cortex , vol. 24, no. 3, pp. 663–676, 2014.[13] V. D. Calhoun et al. , “The chronnectome: time-varying connectivitynetworks as the next frontier in fmri data discovery,”
Neuron , vol. 84,no. 2, pp. 262–274, 2014.[14] V. D. Calhoun et al. , “Time-varying brain connectivity in fmri data:whole-brain data-driven approaches for capturing and characterizingdynamic states,”
IEEE Signal Processing Magazine , vol. 33, no. 3, pp.52–66, 2016.[15] B. Cai et al. , “Estimation of dynamic sparse connectivity patterns fromresting state fmri,”
IEEE transactions on medical imaging , vol. 37, no. 5,pp. 1224–1234, 2017.[16] B. Cai et al. , “Capturing dynamic connectivity from resting state fmriusing time-varying graphical lasso,”
IEEE Transactions on BiomedicalEngineering , vol. 66, no. 7, pp. 1852–1862, 2018.[17] G. Zhang et al. , “Estimating dynamic functional brain cconnectivity witha sparse hidden markov model,”
IEEE transactions on medical imaging ,2019.[18] J. Liu et al. , “Chronnectome fingerprinting: identifying individuals andpredicting higher cognitive functions using dynamic brain connectivitypatterns,”
Human brain mapping , vol. 39, no. 2, pp. 902–915, 2018.[19] S. Wold et al. , “Principal component analysis,”
Chemometrics andintelligent laboratory systems , vol. 2, no. 1-3, pp. 37–52, 1987.[20] E. C. Malthouse, “Limitations of nonlinear pca as performed with genericneural networks,”
IEEE Transactions on neural networks , vol. 9, no. 1,pp. 165–173, 1998.[21] M. A. Kramer, “Nonlinear principal component analysis using autoasso-ciative neural networks,”
AIChE journal , vol. 37, no. 2, pp. 233–243,1991.[22] J. Karhunen et al. , “Generalizations of principal component analysis,optimization problems, and neural networks,”
Neural Networks , vol. 8,no. 4, pp. 549–562, 1995.[23] B. Sch¨olkopf et al. , “Nonlinear component analysis as a kernel eigenvalueproblem,”
Neural computation , vol. 10, no. 5, pp. 1299–1319, 1998.[24] G. E. Hinton et al. , “Reducing the dimensionality of data with neuralnetworks,”
Science , vol. 313, no. 5786, pp. 504–507, 2006.[25] H. Lee et al. , “Sparse deep belief net model for visual area v2,” in
Advances in neural information processing systems , 2008, pp. 873–880.[26] S. Rifai et al. , “Contractive auto-encoders: Explicit invariance duringfeature extraction,” in
Proceedings of the 28th International Conferenceon International Conference on Machine Learning . Omnipress, 2011,pp. 833–840.[27] P. Vincent et al. , “Extracting and composing robust features withdenoising autoencoders,” in
Proceedings of the 25th internationalconference on Machine learning . ACM, 2008, pp. 1096–1103.[28] P. Vincent et al. , “Stacked denoising autoencoders: Learning usefulrepresentations in a deep network with a local denoising criterion,”
Journal of machine learning research , vol. 11, no. Dec, pp. 3371–3408,2010.[29] Q. V. Le, “Building high-level features using large scale unsupervisedlearning,” in . IEEE, 2013, pp. 8595–8598.[30] D. C. Van Essen et al. , “The wu-minn human connectome project: anoverview,”
Neuroimage , vol. 80, pp. 62–79, 2013.[31] H. WU-Minn, “1200 subjects data release reference manual,” , 2017. [32] M. F. Glasser et al. , “The minimal preprocessing pipelines for the humanconnectome project,”
Neuroimage , vol. 80, pp. 105–124, 2013.[33] X. Shen et al. , “Groupwise whole-brain parcellation from resting-statefmri data for network node identification,”
Neuroimage , vol. 82, pp.403–415, 2013.[34] V. Nair et al. , “Rectified linear units improve restricted boltzmannmachines,” in
Proceedings of the 27th international conference onmachine learning (ICML-10) , 2010, pp. 807–814.[35] L. Van Der Maaten, “Learning a parametric embedding by preservinglocal structure,” in
Artificial Intelligence and Statistics , 2009, pp. 384–391.[36] E. S. Finn et al. , “Can brain state be manipulated to emphasize individualdifferences in functional connectivity?”
Neuroimage , vol. 160, pp. 140–151, 2017.[37] M. W. Cole et al. , “Multi-task connectivity reveals flexible hubs foradaptive task control,”
Nature neuroscience , vol. 16, no. 9, p. 1348, 2013.[38] R. Li et al. , “Linking inter-individual variability in functional brainconnectivity to cognitive ability in elderly individuals,”
Frontiers inaging neuroscience , vol. 9, p. 385, 2017.[39] N. Tschentscher et al. , “Fluid intelligence predicts novel rule imple-mentation in a distributed frontoparietal control network,”
Journal ofNeuroscience , vol. 37, no. 18, pp. 4841–4847, 2017.[40] G. Leisman et al. , “Thinking, walking, talking: integratory motor andcognitive brain function,”