Devon R. Hjelm

Deep learning for neuroimaging: a validation study

Deep learning methods have recently made notable advances in the tasks of classification and representation learning. These tasks are important for brain imaging and neuroscience discovery, making the methods attractive for porting to a neuroimagers toolbox. Success of these methods is, in part, explained by the flexibility of deep learning models. However, this flexibility makes the process of porting to new areas a difficult parameter optimization problem. In this work we demonstrate our results (and feasible parameter ranges) in application of deep learning methods to structural and functional brain imaging data. These methods include deep belief networks and their building block the restricted Boltzmann machine. We also describe a novel constraint-based approach to visualizing high dimensional data. We use it to analyze the effect of parameter choices on data transformations. Our results show that deep learning methods are able to learn physiologically important representations and detect latent relations in neuroimaging data.

Assessing dynamic brain graphs of time-varying connectivity in fMRI data: application to healthy controls and patients with schizophrenia

Graph theory-based analysis has been widely employed in brain imaging studies, and altered topological properties of brain connectivity have emerged as important features of mental diseases such as schizophrenia. However, most previous studies have focused on graph metrics of stationary brain graphs, ignoring that brain connectivity exhibits fluctuations over time. Here we develop a new framework for accessing dynamic graph properties of time-varying functional brain connectivity in resting-state fMRI data and apply it to healthy controls (HCs) and patients with schizophrenia (SZs). Specifically, nodes of brain graphs are defined by intrinsic connectivity networks (ICNs) identified by group independent component analysis (ICA). Dynamic graph metrics of the time-varying brain connectivity estimated by the correlation of sliding time-windowed ICA time courses of ICNs are calculated. First- and second-level connectivity states are detected based on the correlation of nodal connectivity strength between time-varying brain graphs. Our results indicate that SZs show decreased variance in the dynamic graph metrics. Consistent with prior stationary functional brain connectivity works, graph measures of identified first-level connectivity states show lower values in SZs. In addition, more first-level connectivity states are disassociated with the second-level connectivity state which resembles the stationary connectivity pattern computed by the entire scan. Collectively, the findings provide new evidence about altered dynamic brain graphs in schizophrenia, which may underscore the abnormal brain performance in this mental illness.

Deep independence network analysis of structural brain imaging: A simulation study

The objective of this paper is to further validate theoretically and empirically a nonlinear independent component analysis (ICA) algorithm implemented with a deep learning architecture. We first revisited its formulation to verify its consistency with the criterion of minimization of mutual information. Then, we applied the nonlinear independent component estimation algorithm (NICE) to synthetic 2D images that resemble structural magnetic resonance imaging (sMRI) data. This data was generated by mixing spatial components that represent axial slices of sMRI tissue concentration images. Next, we generated the images under linear and mildly nonlinear mixtures, being able to show that NICE matches ICA when the data is generated by using the conventional linear mixture and outperforms ICA for the nonlinear mixture of components. The obtained results are promising and suggest that NICE has potential to find richer brain networks if applied to real sMRI data, provided that small conditioning adjustments are performed along with this approach.

Reading the (functional) writing on the (structural) wall: Multimodal fusion of brain structure and function via a deep neural network based translation approach reveals novel impairments in schizophrenia

ABSTRACT This work presents a novel approach to finding linkage/association between multimodal brain imaging data, such as structural MRI (sMRI) and functional MRI (fMRI). Motivated by the machine translation domain, we employ a deep learning model, and consider two different imaging views of the same brain like two different languages conveying some common facts. That analogy enables finding linkages between two modalities. The proposed translation‐based fusion model contains a computing layer that learns “alignments” (or links) between dynamic connectivity features from fMRI data and static gray matter patterns from sMRI data. The approach is evaluated on a multi‐site dataset consisting of eyes‐closed resting state imaging data collected from 298 subjects (age‐ and gender matched 154 healthy controls and 144 patients with schizophrenia). Results are further confirmed on an independent dataset consisting of eyes‐open resting state imaging data from 189 subjects (age‐ and gender matched 91 healthy controls and 98 patients with schizophrenia). We used dynamic functional connectivity (dFNC) states as the functional features and ICA‐based sources from gray matter densities as the structural features. The dFNC states characterized by weakly correlated intrinsic connectivity networks (ICNs) were found to have stronger association with putamen and insular gray matter pattern, while the dFNC states of profuse strongly correlated ICNs exhibited stronger links with the gray matter pattern in precuneus, posterior cingulate cortex (PCC), and temporal cortex. Further investigation with the estimated link strength (or alignment score) showed significant group differences between healthy controls and patients with schizophrenia in several key regions including temporal lobe, and linked these to connectivity states showing less occupancy in healthy controls. Moreover, this novel approach revealed significant correlation between a cognitive score (attention/vigilance) and the function/structure alignment score that was not detected when data modalities were considered separately.

A deep-learning approach to translate between brain structure and functional connectivity

While the majority of exploratory approaches search for correlations among features of different modalities, indirect/nonlinear relations between structure and function have not yet been fully investigated. In this work, we employ a neural machine translation model [1] to relate two modalities: structural MRI (sMRI) spatial components and functional MRI (fMRI) brain states estimated using a dynamic connectivity model. We consider each of the modalities as different “languages” of the same brain and fit a translation model to estimate a model for how structure influences function. Results identify multiple aligned aspects of brain structure and functional brain states showing significantly more or less alignment in the patient group as well as interesting links to other variables such as cognitive scores and symptom assessments. Our novel approach provides a new perspective on combining brain structure and function by incorporating indirect/nonlinear effects and enabling the algorithm to learn the interplay between structural and the functional networks.

Multimodal fusion of brain structural and functional imaging with a deep neural machine translation approach

In this work, we study a novel approach of deep neural machine translation to find linkage between multimodal brain imaging data, such as structural MRI (sMRI) and functional MRI (fMRI). The idea is to consider two different imaging views of the same brain like two different languages conveying some common concepts or facts. An important aspect of the translation model is an attention network module that learns alignment between features from fMRI and sMRI. We use independent component analysis (ICA) based features for the translation model. Our study shows significant group differences between healthy controls and patients with schizophrenia in the learned alignments. Furthermore, this novel approach reveals a group differential relation between a cognitive score (attention and vigilance) and alignments that could not be found when individual modality of data were considered.