Qinglin Dong

Automatic Recognition of fMRI-Derived Functional Networks Using 3-D Convolutional Neural Networks

Current functional magnetic resonance imaging (fMRI) data modeling techniques, such as independent component analysis and sparse coding methods, can effectively reconstruct dozens or hundreds of concurrent interacting functional brain networks simultaneously from the whole brain fMRI signals. However, such reconstructed networks have no correspondences across different subjects. Thus, automatic, effective, and accurate classification and recognition of these large numbers of fMRI-derived functional brain networks are very important for subsequent steps of functional brain analysis in cognitive and clinical neuroscience applications. However, this task is still a challenging and open problem due to the tremendous variability of various types of functional brain networks and the presence of various sources of noises. In recognition of the fact that convolutional neural networks (CNN) has superior capability of representing spatial patterns with huge variability and dealing with large noises, in this paper, we design, apply, and evaluate a deep 3-D CNN framework for automatic, effective, and accurate classification and recognition of large number of functional brain networks reconstructed by sparse representation of whole-brain fMRI signals. Our extensive experimental results based on the Human Connectome Project fMRI data showed that the proposed deep 3-D CNN can effectively and robustly perform functional networks classification and recognition tasks, while maintaining a high tolerance for mistakenly labeled training instances. This study provides a new deep learning approach for modeling functional connectomes based on fMRI data.

Modeling Task fMRI Data Via Deep Convolutional Autoencoder

Task-based fMRI (tfMRI) has been widely used to study functional brain networks. Modeling tfMRI data is challenging due to at least two problems: the lack of the ground truth of underlying neural activity and the intrinsic structure of tfMRI data is highly complex. To better understand brain networks based on fMRI data, data-driven approaches were proposed, for instance, Independent Component Analysis (ICA) and Sparse Dictionary Learning (SDL). However, both ICA and SDL only build shallow models, and they are under the strong assumption that original fMRI signal could be linearly decomposed into time series components with their corresponding spatial maps. As growing evidence shows that human brain function is hierarchically organized, new approaches that can infer and model the hierarchical structure of brain networks are widely called for. Recently, deep convolutional neural network (CNN) has drawn much attention, in that deep CNN has been proven to be a powerful method for learning high-level and mid-level abstractions from low-level raw data. Inspired by the power of deep CNN, in this study, we developed a new neural network structure based on CNN, called Deep Convolutional Auto-Encoder (DCAE), in order to take the advantages of both data-driven approach and CNN’s hierarchical feature abstraction ability for the purpose of learning mid-level and high-level features from complex tfMRI time series in an unsupervised manner. The DCAE has been applied and tested on the publicly available human connectome project (HCP) tfMRI datasets, and promising results are achieved.

What Makes a Good Movie Trailer?: Interpretation from Simultaneous EEG and Eyetracker Recording

What makes a good movie trailer? Its a big challenge to answer this question because of the complexity of multimedia in both low level sensory features and high level semantic features. However, human perception and reactivity could be straightforward evidence for evaluation. Modern Electro-encephalography (EEG) technology provides measurement of consequential brain neural activity to external stimuli. Meanwhile, visual perception and attention could be captured and interpreted by Eye Tracking technology. Intuitively, simultaneous EEG and Eye Tracker recording of human audience with multimedia stimuli could bridge the gap between human comprehension and multimedia analysis, and provide a new way for movie trailer evaluation. In this paper, we propose a novel platform to simultaneously record EEG and eye movement for participants with video stimuli by integrating 256-channel EEG, Eye Tracker and video display device as a system. Based on the proposed system a novel experiment has been designed, in which independent and joint features of EEG and Eye tracking data were mined to evaluate the movie trailer. Our analysis has shown interesting features that are corresponding with trailer quality and video shoot changes.

3-D functional brain network classification using Convolutional Neural Networks

Several recent studies have shown that dictionary learning and sparse representation can effectively reconstruct hundreds of interacting functional brain networks simultaneously from whole-brain fMRI data. However, accurate classification and recognition of those hundreds of functional networks from an individual or a population of many subjects is still a challenging and open problem due to the intrinsic variability of functional networks and other noise sources. To tackle this problem, this paper presents an effective deep learning framework to train convolutional neural networks from a large dataset of hundreds of thousands of available brain network volume maps, which was then applied on testing samples for network classification and recognition. We effectively applied computer-labeled data as training set so the whole process can be automated. Experimental results showed that the proposed method is quite robust in handling noisy patterns in the dataset, which suggests that our work offers a new computational framework for modeling functional connectomes from fMRI big data in the future.

Decoding dynamic auditory attention during naturalistic experience

Equipped with selective auditory attention (SAA), people are able to rapidly shift their attention to auditory events of interest. Although abstract neuroimaging paradigms are fundamental for exploring the neural basis of SAA, whether those findings are valid in a more naturalistic condition and how the types of auditory stimuli affect SAA are largely unknown. Here we propose a brain decoding study to explore SAA using naturalistic auditory excerpts in three categories (pop music, classical music and speech) as stimuli for functional magnetic resonance imaging (fMRI). We adopted a computational auditory attention model to estimate attentional allocation for the excerpts. We then extracted brain activity features from fMRI data via sparse representation and used them to decode the auditory attention allocation. Our experimental results showed that the primary auditory cortex was commonly involved in the attentional processing of the three categories and the contribution of distinct brain networks to the decoding model in each group. Our study on the one hand provides novel insights into neural SAA in naturalistic experience, on the other hand shows the possibility of leveraging neuroimaging studies by integrating naturalistic stimuli and computational auditory information processing approaches.

Predicting cortical 3-hinge locations via structural connective features

Cortical folds encode crucial information of brain development, cytoarchitecture and function. It is widely accepted that common anatomy is preserved across individuals within species, while huge variation still hamper establishing fine-grained anatomical correspondences and predicting the locations of a specific anatomical pattern via conventional image registration methods, especially for complex cortical folding pattern, such as gyral 3-hinge. Recently, white matter axonal wiring patterns have been suggested to be strongly correlative to cortical folding patterns. Therefore, in this work, we studied the relation between complex 3-hinge folding patterns and structural connective patterns, and proposed effective methods to predict the locations of 3-hinges by using structural connective features and spatial distribution patterns. The prediction accuracy of our methods outperforms conventional image registration methods.

Spatio-temporal modeling of connectome-scale brain network interactions via time-evolving graphs

ABSTRACT Many recent literature studies have revealed interesting dynamics patterns of functional brain networks derived from fMRI data. However, it has been rarely explored how functional networks spatially overlap (or interact) and how such connectome‐scale network interactions temporally evolve. To explore these unanswered questions, this paper presents a novel framework for spatio‐temporal modeling of connectome‐scale functional brain network interactions via two main effective computational methodologies. First, to integrate, pool and compare brain networks across individuals and their cognitive states under task performances, we designed a novel group‐wise dictionary learning scheme to derive connectome‐scale consistent brain network templates that can be used to define the common reference space of brain network interactions. Second, the temporal dynamics of spatial network interactions is modeled by a weighted time‐evolving graph, and then a data‐driven unsupervised learning algorithm based on the dynamic behavioral mixed‐membership model (DBMM) is adopted to identify behavioral patterns of brain networks during the temporal evolution process of spatial overlaps/interactions. Experimental results on the Human Connectome Project (HCP) task fMRI data showed that our methods can reveal meaningful, diverse behavior patterns of connectome‐scale network interactions. In particular, those networks’ behavior patterns are distinct across HCP tasks such as motor, working memory, language and social tasks, and their dynamics well correspond to the temporal changes of specific task designs. In general, our framework offers a new approach to characterizing human brain function by quantitative description for the temporal evolution of spatial overlaps/interactions of connectome‐scale brain networks in a standard reference space.

Multple-demand system identification and characterization via sparse representations of fMRI data

Identification of concurrent spatially overlapping functional networks and understanding of their mechanisms of jointly realizing the total brain function have been important yet challenging problems. In this work, we have applied a datadriven sparse representation framework to learn a dictionary consisting of multiple network components and their associated weight coefficients from a given fMRI dataset. Then we analyzed the network component composition at the voxel level by correlating component weights to the characteristics of regions with strong involvements in multiple components, which are defined as functionally highly heterogeneous regions (HHR). Consequently, the spatial overlap of HHRs obtained across multiple tasks of a given subject is defined as the multiple-demand (MD) system. By applying the proposed framework on the recently publicly released Human Connectome Project (HCP) task fMRI dataset, we have obtained reproducible HHR and MD systems that concentrated on the frontal and parietal cortex. Interestingly, the spatial distribution of those MD regions has been found to be highly correlated with the cortical folding and structural connectivities, revealing closely related brain structural and functional architectures.