Itir Onal

Functional Mesh Learning for pattern analysis of cognitive processes

We propose a statistical learning model for classifying cognitive processes based on distributed patterns of neural activation in the brain, acquired via functional magnetic resonance imaging (fMRI). In the proposed learning machine, local meshes are formed around each voxel. The distance between voxels in the mesh is determined by using functional neighborhood concept. In order to define functional neighborhood, the similarities between the time series recorded for voxels are measured and functional connectivity matrices are constructed. Then, the local mesh for each voxel is formed by including the functionally closest neighboring voxels in the mesh. The relationship between the voxels within a mesh is estimated by using a linear regression model. These relationship vectors, called Functional Connectivity aware Local Relational Features (FC-LRF) are then used to train a statistical learning machine. The proposed method was tested on a recognition memory experiment, including data pertaining to encoding and retrieval of words belonging to ten different semantic categories. Two popular classifiers, namely k-Nearest Neighbor and Support Vector Machine, are trained in order to predict the semantic category of the item being retrieved, based on activation patterns during encoding. The classification performance of the Functional Mesh Learning model, which range in 62-68% is superior to the classical multi-voxel pattern analysis (MVPA) methods, which range in 40-48%, for ten semantic categories.

Modeling Voxel Connectivity for Brain Decoding

The massively dynamic nature of human brain cannot be represented by considering only a collection of voxel intensity values obtained from fMRI measurements. It has been observed that the degree of connectivity among voxels provide important information for modeling cognitive activities. Moreover, spatially close voxels act together to generate similar BOLD responses to the same stimuli. In this study, we propose a local mesh model, called Local Mesh Model with Temporal Measurements (LMM-TM), to first estimate spatial relationship among a set of voxels using spatial and temporal data measured at each voxel, and then employ the relationship for the construction of a connectivity model for brain decoding. For this purpose, we first construct a local mesh around each voxel (called seed voxel) by connecting it to its spatially nearest neighbors. Then, we represent the BOLD response of each seed voxel in terms of linear combination of the BOLD responses of its p-nearest neighbors. The relationship between a seed voxel and its neighbors is estimated by solving a linear regression problem. The estimated mesh arc weights are used to model local connectivity among the voxels that reside in a spatial neighborhood. Using these weights as features, we train Support Vector Machines and k-Nearest Neighbor classifiers. We test our model on a visual object recognition experiment. In the experimental analysis, we observe that classifiers that employ our features perform better than classifiers that employ raw voxel intensity values, local mesh model weights and features extracted using distance metrics such as Euclidean distance, cosine similarity and Pearson correlation.

Analyzing the information distribution in the fMRI measurements by estimating the degree of locality

In this study, we propose a new method for analyzing and representing the distribution of discriminative information for data acquired via functional Magnetic Resonance Imaging (fMRI). For this purpose, we form a spatially local mesh with varying size, around each voxel, called the seed voxel. The relationship among each seed voxel and its neighbors is estimated using a linear regression model by minimizing the square error. Then, we estimate the optimal mesh size that represents the connections among each seed voxel and its surroundings by minimizing Akaikes Final Prediction Error (FPE) with respect to the mesh size. The degree of locality is represented by the optimum mesh size. Our results indicate that the local mesh size with the highest discriminative power varies across individual participants. The proposed method was tested on an fMRI study consisting of item recognition (IR) and judgment of recency (JOR) tasks. For each participant, the estimated arc weights of each local mesh with different mesh size are used to classify the type of memory judgment (i.e.IR or JOR). Classification accuracy for each participant was derived using k-Nearest Neighbor (k-NN) method. The results indicate that the proposed local mesh model with optimal mesh size can successfully represent discriminative information for neuroimaging data.

Functional Mesh Model with Temporal Measurements for brain decoding

We propose a method called Functional Mesh Model with Temporal Measurements (FMM-TM) to estimate a functional relationship among voxels using temporal data, and employ these relationships for brain decoding. For each sample, we measure Blood Oxygenation Level Dependent (BOLD) responses from each voxel, and construct a functional mesh around each voxel (called seed voxel) with its nearest neighbors selected using distance metrics namely Pearson correlation, cosine similarity and Euclidean distance. Then, we represent the BOLD response of a seed voxel in terms of linear combination of BOLD responses of its p-nearest neighbors. The relationship between the seed voxel and its neighbors is represented using a set of weights which are estimated by employing linear regression. We train Support Vector Machine and k-Nearest Neighbor classifiers using the estimated weights as features. We test our model in an event-related design experiment, namely object recognition, and observe that our features perform better than raw voxel intensity values, features obtained using various pairwise distance metrics, and local mesh model features extracted using stationary and temporal measurements.

A framework for detecting complex events in surveillance videos

This paper presents a framework for detecting complex events in surveillance videos. Moving objects in the foreground are detected in the object detection component of the system. Whether these foregrounds are human or not is decided in the object recognition component. Then each detected object is tracked and labeled in the object tracking component, in which true labeling of objects in the occlusion situation is also provided. The extracted information is fed to the event detection component. Rule based event models are created and trained using Markov Logic Networks (MLNs) so that each rule is given a weight. Events are inferred using MLNs where the assigned weights are used to determine whether an event occurs or not. The proposed system can be applied to detect many complex events simultaneously. In this paper, detection of left object event is discussed and evaluated using PETS-2006, CANTATA and our dataset.

Mesh learning approach for brain data modeling

The major goal of this study is to model the memory process using neural activation patterns in the brain. To achieve this goal, neural activation was acquired using functional Magnetic Resonance Imaging (fMRI) during memory encoding and retrieval. fMRI are known are trained for each class using a learning system. The most important component of this learning system is feature space. In this project, an original feature space for the fMRI data is proposed. This feature space is defined by a mesh network which models the relationship between voxels. In the suggested mesh network, the distance between voxels is determined by using physical and functional neighborhood concepts. For the functional neighborhood, the similarities between the time series, gained from voxels, are measured. With the proposed method, a data set with 10 classes is used for the encoding and retrieval processes, and the classifier is trained with the learning algorithms in order to predict the class the data belongs.

A New Representation of fMRI Signal by a Set of Local Meshes for Brain Decoding

How neurons influence each others firing depends on the strength of synaptic connections among them. Motivated by the highly interconnected structure of the brain, in this study, we propose a computational model to estimate the relationships among voxels and employ them as features for cognitive state classification. We represent the sequence of functional Magnetic Resonance Imaging (fMRI) measurements recorded during a cognitive stimulus by a set of local meshes. Then, we represent the corresponding cognitive state by the edge weights of these meshes each of which is estimated assuming a regularized linear relationship among voxel time series in a predefined locality. The estimated mesh edge weights provide a better representation of information in the brain for cognitive state or task classification. We examine the representative power of our mesh edge weights on visual recognition and emotional memory retrieval experiments by training a support vector machine classifier. Also, we use mesh edge weights as feature vectors of inter-subject classification on Human Connectome Project task fMRI dataset, and test their performance. We observe that mesh edge weights perform better than the popular fMRI features, such as, raw voxel intensity values, pairwise correlations, features extracted using PCA and ICA, for classifying the cognitive states.

LARGE SCALE FUNCTIONAL CONNECTIVITY FOR BRAIN DECODING

Functional Magnetic Resonance Imaging (fMRI) data consists of time series for each voxel recorded during a cognitive task. In order to extract useful information from this noisy and redundant data, techniques are proposed to select the voxels that are relevant to the underlying cognitive task. We propose a simple and efficient algorithm for decoding the brain states by modelling the correlation patterns between the voxel time series. For each stimulus during the experiment, a separate functional connectivity matrix is computed in voxel level. The elements in connectivity matrices are then filtered out by making use of a minimum spanning tree formed using a global connectivity matrix for the entire experiment in order to reduce dimensionality. For a recognition memory experiment with nine subjects, functional connectivity matrices are computed for encoding and retrieval phases. The class labels of the retrieval samples are predicted within a k-nearest neighbour space constructed by the traversed entries in the functional connectivity matrices for encoding samples. The proposed method is also adapted to large scale functional connectivity tasks by making use of graphics boards. Classification performance in ten categories is comparable and even better compared to both classical and enhanced methods of multi-voxel pattern analysis techniques.

Representation of cognitive processes using the minimum spanning tree of local meshes

A new graphical model called Cognitive Process Graph (CPG) is proposed, for classifying cognitive processes based on neural activation patterns which are acquired via functional Magnetic Resonance Imaging (fMRI) in brain. In the CPG, first local meshes are formed around each voxel. Second, the relationships between a voxel and its neighbors in a local mesh, which are estimated by using a linear regression model, are used to form the edges among the voxels (graph nodes) in the CPG. Then, a minimum spanning tree (MST) of the CPG which spans all the voxels in the region of interest is computed. The arc weights of the MST are used to represent the underlying cognitive processes. The proposed method reduces the curse of dimensionality problem that is caused by very large dimension of the feature space of the fMRI measurements, compared to number of instances. Finally, the arc weights computed over the path of the MST called MST-Features (MST-F) are used to train a statistical learning machine. The proposed method is tested on a recognition memory experiment, including data pertaining to encoding and retrieval of words belonging to ten different semantic categories. Two popular classifiers, namely k-Nearest Neighbor (k-NN) and Support Vector Machine (SVM), are trained in order to predict the semantic category of the item being retrieved, based on activation patterns during encoding. The classification performance of the proposed learning model is superior to the classical multi-voxel pattern analysis (MVPA) methods for the underlying cognitive process.

Effect of using regression in sentiment analysis

In this study, the effect of using regression on sentiment classification of Twitter data was analyzed. In other words, whether the strength of sentiment better discriminates the classes or not. Since our dataset includes class confidence scores rather than discrete class labels, regression analysis was employed on each class separately. Then, each tweet was assigned the class whose estimated confidence score is maximum among others after regression. The feature set used includes unigrams, POS tags, emoticons, sentiments of words and POS tags of sentiments. The results of experiments indicate that using classification on discrete class labels perform much better than using regression on continuous confidence scores.