Svetlana V. Shinkareva

Predicting human brain activity associated with the meanings of nouns.

The question of how the human brain represents conceptual knowledge has been debated in many scientific fields. Brain imaging studies have shown that different spatial patterns of neural activation are associated with thinking about different semantic categories of pictures and words (for example, tools, buildings, and animals). We present a computational model that predicts the functional magnetic resonance imaging (fMRI) neural activation associated with words for which fMRI data are not yet available. This model is trained with a combination of data from a trillion-word text corpus and observed fMRI data associated with viewing several dozen concrete nouns. Once trained, the model predicts fMRI activation for thousands of other concrete nouns in the text corpus, with highly significant accuracies over the 60 nouns for which we currently have fMRI data.

Neural representation of abstract and concrete concepts: a meta-analysis of neuroimaging studies.

A number of studies have investigated differences in neural correlates of abstract and concrete concepts with disagreement across results. A quantitative, coordinate‐based meta‐analysis combined data from 303 participants across 19 functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies to identify the differences in neural representation of abstract and concrete concepts. Studies that reported peak activations in standard space in contrast of abstract > concrete or concrete > abstract concepts at a whole brain level in healthy adults were included in this meta‐analysis. Multilevel kernel density analysis (MKDA) was performed to identify the proportion of activated contrasts weighted by sample size and analysis type (fixed or random effects). Meta‐analysis results indicated consistent and meaningful differences in neural representation for abstract and concrete concepts. Abstract concepts elicit greater activity in the inferior frontal gyrus and middle temporal gyrus compared to concrete concepts, while concrete concepts elicit greater activity in the posterior cingulate, precuneus, fusiform gyrus, and parahippocampal gyrus compared to abstract concepts. These results suggest greater engagement of the verbal system for processing of abstract concepts and greater engagement of the perceptual system for processing of concrete concepts, likely via mental imagery. Hum Brain Mapp, 2010.

Commonality of neural representations of words and pictures

In this work we explore whether the patterns of brain activity associated with thinking about concrete objects are dependent on stimulus presentation format, whether an object is referred to by a written or pictorial form. Multi-voxel pattern analysis methods were applied to brain imaging (fMRI) data to identify the item category associated with brief viewings of each of 10 words (naming 5 tools and 5 dwellings) and, separately, with brief viewings of each of 10 pictures (line drawings) of the objects named by the words. These methods were able to identify the category of the picture the participant was viewing, based on neural activation patterns observed during word-viewing, and identify the category of the word the participant was viewing, based on neural activation patterns observed during picture-viewing, using data from only that participant or only from other participants. These results provide an empirical demonstration of object category identification across stimulus formats and across participants. In addition, we were able to identify the category of the word that the participant was viewing based on the patterns of neural activation generated during word-viewing by that participant or by all other participants. Similarly, we were able to identify with even higher accuracy the category of the picture the participant was viewing, based on the patterns of neural activation demonstrated during picture-viewing by that participant or by all other participants. The brain locations that were important for category identification were similar across participants and were distributed throughout the cortex where various object properties might be neurally represented. These findings indicate consistent triggering of semantic representations using different stimulus formats and suggest the presence of stable, distributed, and identifiable neural states that are common to pictorial and verbal input referring to object categories.

Classification of functional brain images with a spatio-temporal dissimilarity map

Classification of subjects into predefined groups, such as patient vs. control, based on their functional MRI data is a potentially useful procedure for clinical diagnostic purposes. This paper presents an automated method for classifying subjects into groups based on their functional MRI data. The proposed methodology provides general framework using preprocessed time series for the whole brain volume. Using a training set of two groups of subjects, the new methodology identifies spatio-temporal features that distinguish the groups and uses these features to categorize new subjects. We demonstrate the method using simulations and a clinical application that classifies individuals into schizotypy and control groups.

Predicting cognitive state from eye movements.

In human vision, acuity and color sensitivity are greatest at the center of fixation and fall off rapidly as visual eccentricity increases. Humans exploit the high resolution of central vision by actively moving their eyes three to four times each second. Here we demonstrate that it is possible to classify the task that a person is engaged in from their eye movements using multivariate pattern classification. The results have important theoretical implications for computational and neural models of eye movement control. They also have important practical implications for using passively recorded eye movements to infer the cognitive state of a viewer, information that can be used as input for intelligent human-computer interfaces and related applications.

Decoding the neural representation of affective states.

Brain activity was monitored while participants viewed picture sets that reflected high or low levels of arousal and positive, neutral, or negative valence. Pictures within a set were presented rapidly in an incidental viewing task while fMRI data were collected. The primary purpose of the study was to determine if multi-voxel pattern analysis could be used within and between participants to predict valence, arousal and combined affective states elicited by pictures based on distributed patterns of whole brain activity. A secondary purpose was to determine if distributed patterns of whole brain activity can be used to derive a lower dimensional representation of affective states consistent with behavioral data. Results demonstrated above chance prediction of valence, arousal and affective states that was robust across a wide range of number of voxels used in prediction. Additionally, individual differences multidimensional scaling based on fMRI data clearly separated valence and arousal levels and was consistent with a circumplex model of affective states.

Differential Deactivation during Mentalizing and Classification of Autism Based on Default Mode Network Connectivity

The default mode network (DMN) is a collection of brain areas found to be consistently deactivated during task performance. Previous neuroimaging studies of resting state have revealed reduced task-related deactivation of this network in autism. We investigated the DMN in 13 high-functioning adults with autism spectrum disorders (ASD) and 14 typically developing control participants during three fMRI studies (two language tasks and a Theory-of-Mind (ToM) task). Each study had separate blocks of fixation/resting baseline. The data from the task blocks and fixation blocks were collated to examine deactivation and functional connectivity. Deficits in the deactivation of the DMN in individuals with ASD were specific only to the ToM task, with no group differences in deactivation during the language tasks or a combined language and self-other discrimination task. During rest blocks following the ToM task, the ASD group showed less deactivation than the control group in a number of DMN regions, including medial prefrontal cortex (MPFC), anterior cingulate cortex, and posterior cingulate gyrus/precuneus. In addition, we found weaker functional connectivity of the MPFC in individuals with ASD compared to controls. Furthermore, we were able to reliably classify participants into ASD or typically developing control groups based on both the whole-brain and seed-based connectivity patterns with accuracy up to 96.3%. These findings indicate that deactivation and connectivity of the DMN were altered in individuals with ASD. In addition, these findings suggest that the deficits in DMN connectivity could be a neural signature that can be used for classifying an individual as belonging to the ASD group.

Identifying bilingual semantic neural representations across languages

The goal of the study was to identify the neural representation of a nouns meaning in one language based on the neural representation of that same noun in another language. Machine learning methods were used to train classifiers to identify which individual noun bilingual participants were thinking about in one language based solely on their brain activation in the other language. The study shows reliable (p<.05) pattern-based classification accuracies for the classification of brain activity for nouns across languages. It also shows that the stable voxels used to classify the brain activation were located in areas associated with encoding information about semantic dimensions of the words in the study. The identification of the semantic trace of individual nouns from the pattern of cortical activity demonstrates the existence of a multi-voxel pattern of activation across the cortex for a single noun common to both languages in bilinguals.

The Dual-Factor Model of Mental Health Further Study of the Determinants of Group Differences

Consistent with a positive psychology framework, this study examined the contributions of personality, environmental, and perceived social support variables in classifying adolescents using Greenspoon and Saklofske’s Dual-Factor model of mental health. This model incorporates information about positive subjective well-being (SWB), along with psychopathology (PTH), to identify four groups of adolescents: positive mental health (high SWB, low PTH), vulnerable (low SWB, low PTH), symptomatic but content (high SWB, high PTH) and troubled (low SWB, high PTH). Using multinomial logistic regression analyses, adolescents were accurately classified into the four groups above chance. The contribution of the personality, social support, and stressful life events variables differed across the groups. Differences in perceived parent social support statistically significantly differentiated (p < .05) the vulnerable and troubled groups from the positive mental health group. The experience of stressful life events significantly differentiated the troubled group from the positive mental health group. The personality characteristics of Extraversion and Neuroticism significantly differentiated symptomatic but content and troubled students from the positive mental health group. The study thus identified relatively malleable factors (e.g., parent support) that relate to optimal mental health.

Decoding abstract and concrete concept representations based on single-trial fMRI data

Previously, multi‐voxel pattern analysis has been used to decode words referring to concrete object categories. In this study we investigated if single‐trial‐based brain activity was sufficient to distinguish abstract (e.g., mercy) versus concrete (e.g., barn) concept representations. Multiple neuroimaging studies have identified differences in the processing of abstract versus concrete concepts based on the averaged activity across time by using univariate methods. In this study we used multi‐voxel pattern analysis to decode functional magnetic resonance imaging (fMRI) data when participants perform a semantic similarity judgment task on triplets of either abstract or concrete words with similar meanings. Classifiers were trained to identify individual trials as concrete or abstract. Cross‐validated accuracies for classifying trials as abstract or concrete were significantly above chance (P < 0.05) for all participants. Discriminating information was distributed in multiple brain regions. Moreover, accuracy of identifying single trial data for any one participant as abstract or concrete was also reliably above chance (P < 0.05) when the classifier was trained solely on data from other participants. These results suggest abstract and concrete concepts differ in representations in terms of neural activity patterns during a short period of time across the whole brain. Hum Brain Mapp, 2013.