Michael Andric

Gesture's Neural Language.

When people talk to each other, they often make arm and hand movements that accompany what they say. These manual movements, called “co-speech gestures,” can convey meaning by way of their interaction with the oral message. Another class of manual gestures, called “emblematic gestures” or “emblems,” also conveys meaning, but in contrast to co-speech gestures, they can do so directly and independent of speech. There is currently significant interest in the behavioral and biological relationships between action and language. Since co-speech gestures are actions that rely on spoken language, and emblems convey meaning to the effect that they can sometimes substitute for speech, these actions may be important, and potentially informative, examples of language–motor interactions. Researchers have recently been examining how the brain processes these actions. The current results of this work do not yet give a clear understanding of gesture processing at the neural level. For the most part, however, it seems that two complimentary sets of brain areas respond when people see gestures, reflecting their role in disambiguating meaning. These include areas thought to be important for understanding actions and areas ordinarily related to processing language. The shared and distinct responses across these two sets of areas during communication are just beginning to emerge. In this review, we talk about the ways that the brain responds when people see gestures, how these responses relate to brain activity when people process language, and how these might relate in normal, everyday communication.

The neural basis of hand gesture comprehension: A meta-analysis of functional magnetic resonance imaging studies

Gestures play an important role in face-to-face communication and have been increasingly studied via functional magnetic resonance imaging. Although a large amount of data has been provided to describe the neural substrates of gesture comprehension, these findings have never been quantitatively summarized and the conclusion is still unclear. This activation likelihood estimation meta-analysis investigated the brain networks underpinning gesture comprehension while considering the impact of gesture type (co-speech gestures vs. speech-independent gestures) and task demand (implicit vs. explicit) on the brain activation of gesture comprehension. The meta-analysis of 31 papers showed that as hand actions, gestures involve a perceptual-motor network important for action recognition. As meaningful symbols, gestures involve a semantic network for conceptual processing. Finally, during face-to-face interactions, gestures involve a network for social emotive processes. Our finding also indicated that gesture type and task demand influence the involvement of the brain networks during gesture comprehension. The results highlight the complexity of gesture comprehension, and suggest that future research is necessary to clarify the dynamic interactions among these networks.

Parallel Workflows for Data-Driven Structural Equation Modeling in Functional Neuroimaging

We present a computational framework suitable for a data-driven approach to structural equation modeling (SEM) and describe several workflows for modeling functional magnetic resonance imaging (fMRI) data within this framework. The Computational Neuroscience Applications Research Infrastructure (CNARI) employs a high-level scripting language called Swift, which is capable of spawning hundreds of thousands of simultaneous R processes (R Development Core Team, 2008), consisting of self-contained SEMs, on a high performance computing system (HPC). These self-contained R processing jobs are data objects generated by OpenMx, a plug-in for R, which can generate a single model object containing the matrices and algebraic information necessary to estimate parameters of the model. With such an infrastructure in place a structural modeler may begin to investigate exhaustive searches of the model space. Specific applications of the infrastructure, statistics related to model fit, and limitations are discussed in relation to exhaustive SEM. In particular, we discuss how workflow management techniques can help to solve large computational problems in neuroimaging.

Congenital blindness is associated with large-scale reorganization of anatomical networks

Blindness is a unique model for understanding the role of experience in the development of the brains functional and anatomical architecture. Documenting changes in the structure of anatomical networks for this population would substantiate the notion that the brains core network-level organization may undergo neuroplasticity as a result of life-long experience. To examine this issue, we compared whole-brain networks of regional cortical-thickness covariance in early blind and matched sighted individuals. This covariance is thought to reflect signatures of integration between systems involved in similar perceptual/cognitive functions. Using graph-theoretic metrics, we identified a unique mode of anatomical reorganization in the blind that differed from that found for sighted. This was seen in that network partition structures derived from subgroups of blind were more similar to each other than they were to partitions derived from sighted. Notably, after deriving network partitions, we found that language and visual regions tended to reside within separate modules in sighted but showed a pattern of merging into shared modules in the blind. Our study demonstrates that early visual deprivation triggers a systematic large-scale reorganization of whole-brain cortical-thickness networks, suggesting changes in how occipital regions interface with other functional networks in the congenitally blind.

Database-managed grid-enabled analysis of neuroimaging data: the CNARI framework.

Functional magnetic resonance imaging (fMRI) has led to an enormous growth in the study of cognitive neuroanatomy, and combined with advances in high-field electrophysiology (and other methods), has led to a fast-growing field of human neuroscience. Technological advances in both hardware and software will lead to an ever more promising future for fMRI. We have developed a new computational framework that facilitates fMRI experimentation and analysis, and which has led to some rethinking of the nature of experimental design and analysis. The Computational Neuroscience Applications Research Infrastructure (CNARI) incorporates novel methods for maintaining, serving, and analyzing massive amounts of fMRI data. By using CNARI, it is possible to perform naturalistic, network-based, statistically valid experiments in systems neuroscience on a very large scale, with ease of data manipulation and analysis, within reasonable computational time scales. In this article, we describe this infrastructure and then illustrate its use on a number of actual examples in both cognitive neuroscience and neurological research. We believe that these advanced computational approaches will fundamentally change the future shape of cognitive brain imaging with fMRI.

Classification of inter-subject fMRI data based on graph kernels

The analysis of human brain connectivity networks has become an increasingly prevalent task in neuroimaging. A few recent studies have shown the possibility of decoding brain states based on brain graph classification. Graph kernels have emerged as a powerful tool for graph comparison that allows the direct use of machine learning classifiers on brain graph collections. They allow classifying graphs with different number of nodes and therefore the inter-subject analysis without any kind of previous alignment of individual subjects data. Using whole-brain fMRI data, in this paper we present a method based on graph kernels that provides above-chance accuracy results for the inter-subject discrimination of two different types of auditory stimuli. We focus our research on determining whether this method is sensitive to the relational information in the data. Indeed, we show that the discriminative information is not only coming from topological features of the graphs like node degree distribution, but also from more complex relational patterns in the neighborhood of each node. Moreover, we investigate the suitability of two different graph representation methods, both based on data-driven parcellation techniques. Finally, we study the influence of noisy connections in our graphs and provide a way to alleviate this problem.

Repeated movie viewings produce similar local activity patterns but different network configurations

People seek novelty in everyday life, but they also enjoy viewing the same movies or reading the same novels a second time. What changes and what stays the same when re-experiencing a narrative? In examining this question with functional neuroimaging, we found that brain activity reorganizes in a hybrid, scale-dependent manner when individuals processed the same audiovisual narrative a second time. At the most local level, sensory systems (occipital and temporal cortices) maintained a similar temporal activation profile during the two viewings. Nonetheless, functional connectivity between these same lateral temporal regions and other brain regions was stronger during the second viewing. Furthermore, at the level of whole-brain connectivity, we found a significant rearrangement of network partition structure: lateral temporal and inferior frontal regions clustered together during the first viewing but merged within a fronto-parietal cluster in the second. Our findings show that repetition maintains local activity profiles. However, at the same time, it is associated with multiple network-level connectivity changes on larger scales, with these changes strongly involving regions considered core to language processing.

fMRI methods for studying the neurobiology of language under naturalistic conditions

People ordinarily use language in complex, continuously occurring contexts. These contexts include rich and varied sources of information that can combine and exert shared influence. A common example is face-to-face conversation. When two people talk in person, there is not only spoken auditory information, but also visual information from facial and manual movements, as well as from the surrounding environment in which the conversation takes place. There may also be endogenous signals that a person experiences in context, such as memories relating prior conversations with a given speaker. In short, it is typical that a person mediates multiple multifaceted information sources when using language. By contrast, fMRI studies of the neurobiology of language often use conditions that present only features of language in isolation. In large part, this is because researchers need rigorous, reliable experimental protocols that minimize potential sources of variance (“noise”) not directly relating a feature of interest. But such traditional protocols also often minimize, if not eliminate, the way people actually know and use language in their naturalistic, “everyday” experience. Thus, a fundamental challenge for researchers studying the neurobiology of language is to understand brain function as it might occur in typical experience. In this chapter, we highlight some available approaches that can address this challenge using fMRI. With specific examples, we discuss the importance of context and ways to incorporate it in understanding brain function when people process language under more naturalistic conditions.

Visual cortex signals a mismatch between regularity of auditory and visual streams

&NA; Understanding how humans code for and respond to environmental uncertainty/regularity is a question shared by current computational and neurobiological approaches to human cognition. To date, studies investigating neurobiological systems that track input uncertainty have examined responses to uni‐sensory streams. It is not known, however, whether there exist brain systems that combine information about the regularity of input streams presented to different senses. We report an fMRI study that aimed to identify brain systems that relate statistical information across sensory modalities. We constructed temporally extended auditory and visual streams, each of which could be random or highly regular, and presented them concurrently. We found strong signatures of “regularity matching” in visual cortex bilaterally; responses were higher when the level of regularity in the auditory and visual streams mismatched than when it matched, [(AudHigh/VisLow and AudLow/VisHigh) >(AudLow/VisLow and AudHigh/VisHigh)]. In addition, several frontal and parietal regions tracked regularity of the auditory or visual stream independently of the other streams regularity. An individual‐differences analysis suggested that signatures of single‐modality‐focused regularity tracking in these fronto‐parietal regions are inversely related to signatures of regularity‐matching in visual cortex. Our findings suggest that i) visual cortex is a junction for integration of temporally‐extended auditory and visual inputs and that ii) multisensory regularity‐matching depends on balanced processing of both input modalities. We discuss the implications of these findings for neurobiological models of uncertainty and for understanding computations that underlie multisensory interactions in occipital cortex. HighlightsRegularity of auditory and visual channels was manipulated independently within a multisensory stream.Regions in visual cortex showed higher activity when regularity of auditory and visual streams mismatched.Thalamic and parietal regions show mismatch signatures, but only during the early stage of series presentation.The study demonstrates integration of statistical features, across different modalities, in occipital cortex.