Dirk Ostwald

Flexible coding for categorical decisions in the human brain.

Despite the importance of visual categorization for interpreting sensory experiences, little is known about the neural representations that mediate categorical decisions in the human brain. Here, we used psychophysics and pattern classification for the analysis of functional magnetic resonance imaging data to predict the features critical for categorical decisions from brain activity when observers categorized the same stimuli using different rules. Although a large network of cortical and subcortical areas contain information about visual categories, we show that only a subset of these areas shape their selectivity to reflect the behaviorally relevant features rather than simply physical similarity between stimuli. Specifically, temporal and parietal areas show selectivity for the perceived form and motion similarity, respectively. In contrast, frontal areas and the striatum represent the conjunction of spatiotemporal features critical for complex and adaptive categorization tasks and potentially modulate selectivity in temporal and parietal areas. These findings provide novel evidence for flexible neural coding in the human brain that translates sensory experiences to categorical decisions by shaping neural representations across a network of areas with dissociable functional roles in visual categorization.

Perceptual Decisions Formed by Accumulation of Audiovisual Evidence in Prefrontal Cortex

To form perceptual decisions in our multisensory environment, the brain needs to integrate sensory information derived from a common source and segregate information emanating from different sources. Combining fMRI and psychophysics in humans, we investigated how the brain accumulates sensory evidence about a visual source in the context of congruent or conflicting auditory information. In a visual selective attention paradigm, subjects (12 females, 7 males) categorized video clips while ignoring concurrent congruent or incongruent soundtracks. Visual and auditory information were reliable or unreliable. Our behavioral data accorded with accumulator models of perceptual decision making, where sensory information is integrated over time until a criterion amount of information is obtained. Behaviorally, subjects exhibited audiovisual incongruency effects that increased with the variance of the visual and the reliability of the interfering auditory input. At the neural level, only the left inferior frontal sulcus (IFS) showed an “audiovisual-accumulator” profile consistent with the observed reaction time pattern. By contrast, responses in the right fusiform were amplified by incongruent auditory input regardless of sensory reliability. Dynamic causal modeling showed that these incongruency effects were mediated via connections from auditory cortex. Further, while the fusiform interacted with IFS in an excitatory recurrent loop that was strengthened for unreliable task-relevant visual input, the IFS did not amplify and even inhibited superior temporal activations for unreliable auditory input. To form decisions that guide behavioral responses, the IFS may accumulate audiovisual evidence by dynamically weighting its connectivity to auditory and visual regions according to sensory reliability and decisional relevance.

Neural coding of global form in the human visual cortex.

Extensive psychophysical and computational work proposes that the perception of coherent and meaningful structures in natural images relies on neural processes that convert information about local edges in primary visual cortex to complex object features represented in the temporal cortex. However, the neural basis of these mid-level vision mechanisms in the human brain remains largely unknown. Here, we examine functional MRI (fMRI) selectivity for global forms in the human visual pathways using sensitive multivariate analysis methods that take advantage of information across brain activation patterns. We use Glass patterns, parametrically varying the perceived global form (concentric, radial, translational) while ensuring that the local statistics remain similar. Our findings show a continuum of integration processes that convert selectivity for local signals (orientation, position) in early visual areas to selectivity for global form structure in higher occipitotemporal areas. Interestingly, higher occipitotemporal areas discern differences in global form structure rather than low-level stimulus properties with higher accuracy than early visual areas while relying on information from smaller but more selective neural populations (smaller voxel pattern size), consistent with global pooling mechanisms of local orientation signals. These findings suggest that the human visual system uses a code of increasing efficiency across stages of analysis that is critical for the successful detection and recognition of objects in complex environments.

An information theoretic approach to EEG-fMRI integration of visually evoked responses

The integration of signals from electro-encephalography (EEG) and functional magnetic resonance imaging (fMRI), acquired simultaneously from the same observer, holds great potential for the elucidation of the neurobiological underpinnings of human brain function. However, the most appropriate way in which to combine the data in order to achieve this goal is not clear. Here, we apply a novel route to the integration of simultaneously acquired multimodal brain imaging data. We adopt a theoretical framework developed in the study of neuronal population codes which explicitly takes into account the experimentally observed stimulus-response signal probability distributions using the concept of mutual information. We study the implications of this framework using simulated data sets generated from a set of linear Gaussian models, and apply the framework to EEG-fMRI data acquired during checkerboard stimulation of low and high contrast. We focus our evaluation on single-trial time-domain signal features from both modalities and provide evidence for the informativeness of a subset of these features with respect to the stimulus and each other. Specifically, the framework was able to identify the contrast dependency of the haemodynamic response and the P100 peak of the visual evoked potential, and showed that combining EEG and fMRI time-domain features by quantifying the information in their joint distribution was more informative than treating each one in isolation. In addition, the effect of different pre-processing strategies for EEG-fMRI data can be assessed quantitatively, indicating the improvements to be gained by more advanced methods. We conclude that the information theoretic framework is a promising methodology to quantify the relative importance of different response features in neural coding and neurovascular coupling, as well as the success of data pre-processing strategies.

Evidence for neural encoding of Bayesian surprise in human somatosensation

Accumulating empirical evidence suggests a role of Bayesian inference and learning for shaping neural responses in auditory and visual perception. However, its relevance for somatosensory processing is unclear. In the present study we test the hypothesis that cortical somatosensory processing exhibits dynamics that are consistent with Bayesian accounts of brain function. Specifically, we investigate the cortical encoding of Bayesian surprise, a recently proposed marker of Bayesian perceptual learning, using EEG data recorded from 15 subjects. Capitalizing on a somatosensory mismatch roving paradigm, we performed computational single-trial modeling of evoked somatosensory potentials for the entire peri-stimulus time period in source space. By means of Bayesian model selection, we find that, at 140 ms post-stimulus onset, secondary somatosensory cortex represents Bayesian surprise rather than stimulus change, which is the conventional marker of EEG mismatch responses. In contrast, at 250 ms, right inferior frontal cortex indexes stimulus change. Finally, at 360 ms, our analyses indicate additional perceptual learning attributable to medial cingulate cortex. In summary, the present study provides novel evidence for anatomical-temporal/functional segregation in human somatosensory processing that is consistent with the Bayesian brain hypothesis.

Multimodal Functional Network Connectivity: An EEG-fMRI Fusion in Network Space

EEG and fMRI recordings measure the functional activity of multiple coherent networks distributed in the cerebral cortex. Identifying network interaction from the complementary neuroelectric and hemodynamic signals may help to explain the complex relationships between different brain regions. In this paper, multimodal functional network connectivity (mFNC) is proposed for the fusion of EEG and fMRI in network space. First, functional networks (FNs) are extracted using spatial independent component analysis (ICA) in each modality separately. Then the interactions among FNs in each modality are explored by Granger causality analysis (GCA). Finally, fMRI FNs are matched to EEG FNs in the spatial domain using network-based source imaging (NESOI). Investigations of both synthetic and real data demonstrate that mFNC has the potential to reveal the underlying neural networks of each modality separately and in their combination. With mFNC, comprehensive relationships among FNs might be unveiled for the deep exploration of neural activities and metabolic responses in a specific task or neurological state.

Functional source separation improves the quality of single trial visual evoked potentials recorded during concurrent EEG-fMRI.

EEG quality is a crucial issue when acquiring combined EEG-fMRI data, particularly when the focus is on using single trial (ST) variability to integrate the data sets. The most common method for improving EEG data quality following removal of gross MRI artefacts is independent component analysis (ICA), a completely blind source separation technique. In the current study, a different approach is proposed based on the functional source separation (FSS) algorithm. FSS is an extension of ICA that incorporates prior knowledge about the signal of interest into the data decomposition. Since in general the part of the EEG signal that will contain the most relevant information is known beforehand (i.e. evoked potential peaks, spectral bands), FSS separates the signal of interest by exploiting this prior knowledge without renouncing the advantages of using only information contained in the original signal waveforms. A reversing checkerboard stimulus was used to generate visual evoked potentials (VEPs) in healthy control subjects. Gradient and ballistocardiogram artefacts were removed with template subtraction techniques to form the raw data, which were then subjected to ICA denoising and FSS. The resulting EEG data sets were compared using several metrics derived from average and ST data and correlated with fMRI data. In all cases, ICA was an improvement on the raw data, but the most obvious improvement was provided by FSS, which consistently outperformed ICA. The results show the benefit of FSS for the recovery of good quality single trial evoked potentials during concurrent EEG-fMRI recordings.

The relationship between the visual evoked potential and the gamma band investigated by blind and semi-blind methods

Gamma Band Activity (GBA) is increasingly studied for its relation with attention, change detection, maintenance of working memory and the processing of sensory stimuli. Activity around the gamma range has also been linked with early visual processing, although the relationship between this activity and the low frequency visual evoked potential (VEP) remains unclear. This study examined the ability of blind and semi-blind source separation techniques to extract sources specifically related to the VEP and GBA in order to shed light on the relationship between them. Blind (Independent Component Analysis—ICA) and semi-Blind (Functional Source Separation—FSS) methods were applied to dense array EEG data recorded during checkerboard stimulation. FSS was performed with both temporal and spectral constraints to identify specifically the generators of the main peak of the VEP (P100) and of the GBA. Source localisation and time-frequency analyses were then used to investigate the properties and co-dependencies between VEP/P100 and GBA. Analysis of the VEP extracted using the different methods demonstrated very similar morphology and localisation of the generators. Single trial time frequency analysis showed higher GBA when a larger amplitude VEP/P100 occurred. Further examination indicated that the evoked (phase-locked) component of the GBA was more related to the P100, whilst the induced component correlated with the VEP as a whole. The results suggest that the VEP and GBA may be generated by the same neuronal populations, and implicate this relationship as a potential mediator of the correlation between the VEP and the Blood Oxygenation Level Dependent (BOLD) effect measured with fMRI.

Information theoretic approaches to functional neuroimaging

Information theory is a probabilistic framework that allows the quantification of statistical non-independence between signals of interest. In contrast to other methods used for this purpose, it is model free, i.e., it makes no assumption about the functional form of the statistical dependence or the underlying probability distributions. It thus has the potential to unveil important signal characteristics overlooked by classical data analysis techniques. In this review, we discuss how information theoretic concepts have been applied to the analysis of functional brain imaging data such as functional magnetic resonance imaging and magneto/electroencephalography. We review studies from a number of imaging domains, including the investigation of the brains functional specialization and integration, neurovascular coupling and multimodal imaging. We demonstrate how information theoretical concepts can be used to answer neurobiological questions and discuss their limitations as well as possible future developments of the framework to advance our understanding of brain function.

Cortical processing of near-threshold tactile stimuli: An MEG study

In the present study we tested the applicability of a paired-stimulus paradigm for the investigation of near-threshold (NT) stimulus processing in the somatosensory system using magnetoencephalography. Cortical processing of the NT stimuli was studied indirectly by investigating the impact of NT stimuli on the source activity of succeeding suprathreshold test stimuli. We hypothesized that cortical responses evoked by test stimuli are reduced due to the preactivation of the same finger representation by the preceding NT stimulus. We observed attenuation of the magnetic responses in the secondary somatosensory (SII) cortex, with stronger decreases for perceived than for missed NT stimuli. Our data suggest that processing in the primary somatosensory cortex including recovery lasts for <200 ms. Conversely, the occupancy of SII lasts >/=500 ms, which points to its role in temporal integration and conscious perception of sensory input.