M.A.J. van Gerven

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Converging evidence suggests that the primate ventral visual pathway encodes increasingly complex stimulus features in downstream areas. We quantitatively show that there indeed exists an explicit gradient for feature complexity in the ventral pathway of the human brain. This was achieved by mapping thousands of stimulus features of increasing complexity across the cortical sheet using a deep neural network. Our approach also revealed a fine-grained functional specialization of downstream areas of the ventral stream. Furthermore, it allowed decoding of representations from human brain activity at an unsurpassed degree of accuracy, confirming the quality of the developed approach. Stimulus features that successfully explained neural responses indicate that population receptive fields were explicitly tuned for object categorization. This provides strong support for the hypothesis that object categorization is a guiding principle in the functional organization of the primate ventral stream.Converging evidence suggests that the primate ventral visual pathway encodes increasingly complex stimulus features in downstream areas. We quantitatively show that there indeed exists an explicit gradient for feature complexity in the ventral pathway of the human brain. This was achieved by mapping thousands of stimulus features of increasing complexity across the cortical sheet using a deep neural network. Our approach also revealed a fine-grained functional specialization of downstream areas of the ventral stream. Furthermore, it allowed decoding of representations from human brain activity at an unsurpassed degree of accuracy, confirming the quality of the developed approach. Stimulus features that successfully explained neural responses indicate that population receptive fields were explicitly tuned for object categorization. This provides strong support for the hypothesis that object categorization is a guiding principle in the functional organization of the primate ventral stream.

On the use of interaction error potentials for adaptive brain computer interfaces

We propose an adaptive classification method for the Brain Computer Interfaces (BCI) which uses Interaction Error Potentials (IErrPs) as a reinforcement signal and adapts the classifier parameters when an error is detected. We analyze the quality of the proposed approach in relation to the misclassification of the IErrPs. In addition we compare static versus adaptive classification performance using artificial and MEG data. We show that the proposed adaptive framework significantly improves the static classification methods.

MEG-based decoding of the spatiotemporal dynamics of visual category perception

Visual processing is a complex task which is best investigated using sensitive multivariate analysis methods that can capture representation-specific brain activity over both time and space. In this study, we applied a multivariate decoding algorithm to MEG data of subjects engaged in passive viewing of images of faces, scenes, bodies and tools. We used reconstructed source-space time courses as input to the algorithm in order to localize brain regions involved in optimal image discrimination. Applying this method to the interval of 115 to 315 ms after stimulus onset, we show a focal localization of regression coefficients in the inferior occipital, middle occipital, and lingual gyrus that drive decoding of the different perceived image categories. Classifier accuracy was highest (over 90% correctly classified trials, compared to a chance level accuracy of 50%) when dissociating the perception of faces from perception of other object categories. Furthermore, we applied this method to each single time point to extract the temporal evolution of visual perception. This allowed for the detection of differences in visual category perception as early as 85 ms after stimulus onset. Furthermore, localizing the corresponding regression coefficients of each time point allowed us to capture the spatiotemporal dynamics of visual category perception. This revealed initial involvement of sources in the inferior occipital, inferior temporal and superior occipital gyrus. During sustained stimulation additional sources in the anterior inferior temporal gyrus and superior parietal gyrus became involved. We conclude that decoding of source-space MEG data provides a suitable method to investigate the spatiotemporal dynamics of ongoing cognitive processing.

Probabilistic model-based functional parcellation reveals a robust, fine-grained subdivision of the striatum.

The striatum is involved in many different aspects of behaviour, reflected by the variety of cortical areas that provide input to this structure. This input is topographically organized and is likely to result in functionally specific signals. Such specificity can be examined using functional clustering approaches. Here, we propose a Bayesian model-based functional clustering approach applied solely to resting state striatal functional MRI timecourses to identify intrinsic striatal functional modules. Data from two sets of ten participants were used to obtain parcellations and examine their robustness. This stable clustering was used to initialize a more constrained model in order to obtain individualized parcellations in 57 additional participants. Resulting cluster time courses were used to examine functional connectivity between clusters and related to the rest of the brain in a GLM analysis. We find six distinct clusters in each hemisphere, with clear inter-hemispheric correspondence and functional relevance. These clusters exhibit functional connectivity profiles that further underscore their homologous nature and are consistent with existing notions on segregation and integration in parallel cortico-basal ganglia loops. Our findings suggest that multiple territories within both the affective and motor regions can be distinguished solely using resting state functional MRI from these regions.

Bayesian inference of structural brain networks

Structural brain networks are used to model white-matter connectivity between spatially segregated brain regions. The presence, location and orientation of these white matter tracts can be derived using diffusion-weighted magnetic resonance imaging in combination with probabilistic tractography. Unfortunately, as of yet, none of the existing approaches provide an undisputed way of inferring brain networks from the streamline distributions which tractography produces. State-of-the-art methods rely on an arbitrary threshold or, alternatively, yield weighted results that are difficult to interpret. In this paper, we provide a generative model that explicitly describes how structural brain networks lead to observed streamline distributions. This allows us to draw principled conclusions about brain networks, which we validate using simultaneously acquired resting-state functional MRI data. Inference may be further informed by means of a prior which combines connectivity estimates from multiple subjects. Based on this prior, we obtain networks that significantly improve on the conventional approach.

Vividness of visual imagery depends on the neural overlap with perception in visual areas

Research into the neural correlates of individual differences in imagery vividness point to an important role of the early visual cortex. However, there is also great fluctuation of vividness within individuals, such that only looking at differences between people necessarily obscures the picture. In this study, we show that variation in moment-to-moment experienced vividness of visual imagery, within human subjects, depends on the activity of a large network of brain areas, including frontal, parietal, and visual areas. Furthermore, using a novel multivariate analysis technique, we show that the neural overlap between imagery and perception in the entire visual system correlates with experienced imagery vividness. This shows that the neural basis of imagery vividness is much more complicated than studies of individual differences seemed to suggest. SIGNIFICANCE STATEMENT Visual imagery is the ability to visualize objects that are not in our direct line of sight: something that is important for memory, spatial reasoning, and many other tasks. It is known that the better people are at visual imagery, the better they can perform these tasks. However, the neural correlates of moment-to-moment variation in visual imagery remain unclear. In this study, we show that the more the neural response during imagery is similar to the neural response during perception, the more vivid or perception-like the imagery experience is.

Convolutional neural network-based encoding and decoding of visual object recognition in space and time

ABSTRACT Representations learned by deep convolutional neural networks (CNNs) for object recognition are a widely investigated model of the processing hierarchy in the human visual system. Using functional magnetic resonance imaging, CNN representations of visual stimuli have previously been shown to correspond to processing stages in the ventral and dorsal streams of the visual system. Whether this correspondence between models and brain signals also holds for activity acquired at high temporal resolution has been explored less exhaustively. Here, we addressed this question by combining CNN‐based encoding models with magnetoencephalography (MEG). Human participants passively viewed 1,000 images of objects while MEG signals were acquired. We modelled their high temporal resolution source‐reconstructed cortical activity with CNNs, and observed a feed‐forward sweep across the visual hierarchy between 75 and 200 ms after stimulus onset. This spatiotemporal cascade was captured by the network layer representations, where the increasingly abstract stimulus representation in the hierarchical network model was reflected in different parts of the visual cortex, following the visual ventral stream. We further validated the accuracy of our encoding model by decoding stimulus identity in a left‐out validation set of viewed objects, achieving state‐of‐the‐art decoding accuracy.

A generic qualitative characterization of independence of causal influence

Independence of causal influence (ICI) offer a high level starting point for the design of Bayesian networks. However, these models are not as widely applied as they could, as their behavior is often not well-understood. One approach is to employ qualitative probabilistic network theory in order to derive a qualitative characterization of ICI models. In this paper we analyze the qualitative properties of ICI models with binary random variables. Qualitative properties are shown to follow from the characteristics of the Boolean function underlying the model. In addition, it is demonstrated that the theory also allows finding constraints on the model parameters given knowledge of the qualitative properties. This high-level qualitative characterization offers a new way of identifying suitable ICI models and may facilitate their exploitation in developing real-world Bayesian networks.

Parkinsonian rest tremor can be detected accurately based on neuronal oscillations recorded from the subthalamic nucleus

OBJECTIVE To investigate the possibility of tremor detection based on deep brain activity. METHODS We re-analyzed recordings of local field potentials (LFPs) from the subthalamic nucleus in 10 PD patients (12 body sides) with spontaneously fluctuating rest tremor. Power in several frequency bands was estimated and used as input to Hidden Markov Models (HMMs) which classified short data segments as either tremor-free rest or rest tremor. HMMs were compared to direct threshold application to individual power features. RESULTS Applying a threshold directly to band-limited power was insufficient for tremor detection (mean area under the curve [AUC] of receiver operating characteristic: 0.64, STD: 0.19). Multi-feature HMMs, in contrast, allowed for accurate detection (mean AUC: 0.82, STD: 0.15), using four power features obtained from a single contact pair. Within-patient training yielded better accuracy than across-patient training (0.84vs. 0.78, p=0.03), yet tremor could often be detected accurately with either approach. High frequency oscillations (>200Hz) were the best performing individual feature. CONCLUSIONS LFP-based markers of tremor are robust enough to allow for accurate tremor detection in short data segments, provided that appropriate statistical models are used. SIGNIFICANCE LFP-based markers of tremor could be useful control signals for closed-loop deep brain stimulation.

Predictability of depression severity based on posterior alpha oscillations

OBJECTIVE We aimed to integrate neural data and an advanced machine learning technique to predict individual major depressive disorder (MDD) patient severity. METHODS MEG data was acquired from 22 MDD patients and 22 healthy controls (HC) resting awake with eyes closed. Individual power spectra were calculated by a Fourier transform. Sources were reconstructed via beamforming technique. Bayesian linear regression was applied to predict depression severity based on the spatial distribution of oscillatory power. RESULTS In MDD patients, decreased theta (4-8 Hz) and alpha (8-14 Hz) power was observed in fronto-central and posterior areas respectively, whereas increased beta (14-30 Hz) power was observed in fronto-central regions. In particular, posterior alpha power was negatively related to depression severity. The Bayesian linear regression model showed significant depression severity prediction performance based on the spatial distribution of both alpha (r=0.68, p=0.0005) and beta power (r=0.56, p=0.007) respectively. CONCLUSIONS Our findings point to a specific alteration of oscillatory brain activity in MDD patients during rest as characterized from MEG data in terms of spectral and spatial distribution. SIGNIFICANCE The proposed model yielded a quantitative and objective estimation for the depression severity, which in turn has a potential for diagnosis and monitoring of the recovery process.