Radoslaw Martin Cichy

Resolving human object recognition in space and time

A comprehensive picture of object processing in the human brain requires combining both spatial and temporal information about brain activity. Here we acquired human magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) responses to 92 object images. Multivariate pattern classification applied to MEG revealed the time course of object processing: whereas individual images were discriminated by visual representations early, ordinate and superordinate category levels emerged relatively late. Using representational similarity analysis, we combined human fMRI and MEG to show content-specific correspondence between early MEG responses and primary visual cortex (V1), and later MEG responses and inferior temporal (IT) cortex. We identified transient and persistent neural activities during object processing with sources in V1 and IT. Finally, we correlated human MEG signals to single-unit responses in monkey IT. Together, our findings provide an integrated space- and time-resolved view of human object categorization during the first few hundred milliseconds of vision.

Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence

The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans. However, the stage-wise computations therein remain poorly understood. Here, we compared temporal (magnetoencephalography) and spatial (functional MRI) visual brain representations with representations in an artificial deep neural network (DNN) tuned to the statistics of real-world visual recognition. We showed that the DNN captured the stages of human visual processing in both time and space from early visual areas towards the dorsal and ventral streams. Further investigation of crucial DNN parameters revealed that while model architecture was important, training on real-world categorization was necessary to enforce spatio-temporal hierarchical relationships with the brain. Together our results provide an algorithmically informed view on the spatio-temporal dynamics of visual object recognition in the human visual brain.

Encoding the identity and location of objects in human LOC

We are able to recognize objects independent of their location in the visual field. At the same time, we also keep track of the location of objects to orient ourselves and to interact with the environment. The lateral occipital complex (LOC) has been suggested as the prime cortical region for representation of object identity. However, the extent to which LOC also represents object location has remained debated. In this study we used high-resolution fMRI in combination with multivoxel pattern classification to investigate the cortical encoding of three object exemplars from four different categories presented in two different locations. This approach allowed us to study location-tolerant object information and object-tolerant location information in LOC, both at the level of categories and exemplars. We found evidence for both location-tolerant object information and object-tolerant location information in LOC at the level of categories and exemplars. Our results further highlight the mixing of identity and location information in the ventral visual pathway.

An initial transient-state and reliable measures of corticospinal excitability in TMS studies

OBJECTIVE The objective of this study was to determine if an initial transient state influences the acquisition of reliable estimates of corticospinal excitability in transcranial magnetic stimulation (TMS) studies. Whereas muscle evoked potential (MEP) amplitudes are an important index of cortical excitability, these are severely limited by sweep-to-sweep variability. Interesting in this context is the experimental observation that the first MEP amplitudes might be much larger than subsequent responses [Brasil-Neto JP, Cohen LG, Hallet M. Central fatigue as revealed by postexercise decrement of motor evoked potentials. Muscle Nerve 1994;17:713-9]. This led to the hypothesis that an initial transient-state of increased excitability affects MEP amplitude derived estimates of corticospinal excitability. METHODS To address this issue we acquired repeated measures of single pulse MEP amplitudes over the primary motor cortex with and without navigated brain stimulation (NBS) and with various TMS-coils. Importantly, NBS allows for the sweep-to-sweep differentiation of physical and physiological variability. RESULTS We found a significant decline in estimates of corticospinal excitability and a transition from log-Normal to Normal distributed state, after which reliable measures (British Standards Institute) could be acquired. CONCLUSIONS We argue that an initial transient state of physiological origin influences measures of corticospinal excitability. SIGNIFICANCE This has important implications for investigations of cortical excitability. For example, it could reduce variability over studies and within small group comparisons.

Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

Abstract Human scene recognition is a rapid multistep process evolving over time from single scene image to spatial layout processing. We used multivariate pattern analyses on magnetoencephalography (MEG) data to unravel the time course of this cortical process. Following an early signal for lower‐level visual analysis of single scenes at ˜100 ms, we found a marker of real‐world scene size, i.e. spatial layout processing, at ˜250 ms indexing neural representations robust to changes in unrelated scene properties and viewing conditions. For a quantitative model of how scene size representations may arise in the brain, we compared MEG data to a deep neural network model trained on scene classification. Representations of scene size emerged intrinsically in the model, and resolved emerging neural scene size representation. Together our data provide a first description of an electrophysiological signal for layout processing in humans, and suggest that deep neural networks are a promising framework to investigate how spatial layout representations emerge in the human brain.

Similarity-Based Fusion of MEG and fMRI Reveals Spatio-Temporal Dynamics in Human Cortex During Visual Object Recognition

Every human cognitive function, such as visual object recognition, is realized in a complex spatio-temporal activity pattern in the brain. Current brain imaging techniques in isolation cannot resolve the brains spatio-temporal dynamics, because they provide either high spatial or temporal resolution but not both. To overcome this limitation, we developed an integration approach that uses representational similarities to combine measurements of magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) to yield a spatially and temporally integrated characterization of neuronal activation. Applying this approach to 2 independent MEG–fMRI data sets, we observed that neural activity first emerged in the occipital pole at 50–80 ms, before spreading rapidly and progressively in the anterior direction along the ventral and dorsal visual streams. Further region-of-interest analyses established that dorsal and ventral regions showed MEG–fMRI correspondence in representations later than early visual cortex. Together, these results provide a novel and comprehensive, spatio-temporally resolved view of the rapid neural dynamics during the first few hundred milliseconds of object vision. They further demonstrate the feasibility of spatially unbiased representational similarity-based fusion of MEG and fMRI, promising new insights into how the brain computes complex cognitive functions.

Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans

It is a principal open question whether noninvasive imaging methods in humans can decode information encoded at a spatial scale as fine as the basic functional unit of cortex: cortical columns. We addressed this question in five magnetoencephalography (MEG) experiments by investigating a columnar-level encoded visual feature: contrast edge orientation. We found that MEG signals contained orientation-specific information as early as approximately 50 ms after stimulus onset even when controlling for confounds, such as overrepresentation of particular orientations, stimulus edge interactions, and global form-related signals. Theoretical modeling confirmed the plausibility of this empirical result. An essential consequence of our results is that information encoded in the human brain at the level of cortical columns should in general be accessible by multivariate analysis of electrophysiological signals.

Probing principles of large-scale object representation: Category preference and location encoding

Knowledge about the principles that govern large‐scale neural representations of objects is central to a systematic understanding of object recognition. We used functional magnetic resonance imaging (fMRI) and multivariate pattern classification to investigate two such candidate principles: category preference and location encoding. The former designates the preferential activation of distinct cortical regions by a specific category of objects. The latter refers to information about where in the visual field a particular object is located. Participants viewed exemplars of three object categories (faces, bodies, and scenes) that were presented left or right of fixation. The analysis of fMRI activation patterns revealed the following. Category‐selective regions retained their preference to the same categories in a manner tolerant to changes in object location. However, category preference was not absolute: category‐selective regions also contained location‐tolerant information about nonpreferred categories. Furthermore, location information was present throughout high‐level ventral visual cortex and was distributed systematically across the cortical surface. We found more location information in lateral‐occipital cortex than in ventral‐temporal cortex. Our results provide a systematic account of the extent to which the principles of category preference and location encoding determine the representation of objects in the high‐level ventral visual cortex. Hum Brain Mapp, 2013.

The Neural Code for Face Orientation in the Human Fusiform Face Area

Humans recognize faces and objects with high speed and accuracy regardless of their orientation. Recent studies have proposed that orientation invariance in face recognition involves an intermediate representation where neural responses are similar for mirror-symmetric views. Here, we used fMRI, multivariate pattern analysis, and computational modeling to investigate the neural encoding of faces and vehicles at different rotational angles. Corroborating previous studies, we demonstrate a representation of face orientation in the fusiform face-selective area (FFA). We go beyond these studies by showing that this representation is category-selective and tolerant to retinal translation. Critically, by controlling for low-level confounds, we found the representation of orientation in FFA to be compatible with a linear angle code. Aspects of mirror-symmetric coding cannot be ruled out when FFA mean activity levels are considered as a dimension of coding. Finally, we used a parametric family of computational models, involving a biased sampling of view-tuned neuronal clusters, to compare different face angle encoding models. The best fitting model exhibited a predominance of neuronal clusters tuned to frontal views of faces. In sum, our findings suggest a category-selective and monotonic code of face orientation in the human FFA, in line with primate electrophysiology studies that observed mirror-symmetric tuning of neural responses at higher stages of the visual system, beyond the putative homolog of human FFA.

Transcranial cortex stimulation and fMRI: Electrophysiological correlates of dual-pulse BOLD signal modulation

UNLABELLED Are the local hemodynamic changes in BOLD-fMRI correlated to increased or decreased neuronal activity or both? We combined transcranial electrical cortex stimulation (TES) with simultaneous fMRI and electromyographic (EMG) recording to study the influence of inhibitory and excitatory neuronal activity on the concomitant BOLD signal change. Unilateral or bilateral TES was applied with a postero-anterior orientation. This activates pyramidal cells transsynaptically and allows for the induction of cortical inhibition and excitation of the pyramidal cell, respectively. In this project interhemispheric inhibition (IHI) served as an in vivo model to investigate electrophysiologically well defined inhibitory and excitatory effects. METHODOLOGY Included event-related fMRI, which triggered TES; online recording of the EMG response monitored the inhibitory and excitatory influences on discharging corticospinal neurons. RESULTS Revealed that a single suprathreshold stimulus induced a positive BOLD response both in the ipsilateral as well as in the contralateral primary motor cortex (M1). The contralateral co-activation of the homotopic M1 should be a functional correlate of transcallosal connections. If a contralateral conditioning stimulus preceded the test stimulus by 10 ms (IHI), the subsequent ipsilateral BOLD signal was significantly reduced. We find that cortical inhibitory processes are accompanied by attenuation of the local neurovascular signal.