Pieter R. Roelfsema

The distinct modes of vision offered by feedforward and recurrent processing

An analysis of response latencies shows that when an image is presented to the visual system, neuronal activity is rapidly routed to a large number of visual areas. However, the activity of cortical neurons is not determined by this feedforward sweep alone. Horizontal connections within areas, and higher areas providing feedback, result in dynamic changes in tuning. The differences between feedforward and recurrent processing could prove pivotal in understanding the distinctions between attentive and pre-attentive vision as well as between conscious and unconscious vision. The feedforward sweep rapidly groups feature constellations that are hardwired in the visual brain, yet is probably incapable of yielding visual awareness; in many cases, recurrent processing is necessary before the features of an object are attentively grouped and the stimulus can enter consciousness.

Object-based attention in the primary visual cortex of the macaque monkey.

Typical natural visual scenes contain many objects, which need to be segregated from each other and from the background. Present theories subdivide the processes responsible for this segregation into a pre-attentive and attentive system,. The pre-attentive system segregates image regions that ‘pop out’ rapidly and in parallel across the visual field. In the primary visual cortex, responses to pre-attentively selected image regions are enhanced. When objects do not segregate automatically from the rest of the image, the time-consuming attentive system is recruited. Here we investigate whether attentive selection is also associated with a modulation of firing rates in area V1 of the brainin monkeys trained to perform a curve-tracing task,. Neuronal responses to the various segments of a target curve were simultaneously enhanced relative to responses evoked by a distractor curve, even if the two curves crossed each other. This indicates that object-based attention is associated with a response enhancement at the earliest level of the visual cortical processing hierarchy.

Role of reticular activation in the modulation of intracortical synchronization

During aroused states of the brain, electroencephalographic activity is characterized by fast, irregular fluctuations of low amplitude, which are thought to reflect desynchronization of neuronal activity. This phenomenon seems at odds with the proposal that synchronization of cortical responses may play an important role in the processing of sensory signals. Here, activation of the mesencephalic reticular formation (MRF), an effective way to “desynchronize the electroencephalogram,” was shown to facilitate oscillatory activity in the gamma frequency range and to enhance the stimulus-specific synchronization of neuronal spike responses in the visual cortex of cats.

Alpha and gamma oscillations characterize feedback and feedforward processing in monkey visual cortex

Significance One of the main unresolved questions in cognitive neuroscience is how low-level and high-level areas of the visual cortex interact with each other during perception and cognition. We investigated whether cortical oscillations can be used to distinguish feedforward from feedback processing. We studied the propagation of α- and γ-oscillations through the cortical layers and between different visual cortical areas. We induced oscillations in different areas with microstimulation and influenced them using a pharmacological approach. The results of these experiments demonstrate that γ-oscillations propagate in the feedforward direction, whereas α-oscillations propagate in the feedback direction. We conclude that high- and low-frequency oscillations provide markers of feedforward and feedback processing, respectively. Cognitive functions rely on the coordinated activity of neurons in many brain regions, but the interactions between cortical areas are not yet well understood. Here we investigated whether low-frequency (α) and high-frequency (γ) oscillations characterize different directions of information flow in monkey visual cortex. We recorded from all layers of the primary visual cortex (V1) and found that γ-waves are initiated in input layer 4 and propagate to the deep and superficial layers of cortex, whereas α-waves propagate in the opposite direction. Simultaneous recordings from V1 and downstream area V4 confirmed that γ- and α-waves propagate in the feedforward and feedback direction, respectively. Microstimulation in V1 elicited γ-oscillations in V4, whereas microstimulation in V4 elicited α-oscillations in V1, thus providing causal evidence for the opposite propagation of these rhythms. Furthermore, blocking NMDA receptors, thought to be involved in feedback processing, suppressed α while boosting γ. These results provide new insights into the relation between brain rhythms and cognition.

Bottom-Up Dependent Gating of Frontal Signals in Early Visual Cortex

The frontal eye field (FEF) is one of several cortical regions thought to modulate sensory inputs. Moreover, several hypotheses suggest that the FEF can only modulate early visual areas in the presence of a visual stimulus. To test for bottom-up gating of frontal signals, we microstimulated subregions in the FEF of two monkeys and measured the effects throughout the brain with functional magnetic resonance imaging. The activity of higher-order visual areas was strongly modulated by FEF stimulation, independent of visual stimulation. In contrast, FEF stimulation induced a topographically specific pattern of enhancement and suppression in early visual areas, but only in the presence of a visual stimulus. Modulation strength depended on stimulus contrast and on the presence of distractors. We conclude that bottom-up activation is needed to enable top-down modulation of early visual cortex and that stimulus saliency determines the strength of this modulation.

Neuronal assemblies: necessity, signature and detectability.

The ease with which highly developed brains can generate representations of a virtually unlimited diversity of perceptual objects indicates that they have developed very efficient mechanisms to analyse and represent relations among incoming signals. Here, we propose that two complementary strategies are applied to cope with these combinatorial problems. First, elementary relations are represented by the tuned responses of individual neurons that acquire their specific response properties (feature selectivity) through appropriate convergence of input connections in hierarchically structured feed-forward architectures. Second, complex relations that cannot be represented economically by the responses of individual neurons are represented by assemblies of cells that are generated by dynamic association of individual, featureselective cells. The signature identifying the responses of an assembly as components of a coherent code is thought to be the synchronicity of the respective discharges. The compatibility of this hypothesis is examined in the context of recent data on the dynamics of synchronization phenomena, the dependence of synchronization on central states and the relations between the synchronization behaviour of neurons and perception.

Figure–Ground Segregation in a Recurrent Network Architecture

Here we propose a model of how the visual brain segregates textured scenes into figures and background. During texture segregation, locations where the properties of texture elements change abruptly are assigned to boundaries, whereas image regions that are relatively homogeneous are grouped together. Boundary detection and grouping of image regions require different connection schemes, which are accommodated in a single network architecture by implementing them in different layers. As a result, all units carry signals related to boundary detection as well as grouping of image regions, in accordance with cortical physiology. Boundaries yield an early enhancement of network responses, but at a later point, an entire figural region is grouped together, because units that respond to it are labeled with enhanced activity. The model predicts which image regions are preferentially perceived as figure or as background and reproduces the spatio-temporal profile of neuronal activity in the visual cortex during texture segregation in intact animals, as well as in animals with cortical lesions.

Perceptual learning rules based on reinforcers and attention.

How does the brain learn those visual features that are relevant for behavior? In this article, we focus on two factors that guide plasticity of visual representations. First, reinforcers cause the global release of diffusive neuromodulatory signals that gate plasticity. Second, attentional feedback signals highlight the chain of neurons between sensory and motor cortex responsible for the selected action. We here propose that the attentional feedback signals guide learning by suppressing plasticity of irrelevant features while permitting the learning of relevant ones. By hypothesizing that sensory signals that are too weak to be perceived can escape from this inhibitory feedback, we bring attentional learning theories and theories that emphasized the importance of neuromodulatory signals into a single, unified framework.

Reduced Synchronization in the Visual Cortex of Cats with Strabismic Amblyopia

Synchronous firing of spatially separate neurons was studied with multi‐electrode recordings in area 17 of the visual cortex of strabismic cats which had developed behaviourally verified amblyopia of the deviated eye. Responses of neurons were evoked with moving light bars or gratings of different spatial frequency. Neurons driven by the normal eye displayed stronger synchronization of their responses than neurons dominated by the amblyopic eye. These interocular differences were highly significant and particularly pronounced for grating stimuli of high spatial frequency. No interocular differences were noted with respect to the amplitudes of responses to the light bars and gratings. These results suggest reduced synchronization of population responses as a neurophysiological correlate of strabismic amblyopia and underline the importance of correlated firing of spatially separate cortical neurons for normal processing of visual information.

Attention-Gated Reinforcement Learning of Internal Representations for Classification

Animal learning is associated with changes in the efficacy of connections between neurons. The rules that govern this plasticity can be tested in neural networks. Rules that train neural networks to map stimuli onto outputs are given by supervised learning and reinforcement learning theories. Supervised learning is efficient but biologically implausible. In contrast, reinforcement learning is biologically plausible but comparatively inefficient. It lacks a mechanism that can identify units at early processing levels that play a decisive role in the stimulus-response mapping. Here we show that this so-called credit assignment problem can be solved by a new role for attention in learning. There are two factors in our new learning scheme that determine synaptic plasticity: (1) a reinforcement signal that is homogeneous across the network and depends on the amount of reward obtained after a trial, and (2) an attentional feedback signal from the output layer that limits plasticity to those units at earlier processing levels that are crucial for the stimulus-response mapping. The new scheme is called attention-gated reinforcement learning (AGREL). We show that it is as efficient as supervised learning in classification tasks. AGREL is biologically realistic and integrates the role of feedback connections, attention effects, synaptic plasticity, and reinforcement learning signals into a coherent framework.