Victor A. F. Lamme

The distinct modes of vision offered by feedforward and recurrent processing

An analysis of response latencies shows that when an image is presented to the visual system, neuronal activity is rapidly routed to a large number of visual areas. However, the activity of cortical neurons is not determined by this feedforward sweep alone. Horizontal connections within areas, and higher areas providing feedback, result in dynamic changes in tuning. The differences between feedforward and recurrent processing could prove pivotal in understanding the distinctions between attentive and pre-attentive vision as well as between conscious and unconscious vision. The feedforward sweep rapidly groups feature constellations that are hardwired in the visual brain, yet is probably incapable of yielding visual awareness; in many cases, recurrent processing is necessary before the features of an object are attentively grouped and the stimulus can enter consciousness.

Object-based attention in the primary visual cortex of the macaque monkey.

Typical natural visual scenes contain many objects, which need to be segregated from each other and from the background. Present theories subdivide the processes responsible for this segregation into a pre-attentive and attentive system,. The pre-attentive system segregates image regions that ‘pop out’ rapidly and in parallel across the visual field. In the primary visual cortex, responses to pre-attentively selected image regions are enhanced. When objects do not segregate automatically from the rest of the image, the time-consuming attentive system is recruited. Here we investigate whether attentive selection is also associated with a modulation of firing rates in area V1 of the brainin monkeys trained to perform a curve-tracing task,. Neuronal responses to the various segments of a target curve were simultaneously enhanced relative to responses evoked by a distractor curve, even if the two curves crossed each other. This indicates that object-based attention is associated with a response enhancement at the earliest level of the visual cortical processing hierarchy.

Towards a true neural stance on consciousness

Consciousness is traditionally defined in mental or psychological terms. In trying to find its neural basis, introspective or behavioral observations are considered the gold standard, to which neural measures should be fitted. I argue that this poses serious problems for understanding the mind-brain relationship. To solve these problems, neural and behavioral measures should be put on an equal footing. I illustrate this by an example from visual neuroscience, in which both neural and behavioral arguments converge towards a coherent scientific definition of visual consciousness. However, to accept this definition, we need to let go of our intuitive or psychological notions of conscious experience and let the neuroscience arguments have their way. Only by moving our notion of mind towards that of brain can progress be made.

Feedforward, horizontal, and feedback processing in the visual cortex.

The cortical visual system consists of many richly interconnected areas. Each area is characterized by more or less specific receptive field tuning properties. However, these tuning properties reflect only a subset of the interactions that occur within and between areas. Neuronal responses may be modulated by perceptual context or attention. These modulations reflect lateral interactions within areas and feedback from higher to lower areas. Recent work is beginning to unravel how horizontal and feedback connections each contribute to modulatory effects and what the role of these modulations is in vision. Whereas receptive field tuning properties reflect feedforward processing, modulations evoked by horizontal and feedback connections may reflect the integration of information that underlies perception.

Two distinct modes of sensory processing observed in monkey primary visual cortex (V1).

Even salient sensory stimuli are sometimes not detected. What goes wrong in the brain in that case? Here we show that a late (> 100-ms) component of the neural activity in the primary visual cortex of the monkey is selectively suppressed when stimuli are not seen. As there is evidence that this activity depends on feedback from extrastriate areas, these findings suggest a specific role for recurrent processing when stimuli are reaching a perceptual level. Further results show that this perceptual level is situated between purely sensory and decision or motor stages of processing.

Are There Multiple Visual Short-Term Memory Stores?

Background Classic work on visual short-term memory (VSTM) suggests that people store a limited amount of items for subsequent report. However, when human observers are cued to shift attention to one item in VSTM during retention, it seems as if there is a much larger representation, which keeps additional items in a more fragile VSTM store. Thus far, it is not clear whether the capacity of this fragile VSTM store indeed exceeds the traditional capacity limits of VSTM. The current experiments address this issue and explore the capacity, stability, and duration of fragile VSTM representations. Methodology/Principal Findings We presented cues in a change-detection task either just after off-set of the memory array (iconic-cue), 1,000 ms after off-set of the memory array (retro-cue) or after on-set of the probe array (post-cue). We observed three stages in visual information processing 1) iconic memory with unlimited capacity, 2) a four seconds lasting fragile VSTM store with a capacity that is at least a factor of two higher than 3) the robust and capacity-limited form of VSTM. Iconic memory seemed to depend on the strength of the positive after-image resulting from the memory display and was virtually absent under conditions of isoluminance or when intervening light masks were presented. This suggests that iconic memory is driven by prolonged retinal activation beyond stimulus duration. Fragile VSTM representations were not affected by light masks, but were completely overwritten by irrelevant pattern masks that spatially overlapped the memory array. Conclusions/Significance We find that immediately after a stimulus has disappeared from view, subjects can still access information from iconic memory because they can see an after-image of the display. After that period, human observers can still access a substantial, but somewhat more limited amount of information from a high-capacity, but fragile VSTM that is overwritten when new items are presented to the eyes. What is left after that is the traditional VSTM store, with a limit of about four objects. We conclude that human observers store more sustained representations than is evident from standard change detection tasks and that these representations can be accessed at will.

Masking Interrupts Figure–Ground Signals in V1

In a backward masking paradigm, a target stimulus is rapidly (<100 msec) followed by a second stimulus. This typically results in a dramatic decrease in the visibility of the target stimulus. It has been shown that masking reduces responses in V1. It is not known, however, which process in V1 is affected by the mask. In the past, we have shown that in V1, modulations of neural activity that are specifically related to figure-ground segregation can be recorded. Here, we recorded from awake macaque monkeys, engaged in a task where they had to detect figures from background in a pattern backward masking paradigm. We show that the V1 figure-ground signals are selectively and fully suppressed at target-mask intervals that psychophysically result in the target being invisible. Initial response transients, signalling the features that make up the scene, are not affected. As figure-ground modulations depend on feedback from extrastriate areas, these results suggest that masking selectively interrupts the recurrent interactions between V1 and higher visual areas.

Masking Disrupts Reentrant Processing in Human Visual Cortex

In masking, a stimulus is rendered invisible through the presentation of a second stimulus shortly after the first. Over the years, authors have typically explained masking by postulating some early disruption process. In these feedforward-type explanations, the mask somehow catches up with the target stimulus, disrupting its processing either through lateral or interchannel inhibition. However, studies from recent years indicate that visual perceptionand most notably visual awareness itselfmay depend strongly on cortico-cortical feedback connections from higher to lower visual areas. This has led some researchers to propose that masking derives its effectiveness from selectively interrupting these reentrant processes. In this experiment, we used electroencephalogram measurements to determine what happens in the human visual cortex during detection of a texture-defined square under nonmasked (seen) and masked (unseen) conditions. Electro-encephalogram derivatives that are typically associated with reentrant processing turn out to be absent in the masked condition. Moreover, extrastriate visual areas are still activated early on by both seen and unseen stimuli, as shown by scalp surface Laplacian current source-density maps. This conclusively shows that feedforward processing is preserved, even when subject performance is at chance as determined by objective measures. From these results, we conclude that masking derives its effectiveness, at least partly, from disrupting reentrant processing, thereby interfering with the neural mechanisms of figure-ground segmentation and visual awareness itself.

Figure–Ground Segregation in a Recurrent Network Architecture

Here we propose a model of how the visual brain segregates textured scenes into figures and background. During texture segregation, locations where the properties of texture elements change abruptly are assigned to boundaries, whereas image regions that are relatively homogeneous are grouped together. Boundary detection and grouping of image regions require different connection schemes, which are accommodated in a single network architecture by implementing them in different layers. As a result, all units carry signals related to boundary detection as well as grouping of image regions, in accordance with cortical physiology. Boundaries yield an early enhancement of network responses, but at a later point, an entire figural region is grouped together, because units that respond to it are labeled with enhanced activity. The model predicts which image regions are preferentially perceived as figure or as background and reproduces the spatio-temporal profile of neuronal activity in the visual cortex during texture segregation in intact animals, as well as in animals with cortical lesions.

Separate neural definitions of visual consciousness and visual attention: a case for phenomenal awareness

What is the relation between visual attention and visual awareness? It is difficult to imagine being aware of something without attending to it, and by some, visual consciousness is simply equated to what is in the focus of attention. However, findings from psychological as well as from neurophysiological experiments argue strongly against equating attention and visual consciousness. From these experiments clearly separate neural definitions of visual attention and visual consciousness emerge. In the model proposed here, visual attention is defined as a convolution of sensori-motor processing with memory. Consciousness, however, is generated by recurrent activity between cortical areas. The extent to which these recurrent interactions involve areas in executive or mnemonic space depends on attention and determines whether a conscious report is possible about the sensory experience, not whether the sensory experience is there. This way, a strong case can be made for a pure non-cognitive form of seeing, independent of attentional selection, called phenomenal awareness. This can be dissociated from the reportable form, depending on attention, called access awareness. The hypothesis explains why attention and consciousness seem so intricately related, even though they are fully separate phenomena.