H.S. Scholte

Masking Disrupts Reentrant Processing in Human Visual Cortex

In masking, a stimulus is rendered invisible through the presentation of a second stimulus shortly after the first. Over the years, authors have typically explained masking by postulating some early disruption process. In these feedforward-type explanations, the mask somehow catches up with the target stimulus, disrupting its processing either through lateral or interchannel inhibition. However, studies from recent years indicate that visual perceptionand most notably visual awareness itselfmay depend strongly on cortico-cortical feedback connections from higher to lower visual areas. This has led some researchers to propose that masking derives its effectiveness from selectively interrupting these reentrant processes. In this experiment, we used electroencephalogram measurements to determine what happens in the human visual cortex during detection of a texture-defined square under nonmasked (seen) and masked (unseen) conditions. Electro-encephalogram derivatives that are typically associated with reentrant processing turn out to be absent in the masked condition. Moreover, extrastriate visual areas are still activated early on by both seen and unseen stimuli, as shown by scalp surface Laplacian current source-density maps. This conclusively shows that feedforward processing is preserved, even when subject performance is at chance as determined by objective measures. From these results, we conclude that masking derives its effectiveness, at least partly, from disrupting reentrant processing, thereby interfering with the neural mechanisms of figure-ground segmentation and visual awareness itself.

Frontal Cortex Mediates Unconsciously Triggered Inhibitory Control

To further our understanding of the function of conscious experience we need to know which cognitive processes require awareness and which do not. Here, we show that an unconscious stimulus can trigger inhibitory control processes, commonly ascribed to conscious control mechanisms. We combined the metacontrast masking paradigm and the Go/No-Go paradigm to study whether unconscious No-Go signals can actively trigger high-level inhibitory control processes, strongly associated with the prefrontal cortex (PFC). Behaviorally, unconscious No-Go signals sometimes triggered response inhibition to the level of complete response termination and yielded a slow down in the speed of responses that were not inhibited. Electroencephalographic recordings showed that unconscious No-Go signals elicit two neural events: (1) an early occipital event and (2) a frontocentral event somewhat later in time. The first neural event represents the visual encoding of the unconscious No-Go stimulus, and is also present in a control experiment where the masked stimulus has no behavioral relevance. The second event is unique to the Go/No-Go experiment, and shows the subsequent implementation of inhibitory control in the PFC. The size of the frontal activity pattern correlated highly with the impact of unconscious No-Go signals on subsequent behavior. We conclude that unconscious stimuli can influence whether a task will be performed or interrupted, and thus exert a form of cognitive control. These findings challenge traditional views concerning the proposed relationship between awareness and cognitive control and stretch the alleged limits and depth of unconscious information processing.

Brain responses strongly correlate with Weibull image statistics when processing natural images.

The visual appearance of natural scenes is governed by a surprisingly simple hidden structure. The distributions of contrast values in natural images generally follow a Weibull distribution, with beta and gamma as free parameters. Beta and gamma seem to structure the space of natural images in an ecologically meaningful way, in particular with respect to the fragmentation and texture similarity within an image. Since it is often assumed that the brain exploits structural regularities in natural image statistics to efficiently encode and analyze visual input, we here ask ourselves whether the brain approximates the beta and gamma values underlying the contrast distributions of natural images. We present a model that shows that beta and gamma can be easily estimated from the outputs of X-cells and Y-cells. In addition, we covaried the EEG responses of subjects viewing natural images with the beta and gamma values of those images. We show that beta and gamma explain up to 71% of the variance of the early ERP signal, substantially outperforming other tested contrast measurements. This suggests that the brain is strongly tuned to the images beta and gamma values, potentially providing the visual system with an efficient way to rapidly classify incoming images on the basis of omnipresent low-level natural image statistics.

Neural Basis of Individual Differences in Synesthetic Experiences

Little is known about how the properties of our private mental world relate to the physical and functional properties of our brain. Studying synesthesia, where a particular experience evokes a separate additional sensory experience, offers the unique opportunity to study phenomenological experiences as a stable trait in healthy subjects. A common form of synesthesia is grapheme–color synesthesia, where a particular letter or number evokes a particular color experience. We studied the neural basis of qualitative different properties of the synesthetic experience by using individual differences in grapheme–color synesthesia. Specifically, the synesthetic color can be experienced “in the mind” (associator synesthetes) or “in the outside world” (projector synesthetes). Gray matter structure and functioning (imaged using voxel-based morphometry and functional magnetic resonance imaging, respectively) were examined in grapheme–color synesthetes (N = 42, 16 projectors and 26 associators) and nonsynesthetes. Results indicated partly shared mechanisms for all grapheme–color synesthetes, particularly in posterior superior parietal lobe, which is involved in the integration of sensory information. In addition, the nature of synesthetic experience was found to be mediated by distinct neural mechanisms. The outside-world experience is related to brain areas involved in perceiving and acting in the outside world (visual cortex, auditory cortex, motor cortex) as well as frontal brain areas. In contrast, the in-the-mind experience is related to the hippocampus and parahippocampal gyrus, known for their role in memory. Thus, the different subjective experiences are related to distinct neural mechanisms. Moreover, the properties of subjective experiences are in accordance with functional properties of the mediating brain mechanisms.

V4 Activity Predicts the Strength of Visual Short-Term Memory Representations

Recent studies have shown the existence of a form of visual memory that lies intermediate of iconic memory and visual short-term memory (VSTM), in terms of both capacity (up to 15 items) and the duration of the memory trace (up to 4 s). Because new visual objects readily overwrite this intermediate visual store, we believe that it reflects a weak form of VSTM with high capacity that exists alongside a strong but capacity-limited form of VSTM. In the present study, we isolated brain activity related to weak and strong VSTM representations using functional magnetic resonance imaging. We found that activity in visual cortical area V4 predicted the strength of VSTM representations; activity was low when there was no VSTM, medium when there was a weak VSTM representation regardless of whether this weak representation was available for report or not, and high when there was a strong VSTM representation. Altogether, this study suggests that the high capacity yet weak VSTM store is represented in visual parts of the brain. Allegedly, only some of these VSTM traces are amplified by parietal and frontal regions and as a consequence reside in traditional or strong VSTM. The additional weak VSTM representations remain available for conscious access and report when attention is redirected to them yet are overwritten as soon as new visual stimuli hit the eyes.

Bottom-up and top-down attention are independent

What is the relationship between top-down and bottom-up attention? Are both types of attention tightly interconnected, or are they independent? We investigated this by testing a large representative sample of the Dutch population on two attentional tasks: a visual search task gauging the efficiency of top-down attention and a singleton capture task gauging bottom-up attention. On both tasks we found typical performance--i.e., participants displayed a significant search slope on the search task and significant slowing caused by the unique, but irrelevant, object on the capture task. Moreover, the high levels of significance we observed indicate that the current set-up provided very high signal to noise ratios, and thus enough power to accurately unveil existing effects. Importantly, in this robust investigation we did not observe any correlation in performance between tasks. The use of Bayesian statistics strongly confirmed that performance on both tasks was uncorrelated. We argue that the current results suggest that there are two attentional systems that operate independently. We hypothesize that this may have implications beyond our understanding of attention. For instance, it may be that attention and consciousness are intertwined differently for top-down attention than for bottom-up attention.

From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category

The visual system processes natural scenes in a split second. Part of this process is the extraction of “gist,” a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity to low-level summary statistics. Subjects rated a specific instance of a global scene property, naturalness, for a large set of natural scenes while EEG was recorded. For each individual scene, we derived two physiologically plausible summary statistics by spatially pooling local contrast filter outputs: contrast energy (CE), indexing contrast strength, and spatial coherence (SC), indexing scene fragmentation. We show that behavioral performance is directly related to these statistics, with naturalness rating being influenced in particular by SC. At the neural level, both statistics parametrically modulated single-trial event-related potential amplitudes during an early, transient window (100–150 ms), but SC continued to influence activity levels later in time (up to 250 ms). In addition, the magnitude of neural activity that discriminated between man-made versus natural ratings of individual trials was related to SC, but not CE. These results suggest that global scene information may be computed by spatial pooling of responses from early visual areas (e.g., LGN or V1). The increased sensitivity over time to SC in particular, which reflects scene fragmentation, suggests that this statistic is actively exploited to estimate scene naturalness.

Neuronal integration in visual cortex elevates face category tuning to conscious face perception

The human brain has the extraordinary capability to transform cluttered sensory input into distinct object representations. For example, it is able to rapidly and seemingly without effort detect object categories in complex natural scenes. Surprisingly, category tuning is not sufficient to achieve conscious recognition of objects. What neural process beyond category extraction might elevate neural representations to the level where objects are consciously perceived? Here we show that visible and invisible faces produce similar category-selective responses in the ventral visual cortex. The pattern of neural activity evoked by visible faces could be used to decode the presence of invisible faces and vice versa. However, only visible faces caused extensive response enhancements and changes in neural oscillatory synchronization, as well as increased functional connectivity between higher and lower visual areas. We conclude that conscious face perception is more tightly linked to neural processes of sustained information integration and binding than to processes accommodating face category tuning.

Magnetic stimulation of the dorsolateral prefrontal cortex dissociates fragile visual short-term memory from visual working memory

To guide our behavior in successful ways, we often need to rely on information that is no longer in view, but maintained in visual short-term memory (VSTM). While VSTM is usually broken down into iconic memory (brief and high-capacity store) and visual working memory (sustained, yet limited-capacity store), recent studies have suggested the existence of an additional and intermediate form of VSTM that depends on activity in extrastriate cortex. In previous work, we have shown that this fragile form of VSTM can be dissociated from iconic memory. In the present study, we provide evidence that fragile VSTM is different from visual working memory as magnetic stimulation of the right dorsolateral prefrontal cortex (DLPFC) disrupts visual working memory, while leaving fragile VSTM intact. In addition, we observed that people with high DLPFC activity had superior working memory capacity compared to people with low DLPFC activity, and only people with high DLPFC activity really showed a reduction in working memory capacity in response to magnetic stimulation. Altogether, this study shows that VSTM consists of three stages that have clearly different characteristics and rely on different neural structures. On the methodological side, we show that it is possible to predict individual susceptibility to magnetic stimulation based on functional MRI activity.

Prior Knowledge about Objects Determines Neural Color Representation in Human Visual Cortex

To create subjective experience, our brain must translate physical stimulus input by incorporating prior knowledge and expectations. For example, we perceive color and not wavelength information, and this in part depends on our past experience with colored objects ( Hansen et al. 2006; Mitterer and de Ruiter 2008). Here, we investigated the influence of object knowledge on the neural substrates underlying subjective color vision. In a functional magnetic resonance imaging experiment, human subjects viewed a color that lay midway between red and green (ambiguous with respect to its distance from red and green) presented on either typical red (e.g., tomato), typical green (e.g., clover), or semantically meaningless (nonsense) objects. Using decoding techniques, we could predict whether subjects viewed the ambiguous color on typical red or typical green objects based on the neural response of veridical red and green. This shift of neural response for the ambiguous color did not occur for nonsense objects. The modulation of neural responses was observed in visual areas (V3, V4, VO1, lateral occipital complex) involved in color and object processing, as well as frontal areas. This demonstrates that object memory influences wavelength information relatively early in the human visual system to produce subjective color vision.