Johannes J. Fahrenfort

Masking Disrupts Reentrant Processing in Human Visual Cortex

In masking, a stimulus is rendered invisible through the presentation of a second stimulus shortly after the first. Over the years, authors have typically explained masking by postulating some early disruption process. In these feedforward-type explanations, the mask somehow catches up with the target stimulus, disrupting its processing either through lateral or interchannel inhibition. However, studies from recent years indicate that visual perceptionand most notably visual awareness itselfmay depend strongly on cortico-cortical feedback connections from higher to lower visual areas. This has led some researchers to propose that masking derives its effectiveness from selectively interrupting these reentrant processes. In this experiment, we used electroencephalogram measurements to determine what happens in the human visual cortex during detection of a texture-defined square under nonmasked (seen) and masked (unseen) conditions. Electro-encephalogram derivatives that are typically associated with reentrant processing turn out to be absent in the masked condition. Moreover, extrastriate visual areas are still activated early on by both seen and unseen stimuli, as shown by scalp surface Laplacian current source-density maps. This conclusively shows that feedforward processing is preserved, even when subject performance is at chance as determined by objective measures. From these results, we conclude that masking derives its effectiveness, at least partly, from disrupting reentrant processing, thereby interfering with the neural mechanisms of figure-ground segmentation and visual awareness itself.

Frontal Cortex Mediates Unconsciously Triggered Inhibitory Control

To further our understanding of the function of conscious experience we need to know which cognitive processes require awareness and which do not. Here, we show that an unconscious stimulus can trigger inhibitory control processes, commonly ascribed to conscious control mechanisms. We combined the metacontrast masking paradigm and the Go/No-Go paradigm to study whether unconscious No-Go signals can actively trigger high-level inhibitory control processes, strongly associated with the prefrontal cortex (PFC). Behaviorally, unconscious No-Go signals sometimes triggered response inhibition to the level of complete response termination and yielded a slow down in the speed of responses that were not inhibited. Electroencephalographic recordings showed that unconscious No-Go signals elicit two neural events: (1) an early occipital event and (2) a frontocentral event somewhat later in time. The first neural event represents the visual encoding of the unconscious No-Go stimulus, and is also present in a control experiment where the masked stimulus has no behavioral relevance. The second event is unique to the Go/No-Go experiment, and shows the subsequent implementation of inhibitory control in the PFC. The size of the frontal activity pattern correlated highly with the impact of unconscious No-Go signals on subsequent behavior. We conclude that unconscious stimuli can influence whether a task will be performed or interrupted, and thus exert a form of cognitive control. These findings challenge traditional views concerning the proposed relationship between awareness and cognitive control and stretch the alleged limits and depth of unconscious information processing.

Feedforward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging

In texture segregation, an example of scene segmentation, we can discern two different processes: texture boundary detection and subsequent surface segregation [Lamme, V. A. F., Rodriguez-Rodriguez, V., & Spekreijse, H. Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cerebral Cortex, 9, 406413, 1999]. Neural correlates of texture boundary detection have been found in monkey V1 [Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378, 492496, 1995; Grosof, D. H., Shapley, R. M., & Hawken, M. J. Macaque-V1 neurons can signal illusory contours. Nature, 365, 550552, 1993], but whether surface segregation occurs in monkey V1 [Rossi, A. F., Desimone, R., & Ungerleider, L. G. Contextual modulation in primary visual cortex of macaques. Journal of Neuroscience, 21, 16981709, 2001; Lamme, V. A. F. The neurophysiology of figure ground segregation in primary visual-cortex. Journal of Neuroscience, 15, 16051615, 1995], and whether boundary detection or surface segregation signals can also be measured in human V1, is more controversial [Kastner, S., De Weerd, P., & Ungerleider, L. G. Texture segregation in the human visual cortex: A functional MRI study. Journal of Neurophysiology, 83, 24532457, 2000]. Here we present electroencephalography (EEG) and functional magnetic resonance imaging data that have been recorded with a paradigm that makes it possible to differentiate between boundary detection and scene segmentation in humans. In this way, we were able to show with EEG that neural correlates of texture boundary detection are first present in the early visual cortex around 92 msec and then spread toward the parietal and temporal lobes. Correlates of surface segregation first appear in temporal areas (around 112 msec) and from there appear to spread to parietal, and back to occipital areas. After 208 msec, correlates of surface segregation and boundary detection also appear in more frontal areas. Blood oxygenation level-dependent magnetic resonance imaging results show correlates of boundary detection and surface segregation in all early visual areas including V1. We conclude that texture boundaries are detected in a feedforward fashion and are represented at increasing latencies in higher visual areas. Surface segregation, on the other hand, is represented in reverse hierarchical fashion and seems to arise from feedback signals toward early visual areas such as V1.

Dissociable brain mechanisms underlying the conscious and unconscious control of behavior

Cognitive control allows humans to overrule and inhibit habitual responses to optimize performance in challenging situations. Contradicting traditional views, recent studies suggest that cognitive control processes can be initiated unconsciously. To further capture the relation between consciousness and cognitive control, we studied the dynamics of inhibitory control processes when triggered consciously versus unconsciously in a modified version of the stop task. Attempts to inhibit an imminent response were often successful after unmasked (visible) stop signals. Masked (invisible) stop signals rarely succeeded in instigating overt inhibition but did trigger slowing down of response times. Masked stop signals elicited a sequence of distinct ERP components that were also observed on unmasked stop signals. The N2 component correlated with the efficiency of inhibitory control when elicited by unmasked stop signals and with the magnitude of slowdown when elicited by masked stop signals. Thus, the N2 likely reflects the initiation of inhibitory control, irrespective of conscious awareness. The P3 component was much reduced in amplitude and duration on masked versus unmasked stop trials. These patterns of differences and similarities between conscious and unconscious cognitive control processes are discussed in a framework that differentiates between feedforward and feedback connections in yielding conscious experience.

Neuronal integration in visual cortex elevates face category tuning to conscious face perception

The human brain has the extraordinary capability to transform cluttered sensory input into distinct object representations. For example, it is able to rapidly and seemingly without effort detect object categories in complex natural scenes. Surprisingly, category tuning is not sufficient to achieve conscious recognition of objects. What neural process beyond category extraction might elevate neural representations to the level where objects are consciously perceived? Here we show that visible and invisible faces produce similar category-selective responses in the ventral visual cortex. The pattern of neural activity evoked by visible faces could be used to decode the presence of invisible faces and vice versa. However, only visible faces caused extensive response enhancements and changes in neural oscillatory synchronization, as well as increased functional connectivity between higher and lower visual areas. We conclude that conscious face perception is more tightly linked to neural processes of sustained information integration and binding than to processes accommodating face category tuning.

Seeing without knowing: Neural signatures of perceptual inference in the absence of report

Every day, we experience a rich and complex visual world. Our brain constantly translates meaningless fragmented input into coherent objects and scenes. However, our attentional capabilities are limited, and we can only report the few items that we happen to attend to. So what happens to items that are not cognitively accessed? Do these remain fragmentary and meaningless? Or are they processed up to a level where perceptual inferences take place about image composition? To investigate this, we recorded brain activity using fMRI while participants viewed images containing a Kanizsa figure, an illusion in which an object is perceived by means of perceptual inference. Participants were presented with the Kanizsa figure and three matched nonillusory control figures while they were engaged in an attentionally demanding distractor task. After the task, one group of participants was unable to identify the Kanizsa figure in a forced-choice decision task; hence, they were “inattentionally blind.” A second group had no trouble identifying the Kanizsa figure. Interestingly, the neural signature that was unique to the processing of the Kanizsa figure was present in both groups. Moreover, within-subject multivoxel pattern analysis showed that the neural signature of unreported Kanizsa figures could be used to classify reported Kanizsa figures and that this cross-report classification worked better for the Kanizsa condition than for the control conditions. Together, these results suggest that stimuli that are not cognitively accessed are processed up to levels of perceptual interpretation.

Accurate Metacognition for Visual Sensory Memory Representations

The capacity to attend to multiple objects in the visual field is limited. However, introspectively, people feel that they see the whole visual world at once. Some scholars suggest that this introspective feeling is based on short-lived sensory memory representations, whereas others argue that the feeling of seeing more than can be attended to is illusory. Here, we investigated this phenomenon by combining objective memory performance with subjective confidence ratings during a change-detection task. This allowed us to compute a measure of metacognition—the degree of knowledge that subjects have about the correctness of their decisions—for different stages of memory. We show that subjects store more objects in sensory memory than they can attend to but, at the same time, have similar metacognition for sensory memory and working memory representations. This suggests that these subjective impressions are not an illusion but accurate reflections of the richness of visual perception.

Non-Attended Representations are Perceptual Rather than Unconscious in Nature

Introspectively we experience a phenomenally rich world. In stark contrast, many studies show that we can only report on the few items that we happen to attend to. So what happens to the unattended objects? Are these consciously processed as our first person perspective would have us believe, or are they – in fact – entirely unconscious? Here, we attempt to resolve this question by investigating the perceptual characteristics of visual sensory memory. Sensory memory is a fleeting, high-capacity form of memory that precedes attentional selection and working memory. We found that memory capacity benefits from figural information induced by the Kanizsa illusion. Importantly, this benefit was larger for sensory memory than for working memory and depended critically on the illusion, not on the stimulus configuration. This shows that pre-attentive sensory memory contains representations that have a genuinely perceptual nature, suggesting that non-attended representations are phenomenally experienced rather than unconscious.

The relationship between visual awareness, attention, and report

Attention and awareness seem to be intimately related. Intuitively, it seems logical that we become aware of events that we attend to, whereas information outside the scope of our attention remains undetected. Indeed, some have argued that there is a tight relationship between attention and

Neural Correlates of Dynamically Evolving Interpersonal Ties Predict Prosocial Behavior

There is a growing interest for the determinants of human choice behavior in social settings. Upon initial contact, investment choices in social settings can be inherently risky, as the degree to which the other person will reciprocate is unknown. Nevertheless, people have been shown to exhibit prosocial behavior even in one-shot laboratory settings where all interaction has been taken away. A logical step has been to link such behavior to trait empathy-related neurobiological networks. However, as a social interaction unfolds, the degree of uncertainty with respect to the expected payoff of choice behavior may change as a function of the interaction. Here we attempt to capture this factor. We show that the interpersonal tie one develops with another person during interaction – rather than trait empathy – motivates investment in a public good that is shared with an anonymous interaction partner. We examined how individual differences in trait empathy and interpersonal ties modulate neural responses to imposed monetary sharing. After, but not before interaction in a public good game, sharing prompted activation of neural systems associated with reward (striatum), empathy (anterior insular cortex and anterior cingulate cortex) as well as altruism, and social significance [posterior superior temporal sulcus (pSTS)]. Although these activations could be linked to both empathy and interpersonal ties, only tie-related pSTS activation predicted prosocial behavior during subsequent interaction, suggesting a neural substrate for keeping track of social relevance.