H. Steven Scholte

Are There Multiple Visual Short-Term Memory Stores?

Background Classic work on visual short-term memory (VSTM) suggests that people store a limited amount of items for subsequent report. However, when human observers are cued to shift attention to one item in VSTM during retention, it seems as if there is a much larger representation, which keeps additional items in a more fragile VSTM store. Thus far, it is not clear whether the capacity of this fragile VSTM store indeed exceeds the traditional capacity limits of VSTM. The current experiments address this issue and explore the capacity, stability, and duration of fragile VSTM representations. Methodology/Principal Findings We presented cues in a change-detection task either just after off-set of the memory array (iconic-cue), 1,000 ms after off-set of the memory array (retro-cue) or after on-set of the probe array (post-cue). We observed three stages in visual information processing 1) iconic memory with unlimited capacity, 2) a four seconds lasting fragile VSTM store with a capacity that is at least a factor of two higher than 3) the robust and capacity-limited form of VSTM. Iconic memory seemed to depend on the strength of the positive after-image resulting from the memory display and was virtually absent under conditions of isoluminance or when intervening light masks were presented. This suggests that iconic memory is driven by prolonged retinal activation beyond stimulus duration. Fragile VSTM representations were not affected by light masks, but were completely overwritten by irrelevant pattern masks that spatially overlapped the memory array. Conclusions/Significance We find that immediately after a stimulus has disappeared from view, subjects can still access information from iconic memory because they can see an after-image of the display. After that period, human observers can still access a substantial, but somewhat more limited amount of information from a high-capacity, but fragile VSTM that is overwritten when new items are presented to the eyes. What is left after that is the traditional VSTM store, with a limit of about four objects. We conclude that human observers store more sustained representations than is evident from standard change detection tasks and that these representations can be accessed at will.

Function and Structure of the Right Inferior Frontal Cortex Predict Individual Differences in Response Inhibition: A Model-Based Approach

The ability to suppress ones impulses and actions constitutes a fundamental mechanism of cognitive control, thought to be subserved by the right inferior frontal cortex (rIFC). The neural bases of more selective inhibitory control when selecting between two actions have thus far remained articulated with less precision. Selective inhibition can be explored in detail by extracting parameters from response time (RT) distributions as derived from performance in the Simon task. Individual differences in RT distribution parameters not only can be used to probe the efficiency and temporal dynamics of selective response inhibition, but also allow a more detailed analysis of functional neuroimaging data. Such model-based analyses, which capitalize on individual differences, have demonstrated that selective response inhibition is subserved by the rIFC. The aim of the present study was to specify the relationship between model parameters of response inhibition and their functional and structural underpinnings in the brain. Functional magnetic resonance imaging (fMRI) data were obtained from healthy participants while performing a Simon task in which irrelevant information can activate incorrect responses that should be selectively inhibited in favor of selecting the correct response. In addition, structural data on the density of coherency of white matter tracts were obtained using diffusion tensor imaging (DTI). The analyses aimed at quantifying the extent to which RT distribution measures of response inhibition are associated with individual differences in both rIFC function and structure. The results revealed a strong correlation between the model parameters and both fMRI and DTI characteristics of the rIFC. In general, our results reveal that individual differences in inhibition are accompanied by differences in both brain function and structure.

Unconscious Activation of the Prefrontal No-Go Network

Cognitive control processes involving prefrontal cortex allow humans to overrule and inhibit habitual responses to optimize performance in new and challenging situations, and traditional views hold that cognitive control is tightly linked with consciousness. We used functional magnetic resonance imaging to investigate to what extent unconscious “no-go” stimuli are capable of reaching cortical areas involved in inhibitory control, particularly the inferior frontal cortex (IFC) and the pre-supplementary motor area (pre-SMA). Participants performed a go/no-go task that included conscious (weakly masked) no-go trials, unconscious (strongly masked) no-go trials, as well as go trials. Replicating typical neuroimaging findings, response inhibition on conscious no-go stimuli was associated with a (mostly right-lateralized) frontoparietal “inhibition network.” Here, we demonstrate, however, that an unconscious no-go stimulus also can activate prefrontal control networks, most prominently the IFC and the pre-SMA. Moreover, if it does so, it brings about a substantial slowdown in the speed of responding, as if participants attempted to inhibit their response but just failed to withhold it completely. Interestingly, overall activation in this “unconscious inhibition network” correlated positively with the amount of slowdown triggered by unconscious no-go stimuli. In addition, neural differences between conscious and unconscious control are revealed. These results expand our understanding of the limits and depths of unconscious information processing in the human brain and demonstrate that prefrontal cognitive control functions are not exclusively influenced by conscious information.

Feedforward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging

In texture segregation, an example of scene segmentation, we can discern two different processes: texture boundary detection and subsequent surface segregation [Lamme, V. A. F., Rodriguez-Rodriguez, V., & Spekreijse, H. Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cerebral Cortex, 9, 406413, 1999]. Neural correlates of texture boundary detection have been found in monkey V1 [Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378, 492496, 1995; Grosof, D. H., Shapley, R. M., & Hawken, M. J. Macaque-V1 neurons can signal illusory contours. Nature, 365, 550552, 1993], but whether surface segregation occurs in monkey V1 [Rossi, A. F., Desimone, R., & Ungerleider, L. G. Contextual modulation in primary visual cortex of macaques. Journal of Neuroscience, 21, 16981709, 2001; Lamme, V. A. F. The neurophysiology of figure ground segregation in primary visual-cortex. Journal of Neuroscience, 15, 16051615, 1995], and whether boundary detection or surface segregation signals can also be measured in human V1, is more controversial [Kastner, S., De Weerd, P., & Ungerleider, L. G. Texture segregation in the human visual cortex: A functional MRI study. Journal of Neurophysiology, 83, 24532457, 2000]. Here we present electroencephalography (EEG) and functional magnetic resonance imaging data that have been recorded with a paradigm that makes it possible to differentiate between boundary detection and scene segmentation in humans. In this way, we were able to show with EEG that neural correlates of texture boundary detection are first present in the early visual cortex around 92 msec and then spread toward the parietal and temporal lobes. Correlates of surface segregation first appear in temporal areas (around 112 msec) and from there appear to spread to parietal, and back to occipital areas. After 208 msec, correlates of surface segregation and boundary detection also appear in more frontal areas. Blood oxygenation level-dependent magnetic resonance imaging results show correlates of boundary detection and surface segregation in all early visual areas including V1. We conclude that texture boundaries are detected in a feedforward fashion and are represented at increasing latencies in higher visual areas. Surface segregation, on the other hand, is represented in reverse hierarchical fashion and seems to arise from feedback signals toward early visual areas such as V1.

GABA Shapes the Dynamics of Bistable Perception

Sometimes, perception fluctuates spontaneously between two distinct interpretations of a constant sensory input. These bistable perceptual phenomena provide a unique window into the neural mechanisms that create the contents of conscious perception. Models of bistable perception posit that mutual inhibition between stimulus-selective neural populations in visual cortex plays a key role in these spontaneous perceptual fluctuations. However, a direct link between neural inhibition and bistable perception has not yet been established experimentally. Here, we link perceptual dynamics in three distinct bistable visual illusions (binocular rivalry, motion-induced blindness, and structure from motion) to measurements of gamma-aminobutyric acid (GABA) concentrations in human visual cortex (as measured with magnetic resonance spectroscopy) and to pharmacological stimulation of the GABAA receptor by means of lorazepam. As predicted by a model of neural interactions underlying bistability, both higher GABA concentrations in visual cortex and lorazepam administration induced slower perceptual dynamics, as reflected in a reduced number of perceptual switches and a lengthening of percept durations. Thus, we show that GABA, the main inhibitory neurotransmitter, shapes the dynamics of bistable perception. These results pave the way for future studies into the competitive neural interactions across the visual cortical hierarchy that elicit conscious perception.

A neural substrate for atypical low-level visual processing in autism spectrum disorder

An important characteristic of autism spectrum disorder (ASD) is increased visual detail perception. Yet, there is no standing neurobiological explanation for this aspect of the disorder. We show evidence from EEG data, from 31 control subjects (three females) and 13 subjects (two females) aged 16-28 years, for a specific impairment in object boundary detection in ASD, which is present as early as 120 ms after stimulus presentation. In line with a neural network model explicating the role of feedforward, horizontal and recurrent processing in visual perception, we can attribute this deficit to a dysfunction of horizontal connections within early visual areas. Interestingly, ASD subjects showed an increase in subsequent activity at lateral occipital sites (225 ms), which might reflect a compensational mechanism. In contrast, recurrent processing between higher and lower visual areas (around 260 ms), associated with the segregation between figure and background, was normal. Our results show specific neural abnormalities in ASD related to low-level visual processing. In addition, given the reconciliation between our findings and previous neuropathology and neurochemistry research, we suggest that atypical horizontal interactions might reflect a more general neural abnormality in this disorder.

Detailed sensory memory, sloppy working memory

Visual short-term memory (VSTM) enables us to actively maintain information in mind for a brief period of time after stimulus disappearance. According to recent studies, VSTM consists of three stages – iconic memory, fragile VSTM, and visual working memory – with increasingly stricter capacity limits and progressively longer lifetimes. Still, the resolution (or amount of visual detail) of each VSTM stage has remained unexplored and we test this in the present study. We presented people with a change detection task that measures the capacity of all three forms of VSTM, and we added an identification display after each change trial that required people to identify the “pre-change” object. Accurate change detection plus pre-change identification requires subjects to have a high-resolution representation of the “pre-change” object, whereas change detection or identification only can be based on the hunch that something has changed, without exactly knowing what was presented before. We observed that people maintained 6.1 objects in iconic memory, 4.6 objects in fragile VSTM, and 2.1 objects in visual working memory. Moreover, when people detected the change, they could also identify the pre-change object on 88% of the iconic memory trials, on 71% of the fragile VSTM trials and merely on 53% of the visual working memory trials. This suggests that people maintain many high-resolution representations in iconic memory and fragile VSTM, but only one high-resolution object representation in visual working memory.

Confuse Your Illusion Feedback to Early Visual Cortex Contributes to Perceptual Completion

A striking example of the constructive nature of visual perception is how the human visual system completes contours of occluded objects. To date, it is unclear whether perceptual completion emerges during early stages of visual processing or whether higher-level mechanisms are necessary. To answer this question, we used transcranial magnetic stimulation to disrupt signaling in V1/V2 and in the lateral occipital (LO) area at different moments in time while participants performed a discrimination task involving a Kanizsa-type illusory figure. Results show that both V1/V2 and higher-level visual area LO are critically involved in perceptual completion. However, these areas seem to be involved in an inverse hierarchical fashion, in which the critical time window for V1/V2 follows that for LO. These results are in line with the growing evidence that feedback to V1/V2 contributes to perceptual completion.

Neural substrates of individual differences in human fear learning: Evidence from concurrent fMRI, fear-potentiated startle, and US-expectancy data

To provide insight into individual differences in fear learning, we examined the emotional and cognitive expressions of discriminative fear conditioning in direct relation to its neural substrates. Contrary to previous behavioral–neural (fMRI) research on fear learning—in which the emotional expression of fear was generally indexed by skin conductance—we used fear-potentiated startle, a more reliable and specific index of fear. While we obtained concurrent fear-potentiated startle, neuroimaging (fMRI), and US-expectancy data, healthy participants underwent a fear-conditioning paradigm in which one of two conditioned stimuli (CS+ but not CS–) was paired with a shock (unconditioned stimulus [US]). Fear learning was evident from the differential expressions of fear (CS+ > CS–) at both the behavioral level (startle potentiation and US expectancy) and the neural level (in amygdala, anterior cingulate cortex, hippocampus, and insula). We examined individual differences in discriminative fear conditioning by classifying participants (as conditionable vs. unconditionable) according to whether they showed successful differential startle potentiation. This revealed that the individual differences in the emotional expression of discriminative fear learning (startle potentiation) were reflected in differential amygdala activation, regardless of the cognitive expression of fear learning (CS–US contingency or hippocampal activation). Our study provides the first evidence for the potential of examining startle potentiation in concurrent fMRI research on fear learning.

The spatial profile of visual attention in mental curve tracing.

In a curve-tracing task, subjects have to judge whether items are located on a single, continuous curve. Spatially separate segments of such a curve are related to each other through grouping criteria, like collinearity and connectedness. These grouping cues need to be exploited during curve tracing, but it is still an open issue how grouping of contour segments is achieved by the visual system. Many contemporary theories of visual perception assume that grouping operations are carried out pre-attentively, with unlimited capacity. The present study examines this assumption by investigating the involvement of attention in curve tracing. The results show that attention is directed to contour segments that need to be grouped together. The distribution of attention is guided by grouping criteria, such as connectedness. Apparently, attention is required to group spatially separate contour segments into a coherent representation of a curve.