Rufin VanRullen

Rapid natural scene categorization in the near absence of attention

What can we see when we do not pay attention? It is well known that we can be “blind” even to major aspects of natural scenes when we attend elsewhere. The only tasks that do not need attention appear to be carried out in the early stages of the visual system. Contrary to this common belief, we report that subjects can rapidly detect animals or vehicles in briefly presented novel natural scenes while simultaneously performing another attentionally demanding task. By comparison, they are unable to discriminate large Ts from Ls, or bisected two-color disks from their mirror images under the same conditions. We conclude that some visual tasks associated with “high-level” cortical areas may proceed in the near absence of attention.

The Phase of Ongoing EEG Oscillations Predicts Visual Perception

Oscillations are ubiquitous in electrical recordings of brain activity. While the amplitude of ongoing oscillatory activity is known to correlate with various aspects of perception, the influence of oscillatory phase on perception remains unknown. In particular, since phase varies on a much faster timescale than the more sluggish amplitude fluctuations, phase effects could reveal the fine-grained neural mechanisms underlying perception. We presented brief flashes of light at the individual luminance threshold while EEG was recorded. Although the stimulus on each trial was identical, subjects detected approximately half of the flashes (hits) and entirely missed the other half (misses). Phase distributions across trials were compared between hits and misses. We found that shortly before stimulus onset, each of the two distributions exhibited significant phase concentration, but at different phase angles. This effect was strongest in the theta and alpha frequency bands. In this time–frequency range, oscillatory phase accounted for at least 16% of variability in detection performance and allowed the prediction of performance on the single-trial level. This finding indicates that the visual detection threshold fluctuates over time along with the phase of ongoing EEG activity. The results support the notion that ongoing oscillations shape our perception, possibly by providing a temporal reference frame for neural codes that rely on precise spike timing.

The Time Course of Visual Processing: From Early Perception to Decision-Making

Experiments investigating the mechanisms involved in visual processing often fail to separate low-level encoding mechanisms from higher-level behaviorally relevant ones. Using an alternating dual-task event-related potential (ERP) experimental paradigm (animals or vehicles categorization) where targets of one task are intermixed among distractors of the other, we show that visual categorization of a natural scene involves different mechanisms with different time courses: a perceptual, task-independent mechanism, followed by a task-related, category-independent process. Although average ERP responses reflect the visual category of the stimulus shortly after visual processing has begun (e.g. 75-80 msec), this difference is not correlated with the subjects behavior until 150 msec poststimulus.

Is perception discrete or continuous

How does conscious perception evolve following stimulus presentation? The idea that perception relies on discrete processing epochs has been often considered, but never widely accepted. The alternative, a continuous translation of the external world into explicit perception, although more intuitive and subjectively appealing, cannot satisfactorily account for a large body of psychophysical data. Cortical and thalamocortical oscillations in different frequency bands could provide a neuronal basis for such discrete processes, but are rarely analyzed in this context. This article reconciles the unduly abandoned topic of discrete perception with current views and advances in neuroscience.

Spike times make sense

Many behavioral responses are completed too quickly for the underlying sensory processes to rely on estimation of neural firing rates over extended time windows. Theoretically, first-spike times could underlie such rapid responses, but direct evidence has been lacking. Such evidence has now been uncovered in the human somatosensory system. We discuss these findings and their potential generalization to other sensory modalities, and we consider some future challenges for the neuroscientific community.

Spontaneous EEG oscillations reveal periodic sampling of visual attention

An important effect of sustained attention is the facilitation of perception. Although the term “sustained” suggests that this beneficial effect endures continuously as long as something is attended, we present electrophysiological evidence that perception at attended locations is actually modulated periodically. Subjects detected brief light flashes that were presented peripherally at locations that were either attended or unattended. We analyzed the correlation between detection performance for attended and unattended stimuli and the phase of ongoing EEG oscillations, which relate to subsecond fluctuations of neuronal excitability. Although on average, detection performance was improved by attention—indicated by reduced detection thresholds at attended locations—we found that detection performance for attended stimuli actually fluctuated over time along with the phase of spontaneous oscillations in the θ (≈7 Hz) frequency band just before stimulus onset. This fluctuation was absent for unattended stimuli. This pattern of results suggests that “sustained” attention in fact exerts its facilitative effect on perception in a periodic fashion.

Is it a Bird? Is it a Plane? Ultra-Rapid Visual Categorisation of Natural and Artifactual Objects

Visual processing is known to be very fast in ultra-rapid categorisation tasks where the subject has to decide whether a briefly flashed image belongs to a target category or not. Human subjects can respond in under 400 ms, and event-related-potential studies have shown that the underlying processing can be done in less than 150 ms. Monkeys trained to perform the same task have proved even faster. However, most of these experiments have only been done with biologically relevant target categories such as animals or food. Here we performed the same study on human subjects, alternating between a task in which the target category was ‘animal’, and a task in which the target category was ‘means of transport’. These natural images of clearly artificial objects contained targets as varied as cars, trucks, trains, boats, aircraft, and hot-air balloons. However, the subjects performed almost identically in both tasks, with reaction times not significantly longer in the ‘means of transport’ task. These reaction times were much shorter than in any previous study on natural-image processing. We conclude that, at least for these two superordinate categories, the speed of ultra-rapid visual categorisation of natural scenes does not depend on the target category, and that this processing could rely primarily on feed-forward, automatic mechanisms.

Surfing a spike wave down the ventral stream

Numerous theories of neural processing, often motivated by experimental observations, have explored the computational properties of neural codes based on the absolute or relative timing of spikes in spike trains. Spiking neuron models and theories however, as well as their experimental counterparts, have generally been limited to the simulation or observation of isolated neurons, isolated spike trains, or reduced neural populations. Such theories would therefore seem inappropriate to capture the properties of a neural code relying on temporal spike patterns distributed across large neuronal populations. Here we report a range of computer simulations and theoretical considerations that were designed to explore the possibilities of one such code and its relevance for visual processing. In a unified framework where the relation between stimulus saliency and spike relative timing plays the central role, we describe how the ventral stream of the visual system could process natural input scenes and extract meaningful information, both rapidly and reliably. The first wave of spikes generated in the retina in response to a visual stimulation carries information explicitly in its spatio-temporal structure: the most salient information is represented by the first spikes over the population. This spike wave, propagating through a hierarchy of visual areas, is regenerated at each processing stage, where its temporal structure can be modified by (i). the selectivity of the cortical neurons, (ii). lateral interactions and (iii). top-down attentional influences from higher order cortical areas. The resulting model could account for the remarkable efficiency and rapidity of processing observed in the primate visual system.

The Phase of Ongoing Oscillations Mediates the Causal Relation between Brain Excitation and Visual Perception

Why does neuronal activity in sensory brain areas sometimes give rise to perception, and sometimes not? Although neuronal noise is often invoked as the key factor, a portion of this variability could also be due to the history and current state of the brain affecting cortical excitability. Here we directly test this idea by examining whether the phase of prestimulus oscillatory activity is causally linked with modulations of cortical excitability and with visual perception. Transcranial magnetic stimulation (TMS) was applied over human visual cortex to induce illusory perceptions (phosphenes) while electroencephalograms (EEGs) were simultaneously recorded. Subjects reported the presence or absence of an induced phosphene following a single pulse of TMS at perceptual threshold. The phase of ongoing alpha (∼10 Hz) oscillations within 400 ms before the pulse significantly covaried with the perceptual outcome. This effect was observed in occipital regions around the site of TMS, as well as in a distant frontocentral region. In both regions, we found a systematic relationship between prepulse EEG phase and perceptual performance: phosphene probability changed by ∼15% between opposite phases. In summary, we provide direct evidence for a chain of causal relations between the phase of ongoing oscillations, neuronal excitability, and visual perception: ongoing oscillations create periodic “windows of excitability,” with sensory perception being more likely to occur at specific phases.

The blinking spotlight of attention

Increasing evidence suggests that attention can concurrently select multiple locations; yet it is not clear whether this ability relies on continuous allocation of attention to the different targets (a “parallel” strategy) or whether attention switches rapidly between the targets (a periodic “sampling” strategy). Here, we propose a method to distinguish between these two alternatives. The human psychometric function for detection of a single target as a function of its duration can be used to predict the corresponding function for two or more attended targets. Importantly, the predicted curves differ, depending on whether a parallel or sampling strategy is assumed. For a challenging detection task, we found that human performance was best reflected by a sampling model, indicating that multiple items of interest were processed in series at a rate of approximately seven items per second. Surprisingly, the data suggested that attention operated in this periodic regime, even when it was focused on a single target. That is, attention might rely on an intrinsically periodic process.