Petra Vetter

Decoding Sound and Imagery Content in Early Visual Cortex

Summary Human early visual cortex was traditionally thought to process simple visual features such as orientation, contrast, and spatial frequency via feedforward input from the lateral geniculate nucleus (e.g., [1]). However, the role of nonretinal influence on early visual cortex is so far insufficiently investigated despite much evidence that feedback connections greatly outnumber feedforward connections [2–5]. Here, we explored in five fMRI experiments how information originating from audition and imagery affects the brain activity patterns in early visual cortex in the absence of any feedforward visual stimulation. We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants. The coding of nonretinal information in the activity patterns of early visual cortex is common across actual auditory perception and imagery and may be mediated by higher-level multisensory areas. Furthermore, this coding is robust to mild manipulations of attention and working memory but affected by orthogonal, cognitively demanding visuospatial processing. Crucially, the information fed down to early visual cortex is category specific and generalizes to sound exemplars of the same category, providing evidence for abstract information feedback rather than precise pictorial feedback. Our results suggest that early visual cortex receives nonretinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information. Our findings are compatible with feedback of predictive information to the earliest visual input level (e.g., [6]), in line with predictive coding models [7–10].

Varieties of cognitive penetration in visual perception

Is our perceptual experience a veridical representation of the world or is it a product of our beliefs and past experiences? Cognitive penetration describes the influence of higher level cognitive factors on perceptual experience and has been a debated topic in philosophy of mind and cognitive science. Here, we focus on visual perception, particularly early vision, and how it is affected by contextual expectations and memorized cognitive contents. We argue for cognitive penetration based on recent empirical evidence demonstrating contextual and top-down influences on early visual processes. On the basis of a perceptual model, we propose different types of cognitive penetration depending on the processing level on which the penetration happens and depending on where the penetrating influence comes from. Our proposal has two consequences: (1) the traditional controversy on whether cognitive penetration occurs or not is ill posed, and (2) a clear-cut perception-cognition boundary cannot be maintained.

Modulating attentional load affects numerosity estimation: evidence against a pre-attentive subitizing mechanism

Traditionally, the visual enumeration of a small number of items (1 to about 4), referred to as subitizing, has been thought of as a parallel and pre-attentive process and functionally different from the serial attentive enumeration of larger numerosities. We tested this hypothesis by employing a dual task paradigm that systematically manipulated the attentional resources available to an enumeration task. Enumeration accuracy for small numerosities was severely decreased as more attentional resources were taken away from the numerical task, challenging the traditionally held notion of subitizing as a pre-attentive, capacity-independent process. Judgement of larger numerosities was also affected by dual task conditions and attentional load. These results challenge the proposal that small numerosities are enumerated by a mechanism separate from large numerosities and support the idea of a single, attention-demanding enumeration mechanism.

A candidate for the attentional bottleneck: Set-size specific modulation of the right tpj during attentive enumeration

Several recent behavioral studies have shown that the enumeration of a small number of items (a process termed subitizing) depends on the availability of attentional resources and is not a preattentive process as previously thought. Here we studied the neural correlates of visual enumeration under different attentional loads in a dual-task paradigm using fMRI. Relatively intact subitizing under low attentional load compared to impaired subitizing under high attentional load was associated with an increase in BOLD signal in the right temporo-parietal junction (rTPJ). Crucially, attentionally modulated response in the rTPJ was specific to small set sizes (up to 3 items) and did not occur at larger set sizes (5–7 items). This result has two implications: (1) Subitizing involves part of the fronto-parietal network for stimulus-driven attention providing neural evidence against preattentive subitizing. (2) Activity in rTPJ is set-size modulated. Together with similar evidence from studies probing visual short-term memory, this result suggests that rTPJ modulation might reflect the brains ability to attentively handle small set sizes. Thus, the rTPJ may play an important role for the emergence of a capacity limit in both enumeration and visual short-term memory.

Unconscious Numerical Priming Despite Interocular Suppression

Whether high-level properties of stimuli rendered invisible by interocular competition can influence perception and behavior remains controversial. We studied whether suppressed and invisible symbolic and nonsymbolic numerical stimuli can elicit priming. First, we established that participants were objectively unable to discriminate numerical prime stimuli when interocular suppression rendered them invisible. Next, we asked participants to enumerate a visible target set of items after being exposed to a suppressed, invisible (nonsymbolic or symbolic) prime set. Both symbolic and nonsymbolic unconsciously perceived numerical primes induced robust priming effects that were specific to the numerical distance between the target and prime. Comparison with a no-prime condition revealed that primes larger than targets interfered with target enumeration and primes the same as or smaller than targets facilitated target enumeration. Taken together, our findings provide clear evidence for high-level processing of stimuli rendered invisible through interocular suppression.

TMS Over V5 Disrupts Motion Prediction

Given the vast amount of sensory information the brain has to deal with, predicting some of this information based on the current context is a resource-efficient strategy. The framework of predictive coding states that higher-level brain areas generate a predictive model to be communicated via feedback connections to early sensory areas. Here, we directly tested the necessity of a higher-level visual area, V5, in this predictive processing in the context of an apparent motion paradigm. We flashed targets on the apparent motion trace in-time or out-of-time with the predicted illusory motion token. As in previous studies, we found that predictable in-time targets were better detected than unpredictable out-of-time targets. However, when we applied functional magnetic resonance imaging-guided, double-pulse transcranial magnetic stimulation (TMS) over left V5 at 13–53 ms before target onset, the detection advantage of in-time targets was eliminated; this was not the case when TMS was applied over the vertex. Our results are causal evidence that V5 is necessary for a prediction effect, which has been shown to modulate V1 activity (Alink et al. 2010). Thus, our findings suggest that information processing between V5 and V1 is crucial for visual motion prediction, providing experimental support for the predictive coding framework.

Reduced cortico-motor facilitation in a normal sample with high traits of autism

Recent research in social neuroscience proposes a link between mirror neuron system (MNS) and social cognition. The MNS has been proposed to be the neural mechanism underlying action recognition and intention understanding and more broadly social cognition. Pre-motor MNS has been suggested to modulate the motor cortex during action observation. This modulation results in an enhanced cortico-motor excitability reflected in increased motor evoked potentials (MEPs) at the muscle of interest during action observation. Anomalous MNS activity has been reported in the autistic population whose social skills are notably impaired. It is still an open question whether traits of autism in the normal population are linked to the MNS functioning. We measured TMS-induced MEPs in normal individuals with high and low traits of autism as measured by the autistic quotient (AQ), while observing videos of hand or mouth actions, static images of a hand or mouth or a blank screen. No differences were observed between the two while they observed a blank screen. However participants with low traits of autism showed significantly greater MEP amplitudes during observation of hand/mouth actions relative to static hand/mouth stimuli. In contrast, participants with high traits of autism did not show such a MEP amplitude difference between observation of actions and static stimuli. These results are discussed with reference to MNS functioning.

Transfer of Predictive Signals Across Saccades

Predicting visual information facilitates efficient processing of visual signals. Higher visual areas can support the processing of incoming visual information by generating predictive models that are fed back to lower visual areas. Functional brain imaging has previously shown that predictions interact with visual input already at the level of the primary visual cortex (V1; Harrison et al., 2007; Alink et al., 2010). Given that fixation changes up to four times a second in natural viewing conditions, cortical predictions are effective in V1 only if they are fed back in time for the processing of the next stimulus and at the corresponding new retinotopic position. Here, we tested whether spatio-temporal predictions are updated before, during, or shortly after an inter-hemifield saccade is executed, and thus, whether the predictive signal is transferred swiftly across hemifields. Using an apparent motion illusion, we induced an internal motion model that is known to produce a spatio-temporal prediction signal along the apparent motion trace in V1 (Muckli et al., 2005; Alink et al., 2010). We presented participants with both visually predictable and unpredictable targets on the apparent motion trace. During the task, participants saccaded across the illusion whilst detecting the target. As found previously, predictable stimuli were detected more frequently than unpredictable stimuli. Furthermore, we found that the detection advantage of predictable targets is detectable as early as 50–100 ms after saccade offset. This result demonstrates the rapid nature of the transfer of a spatio-temporally precise predictive signal across hemifields, in a paradigm previously shown to modulate V1.

Predictive feedback to V1 dynamically updates with sensory input

Predictive coding theories propose that the brain creates internal models of the environment to predict upcoming sensory input. Hierarchical predictive coding models of vision postulate that higher visual areas generate predictions of sensory inputs and feed them back to early visual cortex. In V1, sensory inputs that do not match the predictions lead to amplified brain activation, but does this amplification process dynamically update to new retinotopic locations with eye-movements? We investigated the effect of eye-movements in predictive feedback using functional brain imaging and eye-tracking whilst presenting an apparent motion illusion. Apparent motion induces an internal model of motion, during which sensory predictions of the illusory motion feed back to V1. We observed attenuated BOLD responses to predicted stimuli at the new post-saccadic location in V1. Therefore, pre-saccadic predictions update their retinotopic location in time for post-saccadic input, validating dynamic predictive coding theories in V1.

Impaired presynaptic function and elimination of synapses at premature stages during postnatal development of the cerebellum in the absence of CALEB (CSPG5/neuroglycan C)

Chicken acidic leucine‐rich EGF‐like domain‐containing brain protein (CALEB), also known as chondroitin sulfate proteoglycan (CSPG)5 or neuroglycan C, is a neural chondroitin sulfate‐containing and epidermal growth factor (EGF)‐domain‐containing transmembrane protein that is implicated in synaptic maturation. Here, we studied the role of CALEB within the developing cerebellum. Adult CALEB‐deficient mice displayed impaired motor coordination in Rota‐Rod experiments. Analysis of the neuronal connectivity of Purkinje cells by patch‐clamp recordings demonstrated impairments of presynaptic maturation of inhibitory synapses. GABAergic synapses on Purkinje cells revealed decreased evoked amplitudes, altered paired‐pulse facilitation and reduced depression after repetitive stimulation at early postnatal but not at mature stages. Furthermore, the elimination of supernumerary climbing fiber synapses on Purkinje cells was found to occur at earlier developmental stages in the absence of CALEB. For example, at postnatal day 8 in wild‐type mice, 54% of Purkinje cells had three or more climbing fiber synapses in contrast to mutants where this number was decreased to less than 25%. The basic properties of the climbing fiber Purkinje cell synapse remained unaffected. Using Sholl analysis of dye‐injected Purkinje cells we revealed that the branching pattern of the dendritic tree of Purkinje cells was not impaired in CALEB‐deficient mice. The alterations observed by patch‐clamp recordings correlated with a specific pattern and timing of expression of CALEB in Purkinje cells, i.e. it is dynamically regulated during development from a high chondroitin sulfate‐containing form to a non‐chondroitin sulfate‐containing form. Thus, our results demonstrated an involvement of CALEB in the presynaptic differentiation of cerebellar GABAergic synapses and revealed a new role for CALEB in synapse elimination in Purkinje cells.