Jacob Jolij

Long-lasting modulation of feature integration by transcranial magnetic stimulation

The human brain analyzes a visual object first by basic feature detectors. On the objects way to a conscious percept, these features are integrated in subsequent stages of the visual hierarchy. The time course of this feature integration is largely unknown. To shed light on the temporal dynamics of feature integration, we applied transcranial magnetic stimulation (TMS) to a feature fusion paradigm. In feature fusion, two stimuli which differ in one feature are presented in rapid succession such that they are not perceived individually but as one single stimulus only. The fused percept is an integration of the features of both stimuli. Here, we show that TMS can modulate this integration for a surprisingly long period of time, even though the individual stimuli themselves are not consciously perceived. Hence, our results reveal a long-lasting integration process of unconscious feature traces.

Feedforward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging

In texture segregation, an example of scene segmentation, we can discern two different processes: texture boundary detection and subsequent surface segregation [Lamme, V. A. F., Rodriguez-Rodriguez, V., & Spekreijse, H. Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cerebral Cortex, 9, 406413, 1999]. Neural correlates of texture boundary detection have been found in monkey V1 [Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378, 492496, 1995; Grosof, D. H., Shapley, R. M., & Hawken, M. J. Macaque-V1 neurons can signal illusory contours. Nature, 365, 550552, 1993], but whether surface segregation occurs in monkey V1 [Rossi, A. F., Desimone, R., & Ungerleider, L. G. Contextual modulation in primary visual cortex of macaques. Journal of Neuroscience, 21, 16981709, 2001; Lamme, V. A. F. The neurophysiology of figure ground segregation in primary visual-cortex. Journal of Neuroscience, 15, 16051615, 1995], and whether boundary detection or surface segregation signals can also be measured in human V1, is more controversial [Kastner, S., De Weerd, P., & Ungerleider, L. G. Texture segregation in the human visual cortex: A functional MRI study. Journal of Neurophysiology, 83, 24532457, 2000]. Here we present electroencephalography (EEG) and functional magnetic resonance imaging data that have been recorded with a paradigm that makes it possible to differentiate between boundary detection and scene segmentation in humans. In this way, we were able to show with EEG that neural correlates of texture boundary detection are first present in the early visual cortex around 92 msec and then spread toward the parietal and temporal lobes. Correlates of surface segregation first appear in temporal areas (around 112 msec) and from there appear to spread to parietal, and back to occipital areas. After 208 msec, correlates of surface segregation and boundary detection also appear in more frontal areas. Blood oxygenation level-dependent magnetic resonance imaging results show correlates of boundary detection and surface segregation in all early visual areas including V1. We conclude that texture boundaries are detected in a feedforward fashion and are represented at increasing latencies in higher visual areas. Surface segregation, on the other hand, is represented in reverse hierarchical fashion and seems to arise from feedback signals toward early visual areas such as V1.

Figure-ground segregation requires two distinct periods of activity in V1: A transcranial magnetic stimulation study

Discriminating objects from their surroundings by the visual system is known as figure–ground segregation. This process entails two different subprocesses: boundary detection and subsequent surface segregation or ‘filling in’. In this study, we used transcranial magnetic stimulation to test the hypothesis that temporally distinct processes in V1 and related early visual areas such as V2 or V3 are causally related to the process of figure–ground segregation. Our results indicate that correct discrimination between two visual stimuli, which relies on figure–ground segregation, requires two separate periods of information processing in the early visual cortex: one around 130–160 ms and the other around 250–280 ms.

Investigating Emotional Top Down Modulation of Ambiguous Faces by Single Pulse TMS on Early Visual Cortices

Top-down processing is a mechanism in which memory, context and expectation are used to perceive stimuli. For this study we investigated how emotion content, induced by music mood, influences perception of happy and sad emoticons. Using single pulse TMS we stimulated right occipital face area (rOFA), primary visual cortex (V1) and vertex while subjects performed a face-detection task and listened to happy and sad music. At baseline, incongruent audio-visual pairings decreased performance, demonstrating dependence of emotion while perceiving ambiguous faces. However, performance of face identification decreased during rOFA stimulation regardless of emotional content. No effects were found between Cz and V1 stimulation. These results suggest that while rOFA is important for processing faces regardless of emotion, V1 stimulation had no effect. Our findings suggest that early visual cortex activity may not integrate emotional auditory information with visual information during emotion top-down modulation of faces.

Act quickly, decide later: Long-latency visual processing underlies perceptual decisions but not reflexive behavior

Humans largely guide their behavior by their visual representation of the world. Recent studies have shown that visual information can trigger behavior within 150 msec, suggesting that visually guided responses to external events, in fact, precede conscious awareness of those events. However, is such a view correct? By using a texture discrimination task, we show that the brain relies on long-latency visual processing in order to guide perceptual decisions. Decreasing stimulus saliency leads to selective changes in long-latency visually evoked potential components reflecting scene segmentation. These latency changes are accompanied by almost equal changes in simple RTs and points of subjective simultaneity. Furthermore, we find a strong correlation between individual RTs and the latencies of scene segmentation related components in the visually evoked potentials, showing that the processes underlying these late brain potentials are critical in triggering a response. However, using the same texture stimuli in an antisaccade task, we found that reflexive, but erroneous, prosaccades, but not antisaccades, can be triggered by earlier visual processes. In other words: The brain can act quickly, but decides late. Differences between our study and earlier findings suggesting that action precedes conscious awareness can be explained by assuming that task demands determine whether a fast and unconscious, or a slower and conscious, representation is used to initiate a visually guided response.

Trial History Effects in Stroop Task Performance Are Independent of Top-Down Control

In this study we sought to elucidate what mechanisms underlie the effects of trial history on information processing. We explicitly focused on the contribution of conflict control and S-R binding to sequential trial effects. Performance and brain activity were measured during two hours of continuous Stroop task performance. Mental fatigue, known to influence top-down processing, was used to elucidate separate effects via top-down and bottom-up mechanisms. Here we confirm that performance in the Stroop task is indeed strongly modulated by stimulus history. Performance was affected by the kind of advance information available; dependent on this information adjustments were made, resulting in differential effects of cognitive conflict, and S-R binding on subsequent performance. The influence of mental fatigue on information processing was mainly related to general effects on attention.

What you may not see might slow you down anyway: masked images and driving.

Many theories of driver behaviour suggest that unconscious or implicit emotions play a functional role in the shaping and control of behaviour. This has not been experimentally tested however. Therefore, in this study the effects of emotive masked images on driver behaviour were examined. While driving a simulator, participants were repeatedly exposed to negative or neutral emotionally laden target images that were sandwich masked by emotionally neutral images. These images were encountered across two different trials each of which consisted of 3–4 minutes of driving on a rural road. The results indicate an effect of the negative target images primarily in reducing the extent of familiarisation occurring between the first and second experimental drives. This is evident in a reduced decrease in heart rate and a reduced increase in high band heart rate variability and actual travelling speed from the first to second drives if the negative target image was presented in the second drive. In addition to these findings there was no clear effect of the target image on subjective ratings of effort or feelings of risk. There was however an effect of gender, with the majority of the effects found in the study being limited to the larger female dataset. These findings suggest that unconscious or implicit emotional stimuli may well influence driver behaviour without explicit awareness.

Why do we see what's not there?

Conscious perception is not the result of passively processing sensory input, but to large extent of active inference based on previous knowledge. This process of inference does go astray from time to time, and may lead to illusory perception: sometimes people see things that are not there. In a recent study we have shown that this inference may also be influenced by mood. Here we present some additional data, suggesting that illusory percepts are the result of increased top-down processing, which is normally helpful in detecting real stimuli. Finally, we speculate on a possible function of mood-dependent modulation of this top-down processing in social perception in particular.

Testing the potential paradoxes in “retrocausal” phenomena

Discussions with regard to potential paradoxes arising from “retrocausal” phenomena have been purely theoretical because so far no empirical effects had been established that allowed for empirical exploration of these potential paradoxes.In this article we describe three human experiments that showed clear “retrocausal” effects. In these neuropsychological, so-called, face-detection experiments, consisting of hundreds of trials per participant, we use brain signals to predict an upcoming random stimulus. The binary random decision, corresponding to showing a noisy cartoon face or showing only noise on a display with equal probability is taken after the brain signals have been measured. The prediction accuracy ranges from 50.5-56.5% for the 3 experiments where chance performance would be 50%.The prediction algorithm is based on a template constructed out of all the pre-stimulus brain signals obtained in other trials of that particular participant. This approach thus controls for individual difference in brain functioning. Subsequently we describe an experiment based upon these findings where the predictive information is used in part of the trials to determine the stimulus rather than randomly select that stimulus. In those trials we analyze what the brain signals tell us what the future stimulus would be and then we reverse the actual future that is presented on the display. This is a ‘bilking’ condition. We analyze what the consequence of the introduction of this bilking condition is on the accuracy of the remaining (normal) trials and, following a suggestion inferred from Thorne et al, we also check what the effect is on the random decision to either bilk or not bilk the specific trial. The bilking experiment is in progress and the results so far do not allow for conclusions and are presented only as an illustration.Discussions with regard to potential paradoxes arising from “retrocausal” phenomena have been purely theoretical because so far no empirical effects had been established that allowed for empirical exploration of these potential paradoxes.In this article we describe three human experiments that showed clear “retrocausal” effects. In these neuropsychological, so-called, face-detection experiments, consisting of hundreds of trials per participant, we use brain signals to predict an upcoming random stimulus. The binary random decision, corresponding to showing a noisy cartoon face or showing only noise on a display with equal probability is taken after the brain signals have been measured. The prediction accuracy ranges from 50.5-56.5% for the 3 experiments where chance performance would be 50%.The prediction algorithm is based on a template constructed out of all the pre-stimulus brain signals obtained in other trials of that particular participant. This approach thus controls for individual difference in br...

Brain Potentials Highlight Stronger Implicit Food Memory for Taste than Health and Context Associations.

Increasingly consumption of healthy foods is advised to improve population health. Reasons people give for choosing one food over another suggest that non-sensory features like health aspects are appreciated as of lower importance than taste. However, many food choices are made in the absence of the actual perception of a food’s sensory properties, and therefore highly rely on previous experiences of similar consumptions stored in memory. In this study we assessed the differential strength of food associations implicitly stored in memory, using an associative priming paradigm. Participants (N = 30) were exposed to a forced-choice picture-categorization task, in which the food or non-food target images were primed with either non-sensory or sensory related words. We observed a smaller N400 amplitude at the parietal electrodes when categorizing food as compared to non-food images. While this effect was enhanced by the presentation of a food-related word prime during food trials, the primes had no effect in the non-food trials. More specifically, we found that sensory associations are stronger implicitly represented in memory as compared to non-sensory associations. Thus, this study highlights the neuronal mechanisms underlying previous observations that sensory associations are important features of food memory, and therefore a primary motive in food choice.