Niklas Wilming

Investigating task-dependent top-down effects on overt visual attention

Different tasks can induce different viewing behavior, yet it is still an open question how or whether at all high-level task information interacts with the bottom-up processing of stimulus-related information. Two possible causal routes are considered in this paper. Firstly, the weak top-down hypothesis, according to which top-down effects are mediated by changes of feature weights in the bottom-up system. Secondly, the strong top-down hypothesis, which proposes that top-down information acts independently of the bottom-up process. To clarify the influences of these different routes, viewing behavior was recorded on web pages for three different tasks: free viewing, content awareness, and information search. The data reveal significant task-dependent differences in viewing behavior that are accompanied by minor changes in feature-fixation correlations. Extensive computational modeling shows that these small but significant changes are insufficient to explain the observed differences in viewing behavior. Collectively, the results show that task-dependent differences in the current setting are not mediated by a reweighting of features in the bottom-up hierarchy, ruling out the weak top-down hypothesis. Consequently, the strong top-down hypothesis is the most viable explanation for the observed data.

Saccadic momentum and facilitation of return saccades contribute to an optimal foraging strategy.

The interest in saccadic IOR is funneled by the hypothesis that it serves a clear functional purpose in the selection of fixation points: the facilitation of foraging. In this study, we arrive at a different interpretation of saccadic IOR. First, we find that return saccades are performed much more often than expected from the statistical properties of saccades and saccade pairs. Second, we find that fixation durations before a saccade are modulated by the relative angle of the saccade, but return saccades show no sign of an additional temporal inhibition. Thus, we do not find temporal saccadic inhibition of return. Interestingly, we find that return locations are more salient, according to empirically measured saliency (locations that are fixated by many observers) as well as stimulus dependent saliency (defined by image features), than regular fixation locations. These results and the finding that return saccades increase the match of individual trajectories with a grand total priority map evidences the return saccades being part of a fixation selection strategy that trades off exploration and exploitation.

Dissociation between saliency signals and activity in early visual cortex

Saliency is a measure that describes how attention is guided by local stimulus properties. Some hypotheses assign its computation to specific topographically organized areas of early human visual cortex. However, in most stimuli, saliency is correlated with luminance contrast, which in turn is known to correlate with activity in these early areas. Thus, any observed correlation of local activity with saliency might be due to the area encoding luminance contrast. Here we disentangle encoding of local luminance contrast and saliency by using stimuli where the two properties are uncorrelated. First, we conducted an eye-tracking study to verify that both negative and positive contrast modifications located in individual quadrants of the visual field increase saliency. Second, subjects viewed identical stimuli while fMRI signals were recorded. We find that positive contrast modifications induce a robust increase of activity in V1-V3 and hV4. However, negative contrast modifications lead to a reduced (V1, V2) or comparable (V3, hV4) activity level compared to unmodified quadrants. Furthermore, even with linear multivariate pattern-classification techniques, it is not possible to decode the location of the salient quadrant independent of the type of the contrast modification. Instead, decoding of the contrast-modified location is only possible separately for the two modification types in V1-V3. These findings suggest that the BOLD activity in V1-V3 is dominated by contrast-dependent processes and does not include the contrast invariance necessary for the computation of feature-invariant saliency.

Predictions in the light of your own action repertoire as a general computational principle.

We argue that brains generate predictions only within the constraints of the action repertoire. This makes the computational complexity tractable and fosters a step-by-step parallel development of sensory and motor systems. Hence, it is more of a benefit than a literal constraint and may serve as a universal normative principle to understand sensorimotor coupling and interactions with the world.

An extensive dataset of eye movements during viewing of complex images.

We present a dataset of free-viewing eye-movement recordings that contains more than 2.7 million fixation locations from 949 observers on more than 1000 images from different categories. This dataset aggregates and harmonizes data from 23 different studies conducted at the Institute of Cognitive Science at Osnabrück University and the University Medical Center in Hamburg-Eppendorf. Trained personnel recorded all studies under standard conditions with homogeneous equipment and parameter settings. All studies allowed for free eye-movements, and differed in the age range of participants (~7–80 years), stimulus sizes, stimulus modifications (phase scrambled, spatial filtering, mirrored), and stimuli categories (natural and urban scenes, web sites, fractal, pink-noise, and ambiguous artistic figures). The size and variability of viewing behavior within this dataset presents a strong opportunity for evaluating and comparing computational models of overt attention, and furthermore, for thoroughly quantifying strategies of viewing behavior. This also makes the dataset a good starting point for investigating whether viewing strategies change in patient groups.

STN-DBS Reduces Saccadic Hypometria but Not Visuospatial Bias in Parkinson's Disease Patients.

In contrast to its well-established role in alleviating skeleto-motor symptoms in Parkinsons disease, little is known about the impact of deep brain stimulation (DBS) of the subthalamic nucleus (STN) on oculomotor control and attention. Eye-tracking data of 17 patients with left-hemibody symptom onset was compared with 17 age-matched control subjects. Free-viewing of natural images was assessed without stimulation as baseline and during bilateral DBS. To examine the involvement of ventral STN territories in oculomotion and spatial attention, we employed unilateral stimulation via the left and right ventralmost contacts respectively. When DBS was off, patients showed shorter saccades and a rightward viewing bias compared with controls. Bilateral stimulation in therapeutic settings improved saccadic hypometria but not the visuospatial bias. At a group level, unilateral ventral stimulation yielded no consistent effects. However, the evaluation of electrode position within normalized MNI coordinate space revealed that the extent of early exploration bias correlated with the precise stimulation site within the left subthalamic area. These results suggest that oculomotor impairments “but not higher-level exploration patterns” are effectively ameliorable by DBS in therapeutic settings. Our findings highlight the relevance of the STN topography in selecting contacts for chronic stimulation especially upon appearance of visuospatial attention deficits.

Entorhinal cortex receptive fields are modulated by spatial attention, even without movement

Grid cells in the entorhinal cortex allow for the precise decoding of position in space. Along with potentially playing an important role in navigation, grid cells have recently been hypothesized to make a general contribution to mental operations. A prerequisite for this hypothesis is that grid cell activity does not critically depend on physical movement. Here, we show that movement of covert attention, without any physical movement, also elicits spatial receptive fields with a triangular tiling of space. In monkeys trained to maintain central fixation while covertly attending to a stimulus moving in the periphery we identified a significant population (20/141, 14% neurons at a FDR <5%) of entorhinal cells with spatially structured receptive fields. This contrasts with recordings obtained in the hippocampus, where grid-like representations were not observed. Our results provide evidence that neurons in macaque entorhinal cortex do not rely on physical movement.

Differential Contribution of Low- and High-level Image Content to Eye Movements in Monkeys and Humans

Abstract Oculomotor selection exerts a fundamental impact on our experience of the environment. To better understand the underlying principles, researchers typically rely on behavioral data from humans, and electrophysiological recordings in macaque monkeys. This approach rests on the assumption that the same selection processes are at play in both species. To test this assumption, we compared the viewing behavior of 106 humans and 11 macaques in an unconstrained free‐viewing task. Our data‐driven clustering analyses revealed distinct human and macaque clusters, indicating species‐specific selection strategies. Yet, cross‐species predictions were found to be above chance, indicating some level of shared behavior. Analyses relying on computational models of visual saliency indicate that such cross‐species commonalities in free viewing are largely due to similar low‐level selection mechanisms, with only a small contribution by shared higher level selection mechanisms and with consistent viewing behavior of monkeys being a subset of the consistent viewing behavior of humans.

Learning and Adaptation of Sensorimotor Contingencies: Prism-Adaptation, a Case Study

This paper focuses on learning and adaptation of sensorimotor contingencies. As a specific case, we investigate the application of prism glasses, which change visual-motor contingencies. After an initial disruption of sensorimotor coordination, humans quickly adapt. However, scope and generalization of that adaptation is highly dependent on the type of feedback and exhibits markedly different degrees of generalization. We apply a model with a specific interaction of forward and inverse models to a robotic setup and subject it to the identical experiments that have been used on previous human psychophysical studies. Our model demonstrates both locally specific adaptation and global generalization in accordance with the psychophysical experiments. These results emphasize the role of the motor system for sensory processes and open an avenue to improve on sensorimotor processing.

Aversive Learning Changes Face-Viewing Strategies, as Revealed by Model-Based Fixation-Pattern Similarity Analysis

Animals can effortlessly adapt their behavior by generalizing from past experiences, and avoid harm in novel aversive situations. In our current understanding, the perceptual similarity between learning and generalization samples is viewed as one major factor driving aversive generalization. Alternatively, the threat-prediction account proposes that perceptual similarity should lead to generalization to the extent it predicts harmful outcomes. We tested these views using a two-dimensional perceptual continuum of faces. During learning, one face is conditioned to predict a harmful event, whereas the most dissimilar face stays neutral; introducing an adversity gradient defined only along one dimension. Learning changed the way how humans sampled information during viewing of faces. These occurred specifically along the adversity gradient leading to an increased dissimilarity of eye-movement patterns along the threat-related dimension. This provides evidence for the threat-prediction account of generalization, which conceives perceptual factors to be relevant to the extent they predict harmful outcomes.To avoid costly situations animals must be able to rapidly predict imminent threat based on past experience and present noisy sensory evidence. We aimed to characterize to what extent active exploration strategies, can be adaptively tuned to achieve this goal. We measured how eye-movement patterns on 8 faces, organized along a circular similarity continuum, were modified after aversive learning and generalization. Using model-based Fixation Similarity Analysis, we characterized how similarity relationships between exploration strategies were modified after volunteers learnt to pair one face (CS+) with a mild electric shock. Initially, viewing patterns reflected the circular physical similarity structure of different faces, indicating that eye-movements were guided by subtle differences between faces. Following aversive learning, the similarity structure of exploration patterns became elliptical and stretched along the adversity gradient defined by the CS+ and the most dissimilar neutral face (CS-), indicating that exploration patterns on these faces were most dissimilar. These findings suggest that the need to predict adversity introduces substantial remodelling of exploration patterns and influences entire sensorimotor loops by selecting fixation locations that can help in categorizing stimuli as aversive or safe.