Wolfgang Einhäuser

Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry

During sustained viewing of an ambiguous stimulus, an individuals perceptual experience will generally switch between the different possible alternatives rather than stay fixed on one interpretation (perceptual rivalry). Here, we measured pupil diameter while subjects viewed different ambiguous visual and auditory stimuli. For all stimuli tested, pupil diameter increased just before the reported perceptual switch and the relative amount of dilation before this switch was a significant predictor of the subsequent duration of perceptual stability. These results could not be explained by blink or eye-movement effects, the motor response or stimulus driven changes in retinal input. Because pupil dilation reflects levels of norepinephrine (NE) released from the locus coeruleus (LC), we interpret these results as suggestive that the LC–NE complex may play the same role in perceptual selection as in behavioral decision making.

Does luminance-contrast contribute to a saliency map for overt visual attention?

In natural environments, humans select a subset of visual stimuli by directing their gaze to locations attended. In previous studies it has been found that at fixation points luminance‐contrast is higher than average. This led to the hypothesis that luminance‐contrast makes a major contribution to a saliency map of visual overt attention, consistent with a computation of stimulus saliency in early visual cortical areas. We re‐evaluate this hypothesis by using natural and modified natural images to uncover the causal effects of luminance‐contrast to human overt visual attention and: (i) we confirm that when viewing natural images, contrasts are elevated at fixation points. This, however, only holds for low spatial frequencies and in a limited temporal window after stimulus onset; (ii) however, despite this correlation between overt attention and luminance‐contrast, moderate modifications of contrast in natural images do not measurably affect the selection of fixation points. Furthermore, strong local reductions of luminance‐contrast do not repel but attract fixation; (iii) neither contrast nor contrast modification is correlated to fixation duration; and (iv), even the moderate contrast modifications used fall into the physiologically relevant range, and subjects are well able to detect them in a forced choice paradigm. In summary, no causal contribution of luminance‐contrast to a saliency map of human overt attention is detectable. In conjunction with recent results on the relation of contrast sensitivity of neuronal activity to the level in the visual cortical hierarchy, the present study provides evidence that, for natural scenes, saliency is computed not early but late during processing.

Pupil Dilation Signals Surprise: Evidence for Noradrenaline’s Role in Decision Making

Our decisions are guided by the rewards we expect. These expectations are often based on incomplete knowledge and are thus subject to uncertainty. While the neurophysiology of expected rewards is well understood, less is known about the physiology of uncertainty. We hypothesize that uncertainty, or more specifically errors in judging uncertainty, are reflected in pupil dilation, a marker that has frequently been associated with decision making, but so far has remained largely elusive to quantitative models. To test this hypothesis, we measure pupil dilation while observers perform an auditory gambling task. This task dissociates two key decision variables – uncertainty and reward – and their errors from each other and from the act of the decision itself. We first demonstrate that the pupil does not signal expected reward or uncertainty per se, but instead signals surprise, that is, errors in judging uncertainty. While this general finding is independent of the precise quantification of these decision variables, we then analyze this effect with respect to a specific mathematical model of uncertainty and surprise, namely risk and risk prediction error. Using this quantification, we find that pupil dilation and risk prediction error are indeed highly correlated. Under the assumption of a tight link between noradrenaline (NA) and pupil size under constant illumination, our data may be interpreted as empirical evidence for the hypothesis that NA plays a similar role for uncertainty as dopamine does for reward, namely the encoding of error signals.

The world from a cat’s perspective – statistics of natural videos

Abstract.The mammalian visual system is one of the most intensively investigated sensory systems. However, our knowledge of the typical input it is operating on is surprisingly limited. To address this issue, we seek to learn about the natural visual environment and the world as seen by a cat. With a CCD camera attached to their head, cats explore several outdoor environments and videos of natural stimuli are recorded from the animals’ perspective. The statistical analysis of these videos reveals several remarkable properties. First, we find an anisotropy of oriented contours with an enhanced occurrence of horizontal orientations, earlier described in the “oblique effect” as a predominance of the two cardinal orientations. Second, contrast is not elevated in the center of the images, suggesting different mechanisms of fixation point selection as compared to humans. Third, analyzing a sequence of images we find that the precise position of contours varies faster than their orientation. Finally, collinear contours prevail over parallel shifted contours, matching recent physiological and anatomical results. These findings demonstrate the rich structure of natural visual stimuli and its direct relation to extensively studied anatomical and physiological issues.

Learning the invariance properties of complex cells from their responses to natural stimuli

Neurons in primary visual cortex are typically classified as either simple or complex. Whereas simple cells respond strongly to grating and bar stimuli displayed at a certain phase and visual field location, complex cell responses are insensitive to small translations of the stimulus within the receptive field [ Hubel & Wiesel (1962)J. Physiol. (Lond.), 160, 106–154; Kjaer et al. (1997)J. Neurophysiol., 78, 3187–3197]. This constancy in the response to variations of the stimuli is commonly called invariance. Hubel and Wiesels classical model of the primary visual cortex proposes a connectivity scheme which successfully describes simple and complex cell response properties. However, the question as to how this connectivity arises during normal development is left open. Based on their work and inspired by recent physiological findings we suggest a network model capable of learning from natural stimuli and developing receptive field properties which match those of cortical simple and complex cells. Stimuli are drawn from videos obtained by a camera mounted to a cats head, so they should approximate the natural input to the cats visual system. The network uses a competitive scheme to learn simple and complex cell response properties. Employing delayed signals to learn connections between simple and complex cells enables the model to utilize temporal properties of the input. We show that the temporal structure of the input gives rise to the emergence and refinement of complex cell receptive fields, whereas removing temporal continuity prevents this processes. This model lends a physiologically based explanation of the development of complex cell invariance response properties.

Pupil Dilation Betrays the Timing of Decisions

The notion of “mind-reading” by carefully observing another individuals physiological responses has recently become commonplace in popular culture, particularly in the context of brain imaging. The question remains, however, whether outwardly accessible physiological signals indeed betray a decision before a person voluntarily reports it. In one experiment we asked observers to push a button at any time during a 10-s period (“immediate overt response”). In a series of three additional experiments observers were asked to select one number from five sequentially presented digits but concealed their decision until the trials end (“covert choice”). In these experiments observers either had to choose the digit themselves under conditions of reward and no reward, or were instructed which digit to select via an external cue provided at the time of the digit presentation. In all cases pupil dilation alone predicted the choice (timing of button response or chosen digit, respectively). Consideration of the average pupil-dilation responses, across all experiments, showed that this prediction of timing was distinct from a general arousal or reward-anticipation response. Furthermore, the pupil dilation appeared to reflect the post-decisional consolidation of the selected outcome rather than the pre-decisional cognitive appraisal component of the decision. Given the tight link between pupil dilation and norepinephrine levels during constant illumination, our results have implications beyond the tantalizing mind-reading speculations. These findings suggest that similar noradrenergic mechanisms may underlie the consolidation of both overt and covert decisions.

Differences of monkey and human overt attention under natural conditions

Rhesus monkeys are widely used as animal models of human attention. Such research rests upon the assumption that similar mechanisms underlie attention in both species. Here, we directly compare the influence of low-level stimulus features on overt attention in monkeys and humans under natural conditions. We recorded eye-movements in humans and rhesus monkeys during free-viewing of natural images. We find that intrinsic low-level features, such luminance-contrast, texture-contrast and saliency-as predicted by a standard model, are elevated at fixation points in the majority of images. These correlative effects are not significantly different between species. However, local image modifications affect both species differently: moderate modifications, which are in the range of natural fluctuations, attract overt attention in monkeys significantly stronger than they do in humans. In addition, humans show a higher inter-individual consistency regarding which locations they fixate than monkeys, in spite of the similarity for intrinsic low-level features. Taken together, these data demonstrate that-under natural conditions-low-level stimulus features affect attention in monkeys and humans differently.

Gaze allocation in natural stimuli: Comparing free exploration to head-fixed viewing conditions

“Natural” gaze is typically measured by tracking eye positions during scene presentation in laboratory settings. How informative are such investigations for real-world conditions? Using a mobile eyetracking setup (“EyeSeeCam”), we measure gaze during free exploration of various in- and outdoor environments, while simultaneously recording head-centred videos. Here, we replay these videos in a laboratory setup. Half of the laboratory observers view the movies continuously, half as sequences of static 1-second frames. We find a bias of eye position to the stimulus centre, which is strongest in the 1 s frame replay condition. As a consequence, interobserver consistency is highest in this condition, though not fully explained by spatial bias alone. This leaves room for image specific bottom-up models to predict gaze beyond generic biases. Indeed, the “saliency map” predicts eye position in all conditions, and best for continuous replay. Continuous replay predicts real-world gaze better than 1 s frame replay does. In conclusion, experiments and models benefit from preserving the spatial statistics and temporal continuity of natural stimuli to improve their validity for real-world gaze behaviour.

Spatial attention increases performance but not subjective confidence in a discrimination task.

Selective attention to a target yields faster and more accurate responses. Faster response times, in turn, are usually associated with increased subjective confidence. Could the decrease in reaction time in the presence of attention therefore simply reflect a shift toward more confident responses? We here addressed the extent to which attention modulates accuracy, processing speed, and confidence independently. To probe the effect of spatial attention on performance, we used two attentional manipulations of a visual orientation discrimination task. We demonstrate that spatial attention significantly increases accuracy, whereas subjective confidence measures reveal overconfidence in non-attended stimuli. At constant confidence levels, reaction times showed a significant decrease (by 15-49%, corresponding to 100-250 ms). This dissociation of objective performance and subjective confidence suggests that attention and awareness, as measured by confidence, are distinct, albeit related, phenomena.

Viewpoint-invariant learning and detection of human heads

We present a method to learn models of human heads for the purpose of detection from different viewing angles. We focus on a model where objects are represented as constellations of rigid features (parts). Variability is represented by a joint probability density function (PDF) on the shape of the constellation. In the first stage, the method automatically identifies distinctive features in the training set using an interest operator followed by vector quantization. The set of model parameters, including the shape PDF, is then learned using expectation maximization. Experiments show good generalization performance to novel viewpoints and unseen faces. Performance is above 90% correct with less than 1 s computation time per image.