Sebastian Pannasch

Time course of information processing during scene perception: The relationship between saccade amplitude and fixation duration

The present study focuses on two aspects of the time course of visual information processing during the perception of natural scenes. The first aspect is the change of fixation duration and saccade amplitude during the first couple of seconds of the inspection period, as has been described by Buswell (1935), among others. This common effect suggests that the saccade amplitude and fixation duration are in some way controlled by the same mechanism. A simple exponential model containing two parameters can describe the phenomena quite satisfactorily. The parameters of the model show that saccade amplitude and fixation duration may be controlled by a common mechanism. The second aspect under scrutiny is the apparent lack of correlation between saccade amplitude and fixation duration (Viviani, 1990). The present study shows that a strong but nonlinear relationship between saccade amplitude and fixation duration does exist in picture viewing. A model, based on notions laid out by Findlay and Walkers (1999) model of saccade generation and on the idea of two modes of visual processing (Trevarthen, 1968), was developed to explain this relationship. The model both fits the data quite accurately and can explain a number of related phenomena.

Detection of isolated covert saccades with the video head impulse test in peripheral vestibular disorders

OBJECTIVES The function of the semicircular canal receptors and the pathway of the vestibulo-ocular-reflex (VOR) can be diagnosed with the clinical head impulse test (cHIT). Recently, the video head impulse test (vHIT) has been introduced but so far there is little clinical experience with the vHIT in patients with peripheral vestibular disorders. The aim of the study was to investigate the horizontal VOR (hVOR) by means of vHIT in peripheral vestibular disorders. METHODS Using the vHIT, we examined the hVOR in a group of 117 patients and a control group of 20 healthy subjects. The group of patients included vestibular neuritis (VN) (n=52), vestibular schwannoma (VS) (n=31), Ménières disease (MD) (n=22) and bilateral vestibulopathy (BV) (n=12). RESULTS Normal hVOR gain was at 0.96 ± 0.08, while abnormal hVOR gain was at 0.44 ± 0.20 (79.1% of all cases). An abnormal vHIT was found in VN (94.2%), VS (61.3%), MD (54.5%) and BV (91.7%). Three conditions of refixation saccades occurred frequently in cases with abnormal hVOR: isolated covert saccades (13.7%), isolated overt saccades (34.3%) and the combination of overt and covert saccades (52.0%). CONCLUSIONS The vHIT detects abnormal hVOR changes in the combination of gain assessment and refixation saccades. Since isolated covert saccades in hVOR changes can only be seen with vHIT, peripheral vestibular disorders are likely to be diagnosed incorrectly with the cHIT to a certain amount.

The effects of self-involvement on attention, arousal, and facial expression during social interaction with virtual others: A psychophysiological study

Abstract Social neuroscience has shed light on the underpinnings of understanding other minds. The current study investigated the effect of self-involvement during social interaction on attention, arousal, and facial expression. Specifically, we sought to disentangle the effect of being personally addressed from the effect of decoding the meaning of another persons facial expression. To this end, eye movements, pupil size, and facial electromyographic (EMG) activity were recorded while participants observed virtual characters gazing at them or looking at someone else. In dynamic animations, the virtual characters then displayed either socially relevant facial expressions (similar to those used in everyday life situations to establish interpersonal contact) or arbitrary facial movements. The results show that attention allocation, as assessed by eye-tracking measurements, was specifically related to self-involvement regardless of the social meaning being conveyed. Arousal, as measured by pupil size, was primarily related to perceiving the virtual characters gender. In contrast, facial EMG activity was determined by the perception of socially relevant facial expressions irrespective of whom these were directed towards.

Experiencing art: the influence of expertise and painting abstraction level

How does expertise influence the perception of representational and abstract paintings? We asked 20 experts on art history and 20 laypersons to explore and evaluate a series of paintings ranging in style from representational to abstract in five categories. We compared subjective esthetic judgments and emotional evaluations, gaze patterns, and electrodermal reactivity between the two groups of participants. The level of abstraction affected esthetic judgments and emotional valence ratings of the laypersons but had no effect on the opinions of the experts: the laypersons’ esthetic and emotional ratings were highest for representational paintings and lowest for abstract paintings, whereas the opinions of the experts were independent of the abstraction level. The gaze patterns of both groups changed as the level of abstraction increased: the number of fixations and the length of the scanpaths increased while the duration of the fixations decreased. The viewing strategies – reflected in the target, location, and path of the fixations – however indicated that experts and laypersons paid attention to different aspects of the paintings. The electrodermal reactivity did not vary according to the level of abstraction in either group but expertise was reflected in weaker responses, compared with laypersons, to information received about the paintings.

Virtual friend or threat? The effects of facial expression and gaze interaction on psychophysiological responses and emotional experience

The present study aimed to investigate the impact of facial expression, gaze interaction, and gender on attention allocation, physiological arousal, facial muscle responses, and emotional experience in simulated social interactions. Participants viewed animated virtual characters varying in terms of gender, gaze interaction, and facial expression. We recorded facial EMG, fixation duration, pupil size, and subjective experience. Subjects rapid facial reactions (RFRs) differentiated more clearly between the characters happy and angry expression in the condition of mutual eye-to-eye contact. This finding provides evidence for the idea that RFRs are not simply motor responses, but part of an emotional reaction. Eye movement data showed that fixations were longer in response to both angry and neutral faces than to happy faces, thereby suggesting that attention is preferentially allocated to cues indicating potential threat during social interaction.

The omnipresent prolongation of visual fixations: saccades are inhibited by changes in situation and in subject's activity

Presenting a distractor prolongs not only saccadic reaction times in paced tasks but also fixation durations in unpaced tasks. To investigate whether the effect of a distractor is a pure optomotor reflex, we used both visual and auditory distractors in an unpaced picture-viewing paradigm. Results show a distractor effect for both modalities. Analysis of data from previous studies showed similar effects, even in amodal shifts of attention. These findings challenge the hypothesis that the effect is modality-specific and suggest that the distractor effect may be another expression of the orienting reflex.

Feature-Specific Information Processing Precedes Concerted Activation in Human Visual Cortex

Current knowledge about the precise timing of visual input to the cortex relies largely on spike timings in monkeys and evoked-response latencies in humans. However, quantifying the activation onset does not unambiguously describe the timing of stimulus-feature-specific information processing. Here, we investigated the information content of the early human visual cortical activity by decoding low-level visual features from single-trial magnetoencephalographic (MEG) responses. MEG was measured from nine healthy subjects as they viewed annular sinusoidal gratings (spanning the visual field from 2 to 10° for a duration of 1 s), characterized by spatial frequency (0.33 cycles/degree or 1.33 cycles/degree) and orientation (45° or 135°); gratings were either static or rotated clockwise or anticlockwise from 0 to 180°. Time-resolved classifiers using a 20 ms moving window exceeded chance level at 51 ms (the later edge of the window) for spatial frequency, 65 ms for orientation, and 98 ms for rotation direction. Decoding accuracies of spatial frequency and orientation peaked at 70 and 90 ms, respectively, coinciding with the peaks of the onset evoked responses. Within-subject time-insensitive pattern classifiers decoded spatial frequency and orientation simultaneously (mean accuracy 64%, chance 25%) and rotation direction (mean 82%, chance 50%). Classifiers trained on data from other subjects decoded the spatial frequency (73%), but not the orientation, nor the rotation direction. Our results indicate that unaveraged brain responses contain decodable information about low-level visual features already at the time of the earliest cortical evoked responses, and that representations of spatial frequency are highly robust across individuals.

On the control of visual fixation durations in free viewing of complex images.

The mechanisms for the substantial variation in the durations of visual fixations in scene perception are not yet well understood. During free viewing of paintings, gaze-contingent irrelevant distractors (Exp. 1) and non-gaze-related time-locked display changes (Exp. 2) were presented. We demonstrated that any visual change—its onset and offset—prolongs the ongoing fixation (i.e., delays the following saccade), strongly suggesting that fixation durations are under the direct control of the stimulus information. The strongest influence of distraction was observed for fixations preceded by saccades within the parafoveal range (<5° of visual angle). We assume that these fixations contribute to the focal in contrast to the ambient mode of attention (Pannasch & Velichkovsky, Visual Cognition, 17, 1109–1131, 2009; Velichkovsky, Memory, 10, 405–419, 2002). Recent findings about two distinct “subpopulations of fixations,” one under the direct and another under the indirect control of stimulation (e.g., Henderson & Smith, Visual Cognition, 17, 1055–1082, 2009), are reconsidered in view of these results.

Distractor effect and saccade amplitudes: Further evidence on different modes of processing in free exploration of visual images

In view of a variety of everyday tasks, it is highly implausible that all visual fixations fulfil the same role. Earlier we demonstrated that a combination of fixation duration and amplitude of related saccades strongly correlates with the probability of correct recognition of objects and events both in static and in dynamic scenes (Velichkovsky, Joos, Helmert, & Pannasch, 2005; Velichkovsky, Rothert, Kopf, Dornhoefer, & Joos, 2002). In the present study, this observation is extended by measuring the amount of the distractor effect (characterized as a prolongation of visual fixation after a sudden change in stimulation; see Pannasch, Dornhoefer, Unema, & Velichkovsky, 2001) in relation to amplitudes of the preceding saccade. In Experiment 1, it is shown that retinotopically identical visual events occurring 100 ms after the onset of a fixation have significantly less influence on fixation duration if the amplitude of the previous saccade exceeds the parafoveal range (set on 5° of arc). Experiment 2 demonstrates that this difference diminishes for distractors of obvious biological value such as looming motion patterns. In Experiment 3, we show that saccade amplitudes influence visual but not acoustic or haptic distractor effects. These results suggest an explanation in terms of a shifting balance of at least two modes of visual processing in free viewing of complex visual images.

The maturation of eye movement behavior: Scene viewing characteristics in children and adults

While the close link between eye movements and visual attention has often been demonstrated, recently distinct attentional modes have been associated with specific eye movement patterns. The ambient mode-serving the localization of objects and dominating early scene inspection-is expressed by short fixations and large saccade amplitudes. The focal mode-associated with the identification of object details and dominating later stages of scene exploration-is indicated by longer fixations embedded in short saccades. The relationship between these processing modes and eye movement characteristics has so far only been examined in adults. While studies in children revealed a maturation of oculomotor behavior up to adolescence, developmental aspects of the processing modes are still unknown. Here we explored these mechanisms by comparing eye movements during the inspection of naturalistic scenes. Therefore, gaze behavior from adults and children in four different age groups (2, 4-6, 6-8, 8-10years old) was examined. We found a general effect of age, revealing that with age fixation durations decrease and saccade amplitudes increase. However, in all age groups fixations were shorter and saccades were longer at the beginning of scene inspection but fixations became longer and saccades became shorter over time. While saliency influenced eye guidance in the two youngest groups over the full inspection period, for the older groups this influence was found only at the beginning of scene inspection. The results reveal indications for ambient and focal processing strategies for as early as 2 years of age.