Thierry Baccino

Methods for comparing scanpaths and saliency maps: strengths and weaknesses

In this article, we are interested in the computational modeling of visual attention. We report methods commonly used to assess the performance of these kinds of models. We survey the strengths and weaknesses of common assessment methods based on diachronic eye-tracking data. We then illustrate the use of some methods to benchmark computational models of visual attention.

E-readers and visual fatigue.

The mass digitization of books is changing the way information is created, disseminated and displayed. Electronic book readers (e-readers) generally refer to two main display technologies: the electronic ink (E-ink) and the liquid crystal display (LCD). Both technologies have advantages and disadvantages, but the question whether one or the other triggers less visual fatigue is still open. The aim of the present research was to study the effects of the display technology on visual fatigue. To this end, participants performed a longitudinal study in which two last generation e-readers (LCD, E-ink) and paper book were tested in three different prolonged reading sessions separated by - on average - ten days. Results from both objective (Blinks per second) and subjective (Visual Fatigue Scale) measures suggested that reading on the LCD (Kindle Fire HD) triggers higher visual fatigue with respect to both the E-ink (Kindle Paperwhite) and the paper book. The absence of differences between E-ink and paper suggests that, concerning visual fatigue, the E-ink is indeed very similar to the paper.

Automatic Stress Classification With Pupil Diameter Analysis

This article proposes a method based on wavelet transform and neural networks for relating pupillary behavior to psychological stress. The proposed method was tested by recording pupil diameter and electrodermal activity during a simulated driving task. Self-report measures were also collected. Participants performed a baseline run with the driving task only, followed by three stress runs where they were required to perform the driving task along with sound alerts, the presence of two human evaluators, and both. Self-reports and pupil diameter successfully indexed stress manipulation, and significant correlations were found between these measures. However, electrodermal activity did not vary accordingly. After training, the four-way parallel neural network classifier could guess whether a given unknown pupil diameter signal came from one of the four experimental trials with 79.2% precision. The present study shows that pupil diameter signal has good discriminating power for stress detection.

Effects of luminance and illuminance on visual fatigue and arousal during digital reading

We studied the effect of luminance and illuminance on visual fatigue and arousal.We employed an approach based on eye, performance and subjective measures.Higher levels of screen luminance increase visual fatigue.Higher levels of either screen luminance or ambient illuminance increase arousal.Findings might have practical implications for adaptive brightness solutions. We investigated the conjoint effect of screen luminance and ambient illuminance on visual fatigue and arousal during prolonged digital reading (one hour) by means of a multidimensional approach based on eye, performance and subjective measures. Two levels of screen luminance (low, high) and two levels of ambient illuminance (low, high) were tested in a 2×2 between-subjects design in which participants were arbitrarily allocated to four groups, one for each combined level of luminance and illuminance. Results showed that reading under high levels of screen luminance increases visual fatigue, as reflected by a decrease of eye blinks. Concerning arousal, exposure to higher levels of either luminance or illuminance increased alertness and performance. Faster saccades, increased reading speed and less microsaccades were found under high screen luminance. Fewer regressive saccades and shorter reaction times were observed under high ambient illuminance. However, the reason why some of these measures are sensitive to screen luminance while other to ambient illuminance remains unknown. These findings might have practical implications for the implementation of adaptive brightness solutions and for the online detection of both visual fatigue and arousal levels during digital reading.

New insights into ambient and focal visual fixations using an automatic classification algorithm

Overt visual attention is the act of directing the eyes toward a given area. These eye movements are characterised by saccades and fixations. A debate currently surrounds the role of visual fixations. Do they all have the same role in the free viewing of natural scenes? Recent studies suggest that at least two types of visual fixations exist: focal and ambient. The former is believed to be used to inspect local areas accurately, whereas the latter is used to obtain the context of the scene. We investigated the use of an automated system to cluster visual fixations in two groups using four types of natural scene images. We found new evidence to support a focal-ambient dichotomy. Our data indicate that the determining factor is the saccade amplitude. The dependence on the low-level visual features and the time course of these two kinds of visual fixations were examined. Our results demonstrate that there is an interplay between both fixation populations and that focal fixations are more dependent on low-level visual features than are ambient fixations.

Affective processing in natural scene viewing: valence and arousal interactions in eye-fixation-related potentials.

Attention is drawn to emotionally salient stimuli. The present study investigates processing of emotionally salient regions during free viewing of emotional scenes that were categorized according to the two-dimensional model comprising of valence (unpleasant, pleasant) and arousal (high, low). Recent studies have reported interactions between these dimensions, indicative of stimulus-evoked approach or withdrawal tendencies. We addressed the valence and arousal effects when emotional items were embedded in complex real-world scenes by analyzing both eye movement behavior and eye-fixation-related potentials (EFRPs) time-locked to the critical event of fixating the emotionally salient items for the first time. Both data sets showed an interaction between the valence and arousal dimensions. First, the fixation rates and gaze durations on emotionally salient regions were enhanced for unpleasant versus pleasant images in the high arousal condition. In the low arousal condition, both measures were enhanced for pleasant versus unpleasant images. Second, the EFRP results at 140-170 ms [P2] over the central site showed stronger responses for high versus low arousing images in the unpleasant condition. In addition, the parietal LPP responses at 400-500 ms post-fixation were enhanced for stimuli reflecting congruent stimulus dimensions, that is, stronger responses for high versus low arousing images in the unpleasant condition and stronger responses for low versus high arousing images in the pleasant condition. The present findings support the interactive two-dimensional approach, according to which the integration of valence and arousal recruits brain regions associated with action tendencies of approach or withdrawal.

Rapid serial visual presentation in reading

With personal computing devices getting smaller, reading is called into question.Rapid serial visual presentation (RSVP) seems to be one of the main alternatives.We studied the effects of a RSVP app (i.e. Spritz) on comprehension and visual fatigue.We found that Spritz impairs literal comprehension and increases visual fatigue.Findings might have practical implications for implementing alternative solutions. In the era of small screens, traditional reading (i.e. left-to-right, top-to-bottom) is called into question and rapid serial visual presentation (RSVP) represents one of the main alternatives. RSVP consists of displaying in sequential order one or more words at a time, thus minimizing saccades and eye blinks. Recently, a RSVP application has received a lot of media attention: it is the case of Spritz. According to Spritzs developers, the elimination of saccades should reduce visual fatigue and improve comprehension. In this study, we had people read on a computer screen a selected part of a book either with Spritz or in the traditional way. Results seem to contradict these claims. The fact that Spritz suppresses parafoveal processing and regressions (i.e. rereadings of words) negatively affected literal comprehension. Furthermore, the important reduction of eye blinks observed for Spritz might contribute to the increase of visual fatigue.

Sight-reading expertise: cross-modality integration investigated using eye tracking.

It is often said that experienced musicians are capable of hearing what they read (and vice versa). This suggests that they are able to process and to integrate multimodal information. The present study investigates this issue with an eye-tracking technique. Two groups of musicians chosen on the basis of their level of expertise (experts, non-experts) had to read excerpts of poorly-known classical piano music and play them on a keyboard. The experiment was run in two consecutive phases during which each excerpt was (1) read without playing and (2) sight-read (read and played). In half the conditions, the participants heard the music before the reading phases. The excerpts contained suggested fingering of variable difficulty (difficult, easy, or no fingering). Analyses of first-pass fixation duration, second-pass fixation duration, probability of re-fixation, and playing mistakes validated the hypothesized modal independence of information among expert musicians as compared to non-experts. The results are discussed in terms of the processing cues and retrieval structures postulated by Ericsson and Kintsch (1995) in their model of expert memory.

Preparation of forefinger’s sequence on keyboard orients ocular fixations on computer screen

This study examined the links between attention, hand movements and eye movements when performed in different spatial areas. Participants performed a visual search task on a computer screen while preparing to press two keyboard keys sequentially with their index. Results showed that the planning of the manual sequence influenced the latency of the first saccade and the placement of the first fixation. In particular, even if the first fixation placement was influenced by the combination of both components of the prepared manual sequence in some trials, it was affected principally by the first component of the prepared manual sequence. Moreover, the probability that the first fixation placement did reflect a combination of both components of the manual sequence was correlated with the speed of the second component. This finding suggests that the preparation of the second component of the sequence influence simultaneous oculomotor behavior when motor control of the manual sequence relied on proactive motor planning. These results are discussed taking into account the current debate on the eye/hand coordination research.

A regression-based method for the prediction of the indecisiveness degree through eye movement patterns

The development of eye-tracking-based methods to describe a persons indecisiveness is not commonly explored, even though research has shown that indecisiveness is involved in many unwanted cognitive states, such as a reduction in self-confidence during the decision-making process, doubts about past decisions, reconsidering, trepidation, distractibility, procrastination, neuroticism and even revenge. The purpose of our work is to propose a predictive model of a subjects degree of indecisiveness. To reach this goal, we first need to extract statistically relevant. Using eye-tracking methodology, we build a list of patterns that best distinguish decisive people from indecisive people; this segmentation is made according to the state of the art. The final list of eye-tracking patterns is also coherent with the state of art. A comparison between Multiple Linear Regression (MLR) and Support Vector Regression (SVR) is made so as to select the best predictive model.