Nanxiang Li

Evaluating the robustness of an appearance-based gaze estimation method for multimodal interfaces

Given the crucial role of eye movements on visual attention, tracking gaze behaviors is an important research problem in various applications including biometric identification, attention modeling and human-computer interaction. Most of the existing gaze tracking methods require a repetitive system calibration process and are sensitive to the users head movements. Therefore, they cannot be easily implemented in current multimodal interfaces. This paper investigates an appearance-based approach for gaze estimation that requires minimum calibration and is robust against head motion. The approach consists in building an orthonormal basis, or eigenspace, of the eye appearance with principal component analysis (PCA). Unlike previous studies, we build the eigenspace using image patches displaying both eyes. The projections into the basis are used to train regression models which predict the gaze location. The approach is trained and tested with a new multimodal corpus introduced in this paper. We consider several variables such as the distance between user and the computer monitor, and head movement. The evaluation includes the performance of the proposed gaze estimation system with and without head movement. It also evaluates the results in subject-dependent versus subject-independent conditions under different distances. We report promising results which suggest that the proposed gaze estimation approach is a feasible and flexible scheme to facilitate gaze-based multimodal interfaces.

Using Perceptual Evaluation to Quantify Cognitive and Visual Driver Distractions

Developing feedback systems that can detect the attention level of the driver can play a key role in preventing accidents by alerting the driver about possible hazardous situations. Monitoring drivers’ distraction is an important research problem, especially with new forms of technology that are made available to drivers. An important question is how to define reference labels that can be used as ground truth to train machine-learning algorithms to detect distracted drivers. The answer to this question is not simple since drivers are affected by visual, cognitive, auditory, psychological, and physical distractions. This chapter proposes to define reference labels with perceptual evaluations from external evaluators. We describe the consistency and effectiveness of using a visual-cognitive space for subjective evaluations. The analysis shows that this approach captures the multidimensional nature of distractions. The representation also defines natural modes to characterize driving behaviors.

User Independent Gaze Estimation by Exploiting Similarity Measures in the Eye Pair Appearance Eigenspace

The design of gaze-based computer interfaces has been an active research area for over 40 years. One challenge of using gaze detectors is the repetitive calibration process required to adjust the parameters of the systems, and the constrained conditions imposed on the user for robust gaze estimation. We envision user-independent gaze detectors that do not require calibration, or any cooperation from the user. Toward this goal, we investigate an appearance-based approach, where we estimate the eigenspace for the gaze using principal component analysis (PCA). The projections are used as features of regression models that estimate the screens coordinates. As expected, the performance of the approach decreases when the models are trained without data from the target user (i.e., user-independent condition). This study proposes an appealing training approach to bridge the gap in performance between user-dependent and user-independent conditions. Using the projections onto the eigenspace, the scheme identifies samples in training set that are similar to the testing images. We build the sample covariance matrix and the regression models only with these samples. We consider either similar frames or data from subjects with similar eye appearance. The promising results suggest that the proposed training approach is a feasible and convenient scheme for gaze-based multimodal interfaces.

Calibration free, user-independent gaze estimation with tensor analysis

Abstract Human gaze directly signals visual attention, therefore, estimation of gaze has been an important research topic in fields such as human attention modeling and human-computer interaction. Accurate gaze estimation requires user, system and even session dependent parameters, which can be obtained by calibration process. However, this process has to be repeated whenever the parameter changes (head movement, camera movement, monitor movement). This study aims to eliminate the calibration process of gaze estimation by building a user-independent, appearance-based gaze estimation model. The system is ideal for multimodal interfaces, where the gaze is tracked without the cooperation from the users. The main goal is to capture the essential representation of the gaze appearance of the target user. We investigate the tensor analysis framework that decomposes the high dimension gaze data into different factors including individual differences, gaze differences, user-screen distances and session differences. The axis that is representative for a particular subject is automatically chosen in the tensor analysis framework using LASSO regression. The proposed approaches show promising results on capturing the test subject gaze changes. To address the estimation shift caused by the variations in individual heights, or relative position to the monitor, we apply domain adaptation to adjust the gaze estimation, observing further improvements. These promising results suggest that the proposed gaze estimation approach is a feasible and flexible scheme to facilitate gaze-based multimodal interfaces.

Appearance based user-independent gaze estimation

An ideal gaze user interface should be able to accurately estimates the users gaze direction in a non-intrusive setting. Most studies on gaze estimation focus on the accuracy of the estimation results, imposing important constraints on the user such as no head movement, intrusive head mount setting and repetitive calibration process. Due to these limitations, most graphic user interfaces (GUIs) are reluctant to include gaze as an input modality. We envision user-independent gaze detectors for user computer interaction that do not impose any constraints on the users. We believe the appearance of the eye pairs, which implicitly reveals head pose, provides conclusive information on the gaze direction. More importantly, the relative appearance changes in the eye pairs due to the different gaze direction should be consistent among different human subjects. We collected a multimodal corpus (MSP-GAZE) to study and evaluate user independent, appearance based gaze estimation approaches. This corpus considers important factors that affect the appearance based gaze estimation: the individual difference, the head movement, and the distance between the user and the interfaces screen. Using this database, our initial study focused on the eye pair appearance eigenspace approach, where the projections into the eye appearance eigenspace basis are used to build regression models to estimate the gaze position. We compare the results between user dependent (training and testing on the same subject) and user independent (testing subject is not included in the training data) models. As expected, due to the individual differences between subjects, the performance decreases when the models are trained without data from the target user. The study aims to reduce the gap between user dependent and user independent conditions.