Elias Daniel Guestrin

General theory of remote gaze estimation using the pupil center and corneal reflections

This paper presents a general theory for the remote estimation of the point-of-gaze (POG) from the coordinates of the centers of the pupil and corneal reflections. Corneal reflections are produced by light sources that illuminate the eye and the centers of the pupil and corneal reflections are estimated in video images from one or more cameras. The general theory covers the full range of possible system configurations. Using one camera and one light source, the POG can be estimated only if the head is completely stationary. Using one camera and multiple light sources, the POG can be estimated with free head movements, following the completion of a multiple-point calibration procedure. When multiple cameras and multiple light sources are used, the POG can be estimated following a simple one-point calibration procedure. Experimental and simulation results suggest that the main sources of gaze estimation errors are the discrepancy between the shape of real corneas and the spherical corneal shape assumed in the general theory, and the noise in the estimation of the centers of the pupil and corneal reflections. A detailed example of a system that uses the general theory to estimate the POG on a computer screen is presented.

Remote Point-of-Gaze Estimation with Free Head Movements Requiring a Single-Point Calibration

This paper describes a method for remote, non- contact point-of-gaze (POG) estimation that tolerates free head movements and requires a simple calibration procedure in which the subject has to fixate only on a single point. This method uses the centers of the pupil and at least two corneal reflections (virtual images of light sources) that are estimated from eye images captured by at least two cameras. Experimental results obtained with a prototype system that tolerates head movements in a volume of about 1 dm3, exhibited RMS POG estimation errors of approximately 0.6-1deg of visual angle. This system can enable applications with infants that, otherwise, would not be possible with existing POG estimation methods, which typically require multiple-point calibration procedures.

Identification and robust control of an experimental servo motor.

In this work, the design of a robust controller for an experimental laboratory-scale position control system based on a dc motor drive as well as the corresponding identification and robust stability analysis are presented. In order to carry out the robust design procedure, first, a classic closed-loop identification technique is applied and then, the parametrization by internal model control is used. The model uncertainty is evaluated under both parametric and global representation. For the latter case, an interesting discussion about the conservativeness of this description is presented by means of a comparison between the uncertainty disk and the critical perturbation radius approaches. Finally, conclusions about the performance of the experimental system with the robust controller are discussed using comparative graphics of the controlled variable and the Nyquist stability margin as a robustness measurement.

Analysis of subject-dependent point-of-gaze estimation bias in the cross-ratios method

The cross-ratios method for point-of-gaze estimation uses the invariance property of cross-ratios in projective transformations. The inherent causes of the subject-dependent point-of-gaze estimation bias exhibited by this method have not been well characterized in the literature. Using a model of the eye and the components of a system (camera, light sources) that estimates point-of-gaze, a theoretical framework for the cross-ratios method is developed. The analysis of the cross-ratios method within this framework shows that the subject-dependent estimation bias is caused mainly by (i) the angular deviation of the visual axis from the optic axis and (ii) the fact that the virtual image of the pupil center is not coplanar with the virtual images of the light sources that illuminate the eye (corneal reflections). The theoretical framework provides a closed-form analytical expression that predicts the estimation bias as a function of subject-specific eye parameters.

Remote point-of-gaze estimation with single-point personal calibration based on the pupil boundary and corneal reflections

This paper describes a new method for remote, non-contact point-of-gaze (PoG) estimation that tolerates head movements and requires a simple personal calibration procedure in which the subject has to fixate a single calibration point. This method uses the pupil boundary and at least two corneal reflections (virtual images of light sources) that are extracted from eye images captured by at least two video cameras. Experimental results obtained with a prototype system exhibited RMS PoG estimation errors of approx. 0.4–0.6° of visual angle. Such accuracy is comparable to that of the best commercially available systems, which use multiple-point personal calibration procedures, and significantly better than that of any other systems that use a single-point personal calibration procedure and have been previously described in the literature.

An automatic calibration procedure for remote eye-gaze tracking systems

Remote gaze estimation systems use calibration procedures to estimate subject-specific parameters that are needed for the calculation of the point-of-gaze. In these procedures, subjects are required to fixate on a specific point or points at specific time instances. Advanced remote gaze estimation systems can estimate the optical axis of the eye without any personal calibration procedure, but use a single calibration point to estimate the angle between the optical axis and the visual axis (line-of-sight). This paper presents a novel automatic calibration procedure that does not require active user participation. To estimate the angles between the optical and visual axes of each eye, this procedure minimizes the distance between the intersections of the visual axes of the left and right eyes with the surface of a display while subjects look naturally at the display (e.g., watching a video clip). Simulation results demonstrate that the performance of the algorithm improves as the range of viewing angles increases. For a subject sitting 75 cm in front of an 80 cm × 60 cm display (40” TV) the standard deviation of the error in the estimation of the angles between the optical and visual axes is 0.5°.

Covert monitoring of the point-of-gaze

Gaze estimation systems use calibration procedures that require active subject participation to estimate the point-of-gaze accurately. Consequently, these systems do not support covert monitoring of visual scanning patterns. This paper presents a novel gaze estimation methodology that does not use calibration procedures that require active user participation. This methodology uses multiple infrared light sources for illumination and a stereo pair of video cameras to obtain images of the eyes. Each pair of images is analyzed and the centers of the pupils and the centers of curvature of the corneas are estimated. These points, which are estimated without a personal calibration procedure, define the optical axis of each eye. To estimate the point-of-gaze, which lies along the visual axis, the angle between the optical and visual axes is estimated by a procedure that minimizes the distance between the intersections of the visual axes of the left and right eyes with the surface of a display while subjects look naturally at the display (e.g., watching a video clip). Simulation results demonstrate that for a subject sitting 75 cm in front of an 80 cm × 60 cm display (40″ TV) the RMS error of the estimated point-of-gaze is 17.8 mm (1.3°).

Erratum to “General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections”

Manuscript received May 26, 2006. Asterisk indicates corresponding author. *E. D. Guestrin is with the Department of Electrical and Computer Engineering, and the Institute of Biomaterials and Biomedical Engineering, University of Toronto, 164 College Street, Toronto, ON M5S 3G9, Canada (e-mail: elias.guestrin@utoronto.ca). M. Eizenman is with the Departments of Electrical and Computer Engineering and Ophthalmology, and the Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada (e-mail: eizenm@ecf.utoronto.ca). Digital Object Identifier 10.1109/TBME.2006.880503 In [1], the first sentence of the last paragraph of Section II-A should read as follows. In general, gaze estimation systems that use one camera and one light source do not solve the above system of equations, but rather use the vector from the pupil center to the corneal reflection in the eye image to compute the gaze direction relative to the camera axis [13], and either assume that the head movements are negligible or have means to estimate the position of the eye in space (e.g., combination of a moving camera or moving mirrors that track the eye and an auto-focus system or an ultrasonic transducer) [7], [11], [21]–[23]. In the second line of the first paragraph of Appendix B in [1], it should read WCS.