Borna Noureddin

A single camera eye-gaze tracking system with free head motion

Eye-gaze as a form of human machine interface holds great promise for improving the way we interact with machines. Eye-gaze tracking devices that are non-contact, non-restrictive, accurate and easy to use will increase the appeal for including eye-gaze information in future applications. The system we have developed and which we describe in this paper achieves these goals using a single high resolution camera with a fixed field of view. The single camera system has no moving parts which results in rapid reacquisition of the eye after loss of tracking. Free head motion is achieved using multiple glints and 3D modeling techniques. Accuracies of under 1° of visual angle are achieved over a field of view of 14x12x20 cm and over various hardware configurations, camera resolutions and frame rates.

A non-contact device for tracking gaze in a human computer interface

This paper presents a novel design for a non-contact eye detection and gaze tracking device. It uses two cameras to maintain real-time tracking of a persons eye in the presence of head motion. Image analysis techniques are used to obtain accurate locations of the pupil and corneal reflections. All the computations are performed in software and the device only requires simple, compact optics and electronics attached to the users computer. Three methods of estimating the users point of gaze on a computer monitor are evaluated. The camera motion system is capable of tracking the users eye in real-time (9fps) in the presence of natural head movements as fast as 100^o/s horizontally and 77^o/s vertically. Experiments using synthetic images have shown its ability to track the location of the eye in an image to within 0.758 pixels horizontally and 0.492 pixels vertically. The system has also been tested with users with different eye colors and shapes, different ambient lighting conditions and the use of eyeglasses. A gaze accuracy of 2.9^o was observed.

Fixation Precision in High-Speed Noncontact Eye-Gaze Tracking

The precision of point-of-gaze (POG) estimation during a fixation is an important factor in determining the usability of a noncontact eye-gaze tracking system for real-time applications. The objective of this paper is to define and measure POG fixation precision, propose methods for increasing the fixation precision, and examine the improvements when the methods are applied to two POG estimation approaches. To achieve these objectives, techniques for high-speed image processing that allow POG sampling rates of over 400 Hz are presented. With these high-speed POG sampling rates, the fixation precision can be improved by filtering while maintaining an acceptable real-time latency. The high-speed sampling and digital filtering techniques developed were applied to two POG estimation techniques, i.e., the highspeed pupil-corneal reflection (HS P-CR) vector method and a 3-D model-based method allowing free head motion. Evaluation on the subjects has shown that when operating at 407 frames per second (fps) with filtering, the fixation precision for the HS P-CR POG estimation method was improved by a factor of 5.8 to 0.035deg (1.6 screen pixels) compared to the unfiltered operation at 30 fps. For the 3-D POG estimation method, the fixation precision was improved by a factor of 11 to 0.050deg (2.3 screen pixels) compared to the unfiltered operation at 30 fps.

Online Removal of Eye Movement and Blink EEG Artifacts Using a High-Speed Eye Tracker

A novel approach is presented for using an eye tracker-based reference instead of EOG for methods that require an EOG reference to remove ocular artifacts (OA) from EEG. It uses a high-speed eye tracker and a new online algorithm for extracting the time course of a blink from eye tracker images to remove both eye movement and blink artifacts. It eliminates the need for EOG electrodes attached to the face, which is critical for practical daily applications. The ability of two adaptive filters (RLS and H^ ) to remove OA is measured using: 1) EOG; 2) frontal EEG only (fEEG); and 3) the eye tracker with frontal EEG (ET + fEEG) as reference inputs. The results are compared for different eye movements and blinks of varying amplitudes at electrodes across the scalp. Both the RLS and H^ methods were shown to benefit from using the proposed eye tracker-based reference (ET + fEEG) instead of either an EOG reference or a reference based on frontal EEG alone.

Time-frequency Analysis of Eye Blinks and Saccades in EOG for EEG Artifact Removal

The electro-oculogram (EOG) is commonly used to detect, reject and remove ocular artifacts (OAs) from the electroencephalogram (EEG). We present a new time-frequency analysis of OAs found in the EOG. Our results indicate that in some tasks and subjects, frequencies up to 181Hz exist. Based on these results, we propose a minimum sampling rate for EOG. In addition, since EOG measurements typically contain signals other than OAs (including underlying EEG), we propose an alternate measurement of OAs based on a video-based eye tracker, and suggest a minimum frame rate for the use of such devices

Quantitative evaluation of ocular artifact removal methods based on real and estimated EOG signals

We propose a novel metric for quantitatively evaluating ocular artifact (OA) removal methods on real electroencephalogram (EEG) data. For real EEG, existing metrics measure the amount of artifact removed. Our metric measures how much a given method is likely to distort the underlying EEG. The new metric was used to evaluate two existing OA removal algorithms that use the electro-oculogram (EOG) as a reference signal. The combination of a previous metric and our new metric showed there is a trade-off between how well an algorithm removes OAs and how likely it is to distort the underlying EEG. These algorithms require a reference EOG signal, yet for certain applications (e.g., a brain computer interface or BCI) it is preferable or necessary to avoid attaching electrodes around the eyes. We thus also used various combinations of up to 55 channels of EEG to estimate the EOG reference. The metric was again used to compare the use of estimated vs. measured EOG. Our initial results showed that using EOG estimated from as few as 4 EEG electrodes increased the likelihood of distorting the EEG from 14% to 19% and from 21% to 23% for the two algorithms. For some applications (e.g., BCI), the slight reduction in performance may be acceptable in order to avoid using EOG electrodes.

An experimental study to investigate the effects of a motion tracking electromagnetic sensor during EEG data acquisition

A power spectral analysis study was conducted to investigate the effects of using an electromagnetic motion tracking sensor on an electroencephalogram (EEG) recording system. The results showed that the sensors do not generate any consistent frequency component(s) in the power spectrum of the EEG in the frequencies of interest (0.1-55 Hz).

Characterizing the community use of an ultralight wheelchair with “on the fly” adjustable seating functions: A pilot study

An ultralight manual wheelchair that allows users to independently adjust rear seat height and backrest angle during normal everyday usage was recently commercialized. Prior research has been performed on wheelchair tilt, recline, and seat elevation use in the community, however no such research has been done on this new class of manual ultralight wheelchair with “on the fly” adjustments. The objective of this pilot study was to investigate and characterize the use of the two adjustable seating functions available on the Elevation™ ultralight dynamic wheelchair during its use in the community. Eight participants had data loggers installed onto their own wheelchair for seven days to measure rear seat height, backrest angle position, occupied sitting time, and distance traveled. Analysis of rear seat height and backrest adjustment data revealed considerable variability in the frequency of use and positions used by participants. There was a wide spread of mean daily rear seat heights among participants, from 34.1 cm to 46.7 cm. Two sub-groups of users were further identified: those who sat habitually at a single typical rear seat height, and those who varied their rear seat height more continuously. Findings also showed that participants used the rear seat height adjustment feature significantly more often than the backrest adjustment feature. This obvious contrast in feature use may indicate that new users of this class of wheelchair may benefit from specific training. While the small sample size and exploratory nature of this study limit the generalizability of our results, our findings offer a first look at how active wheelchairs users are using a new class of ultralight wheelchair with “on the fly” seating adjustments in their communities. Further studies are recommended to better understand the impact of dynamic seating and positioning on activity, participation and quality of life.

Effects of task and EEG-based reference signal on performance of on-line ocular artifact removal from real EEG

The effects of two factors on the performance of online ocular artifact (OA) removal methods on real electroencephalogram (EEG) data are evaluated. A new metric is proposed that captures both the amount of artifact removed and the likelihood that a given method would distort the underlying EEG. The metric is then used to measure the performance of an existing on-line OA removal algorithm during periods of (i) eye movements and blinks (EM), (ii) a motor related potential (MRP), and (iii) neither EM nor MRP. The results show that the performance of the algorithm is significantly (p ≪ .05) affected by both EM and MRP. Also, the algorithm - like many others - requires a reference electro-oculogram (EOG) signal, yet for certain applications (e.g., a brain computer interface or BCI) it is preferable or necessary to avoid attaching electrodes around the eyes. Thus, the new metric is also used to investigate the effects of using 3 frontal EEG channels as the reference signal instead of EOG. The results show that replacing the EOG with the selected 3 EEG channels does not significantly (p ≪ .05) affect the performance of the algorithm, making it a viable option for online applications where the use of EOG is not suitable (e.g., BCI).

Online removal of eye movement and blink artifacts from EEG signals without EOG

In this thesis, two novel methods are presented for online removal of ocular artifacts (OA) from EEG without the need for EOG electrodes attached to the face. Both methods are fully automated and can remove the effects of both eye movements and blinks. The first method employs a high speed eye tracker and three frontal EEG electrodes as a reference to any nonlinear adaptive filter to remove OAs without any calibration. For the filters considered, at some frontal electrodes, using the eye tracker-based reference was shown to significantly (p < .05) improve the ability to remove OAs over using either EOG or only frontal EEG as a reference. Using an eye tracker provides the means for recording point-of-gaze and blink dynamics simultaneously with EEG, which is often desired or required in clinical studies and a variety of human computer interface applications. The second method uses a biophysical model of the head and movement of the eyes to remove OAs. It only requires a short once-per-subject calibration and does not require subject-specific MRI. It was compared to four existing methods, and was shown to perform consistently over a variety of tasks. In removing both saccades and blinks, it removed more than 4 times as much OA as the other methods. In terms of distortion, it was the only method that never removed more power than was present in the original EEG. To carry out the above studies, several related original investigations and developments were needed. These included a novel algorithm to extract the blink time course from eye tracker images, a new measure of OA removal distortion, a high speed eye tracker recording system, a study to determine whether frontal EEG could be used to replace EOG for OA removal and studies of the frequency content of blinks, the effects of an electromagnetic sensor on EEG, and the effects of varying mental states on OA removal methods. In summary this thesis has helped pave the way towards a real-time EEG-based human interface that is free of OAs and does not require EOG electrodes in its operation.