Derrick J. Parkhurst

Starburst: A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches

Knowing the user’s point of gaze has significant potential to enhance current human-computer interfaces, given that eye movements can be used as an indicator of the attentional state of a user. The primary obstacle of integrating eye movements into today’s interfaces is the availability of a reliable, low-cost open-source eye-tracking system. Towards making such a system available to interface designers, we have developed a hybrid eye-tracking algorithm that integrates feature-based and model-based approaches and made it available in an open-source package. We refer to this algorithm as starburst because of the novel way in which pupil features are detected. This starburst algorithm is more accurate than pure feature-based approaches yet is signi?cantly less time consuming than pure modelbased approaches. The current implementation is tailored to tracking eye movements in infrared video obtained from an inexpensive head-mounted eye-tracking system. A validation study was conducted and showed that the technique can reliably estimate eye position with an accuracy of approximately one degree of visual angle.

openEyes: a low-cost head-mounted eye-tracking solution

Eye tracking has long held the promise of being a useful methodology for human computer interaction. However, a number of barriers have stood in the way of the integration of eye tracking into everyday applications, including the intrusiveness, robustness, availability, and price of eye-tracking systems. To lower these barriers, we have developed the openEyes system. The system consists of an open-hardware design for a digital eye tracker that can be built from low-cost off-the-shelf components, and a set of open-source software tools for digital image capture, manipulation, and analysis in eye-tracking applications. We expect that the availability of this system will facilitate the development of eye-tracking applications and the eventual integration of eye tracking into the next generation of everyday human computer interfaces. We discuss the methods and technical challenges of low-cost eye tracking as well as the design decisions that produced our current system.

Enhancing Multi-user Interaction with Multi-touch Tabletop Displays Using Hand Tracking

A rear-projection multi-touch tabletop display was augmented with hand tracking utilizing computer vision techniques. Touch detection by frustrated total internal reflection is useful for achieving interaction with tabletop displays, but the technique is not always reliable when multiple users in close proximity simultaneously interact with the display. To solve this problem, we combine touch detection and hand tracking techniques in order to allow multiple users to simultaneously interact with the display without interference. Our hope is that by considering activities occurring on and above a tabletop display, multiuser interaction will become more natural and useful, which should ultimately support collaborative work.

openEyes: an open-hardware open-source system for low-cost eye tracking

Eye tracking has long held the promise of being a useful methodology for human computer interaction. However, a number of barriers have stood in the way of the integration of eye tracking into everyday applications, including the intrusiveness, robustness, availability and price of eye-tracking systems. To lower these barriers, we have developed the openEyes system. The system consists of an open-hardware design for a digital eye tracker that can be built from low-cost off-the-shelf components, and a set of open-source software tools for digital image capture, manipulation, and analysis in eye-tracking applications. We expect that the availability of this system will facilitate the development of eye-tracking applications and the eventual integration of eye tracking into the next generation of everyday human computer interfaces. We discuss the methods and technical challenges of low-cost eye tracking as well as the design decisions that produced our current system.

Perceptually Adaptive Rendering of Immersive Virtual Environments

When rendering immersive virtual environments, it is necessary to limit the geometric complexity of the scene in order to retain interactive frame rates. Both reduced frame rendering rates and reduced geometric complexity have been shown to have a negative impact on task performance. We present a perceptually adaptive rendering system which optimizes rendering performance in immersive virtual environments. Our system currently uses three discrete levels of detail for each model, which are created prior to run time. We use head position and orientation in the 3D environment to determine the appropriate geometric level of detail (LOD) of the objects in the scene. Based upon the thresholds used to trigger a change between two LODs, interactivity can be significantly improved without overtly distracting the user.

Multimodal UAV ground control system

As unmanned units become more capable of self-control, and as their integration into our military forces increase, the role of the operator of these units is going to change. Current UAV ground control systems typically require to much of the operator’s attention per unit, and the role of the future operator will be more like that of a manager than that of a pilot. This paper describes research being done on a user interface for a future ground control system. Beginning with a visualization of a virtual battlefield, this next-generation UAV ground control system incorporates all available information and analyses in combination with a synthetic battlespace in a context-relevant manner. This marriage of synthetic and real information feeds, within a single visualization space, is designed to increase the situational awareness of a single operator. This virtual battlefield provides the operator with the information necessary to accomplish their task, with multiple modes of interaction for the user. The UAV Ground Control System detailed here utilizes a speech command interface, a wireless joystick interface, as well as a tablet-based direct manipulation interface similar to those used in general air unit ground control systems. Through these different types of interactions, an operator is presented with various sources of data to manage and command military units as clearly and simply as possible.

VSARD: a low-cost augmented reality system for desktop applications

Augmented Reality (AR) is a quickly developing field in the domain of human computer interaction. The goal of an AR application is to supplement the users perception of the real world with synthetic stimuli. For example, in visual AR applications a head-mounted display (HMD) can be used to overlay graphical renderings of virtual objects onto a view of the real world such that both virtual and real objects appear to exist in the same space. With the use of an HMD, the relationship between the users head and the image display is fixed and thus the insertion of computer graphics into the scene is straight forward. However, HMDs are limited in a number of important ways, including their high expense and low comfort. An alternative to HMDs for AR applications is to use a desktop display. For example, the PARIS (Personal Augmented Reality Immersive System [Johnson et al. 2000]) is a desk-based AR system. In this system, computer-generated graphics are overlayed on the scene of the desk (i.e., the desktop). An optical see-through technique is used to combine the real and virtual scenes. This is achieved by equiping a desk with a half-silvered mirror placed between the user and the desktop and a projector. The half-silvered mirror is angled such that light from the projector positioned above the desk is reflected back to the user and combined with light reflecting from the desktop. The ultimate result is that the user of this system views a single desktop containing both real and virtual objects.

Viewpoint invariant object features attract overt visual attention

Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94, 115-147. Geusebroek, J., Burghouts, G. J., & Smeulders, A. W. M. (2005). The Amsterdam library of object images. International Journal of Computer Vision, 61(1), 103-112. Itti, L., Niebur, E., & Koch, C. (1998). A model of saliency-based fast visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254-1259. Lowe, D. G. (1999, September). Object recognition from local scale-invariant features. Paper presented at the International Conference on Computer Vision, Corfu, Greece, 1150-1157. Parkhurst, D., Law, K., & Niebur, E. (2002). Modeling the role of salience in the allocation of overt visual attention. Vision Research, 42, 107-123. Wolff, T., Still, J. D., Parkhurst, D. J. & Dark, V. J. (2007, May). Invariant Features Detected with Computer Vision Allow Better Human Object Recognition in Photographs. Poster presented at the meeting of the Midwestern Psychological Association, Chicago, IL. Scale Invariant Feature Transform (SIFT)