Joshua Newn

Motion Correlation: Selecting Objects by Matching Their Movement

Selection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking its specific motion, and use the correlation between the system’s output with the user’s input to determine the selection. The resulting interaction has compelling properties, as users are guided by motion feedback, and only need to copy a presented motion. Motion correlation has been explored in earlier work but only recently begun to feature in holistic interface designs. We provide a first comprehensive review of the principle, and present an analysis of five previously published works, in which motion correlation underpinned the design of novel gaze and gesture interfaces for diverse application contexts. We derive guidelines for motion correlation algorithms, motion feedback, choice of modalities, overall design of motion correlation interfaces, and identify opportunities and challenges identified for future research and design.

Remote Gaze and Gesture Tracking on the Microsoft Kinect: Investigating the Role of Feedback

In this paper we present the results of a user experience and preference study into the combination of gaze and gesture in a lounge-style remote-interaction, using a novel system that tracks gaze and gesture using only the Kinect device at a distance of 2m from the user. Our results indicate exciting opportunities for gaze-tracking interfaces that use existing technologies, but suggest that findings from studies of highly-accurate gaze systems may not apply in these real-world simulations where the gaze-tracking is inherently less accurate. We contribute a series of design recommendations for gaze and gesture interfaces in this context, and based on these limitations.

Multimodal Segmentation on a Large Interactive Tabletop: Extending Interaction on Horizontal Surfaces with Gaze

Eye tracking is a promising input modality for interactive tabletops. However, issues such as eyelid occlusion and the viewing angle at distant positions present significant challenges for remote gaze tracking in this setting. We present the results of two studies that explore the way gaze interaction can be enabled. Our first study contributes the results from an empirical investigation of gaze accuracy on a large horizontal surface, finding gaze to be unusable close to the user (due to eyelid occlusion), accurate at arms length, and only precise horizontally at large distances. In consideration of these results, we propose two solutions for the design of interactive systems that utilise remote gaze-tracking on the tabletop; multimodal segmentation and the use of X-Gaze-our novel technique-to interact with out-of-reach objects. Our second study evaluates and validates both these solutions in a Video-on-Demand application, presenting immediate opportunities for remote-gaze interaction on horizontal surfaces.

EyePlay Revisited: Past, Present and Future Challenges for Eye-Based Interaction in Games

What are current and future challenges that incorporating eye tracking into game design and development creates? The Second EyePlay workshop brings together academic researchers and industry practitioners from the fields of eye tracking and games to explore these questions. In recent years, gaming has been at the forefront of the commercial popularization of eye tracking. In this workshop, we will share experiences in the development of gaze-enabled games, discuss best practices and tools, and explore future challenges for research. Topics of interest lie at the intersection of eye tracking and games including, but limited to, novel interaction techniques and game mechanics, development processes and tools, accessible games, evaluation, and future visions.