Jeff Klingner

Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks

Degree of pupil dilation has been shown to be a valid and reliable measure of cognitive load, but the effect of aural versus visual task presentation on pupil dilation is unknown. To evaluate effects of presentation mode, pupil dilation was measured in three tasks spanning a range of cognitive activities: mental multiplication, digit sequence recall, and vigilance. Stimuli were presented both aurally and visually, controlling for all known visual influences on pupil diameter. The patterns of dilation were similar for both aural and visual presentation for all three tasks, but the magnitudes of pupil response were greater for aural presentation. Accuracy was higher for visual presentation for mental arithmetic and digit recall. The findings can be accounted for in terms of dual codes in working memory and suggest that cognitive load is lower for visual than for aural presentation.

Measuring the task-evoked pupillary response with a remote eye tracker

The pupil-measuring capability of video eye trackers can detect the task-evoked pupillary response: subtle changes in pupil size which indicate cognitive load. We performed several experiments to measure cognitive load using a remote video eye tracker, which demonstrate two extensions to current research in this area. First, we show that cognitive pupillometry can be extended from head-mounted to remote eye tracking systems. Second, we demonstrate the feasibility of a more fine-grained approach to analyzing pupil size data gathered with an eye tracker, which provides more detail about the timing and magnitude of changes in cognitive load.

Conveying shape and features with image-based relighting

Hand-crafted illustrations are often more effective than photographs for conveying the shape and important features of an object, but they require expertise and time to produce. We describe an image compositing system and user interface that allow an artist to quickly and easily create technical illustrations from a set of photographs of an object taken from the same point of view under variable lighting conditions. Our system uses a novel compositing process in which images are combined using spatially-varying light mattes, enabling the final lighting in each area of the composite to be manipulated independently. We describe an interface that provides for the painting of local lighting effects (e.g. shadows, highlights, and tangential lighting to reveal texture) directly onto the composite. We survey some of the techniques used in illustration and lighting design to convey the shape and features of objects and describe how our system can be used to apply these techniques.

Case study: visualizing sets of evolutionary trees

We describe a visualization tool which allows a biologist to explore a large set of hypothetical evolutionary trees. Interacting with such a dataset allows the biologist to identify distinct hypotheses about how different species or organisms evolved, which would not have been clear from traditional analyses. Our system integrates a point-set visualization of the distribution of hypothetical trees with detail views of an individual tree, or of a consensus tree summarizing a subset of trees. Efficient algorithms were required for the key tasks of computing distances between trees, finding consensus trees, and laying out the point-set visualization.

Improving the accuracy of gaze input for interaction

Using gaze information as a form of input poses challenges based on the nature of eye movements and how we humans use our eyes in conjunction with other motor actions. In this paper, we present three techniques for improving the use of gaze as a form of input. We first present a saccade detection and smoothing algorithm that works on real-time streaming gaze information. We then present a study which explores some of the timing issues of using gaze in conjunction with a trigger (key press or other motor action) and propose a solution for resolving these issues. Finally, we present the concept of Focus Points, which makes it easier for users to focus their gaze when using gaze-based interaction techniques. Though these techniques were developed for improving the performance of gaze-based pointing, their use is applicable in general to using gaze as a practical form of input.

Sketches for Design and Design of Sketches

It is said, though not without controversy, that what distinguishes design from art is function. Design is for a purpose, usually a human one. As such, design entails both generating ideas and adapting those ideas to intended uses. This occurs iteratively. Form and function. Studying how people go about both these tasks gives insights that can facilitate the design process. Two relevant projects will be described. The first investigates how designers and novices get ideas from sketches and applies those insights to suggestions for promoting generation of ideas. The second seeks to develop computer algorithms for designing individualized visualizations, algorithms that are informed by cognitive design principles.

Fixation-aligned pupillary response averaging

We propose a new way of analyzing pupil measurements made in conjunction with eye tracking: fixation-aligned pupillary response averaging, in which short windows of continuous pupil measurements are selected based on patterns in eye tracking data, temporally aligned, and averaged together. Such short pupil data epochs can be selected based on fixations on a particular spot or a scan path. The windows of pupil data thus selected are aligned by temporal translation and linear warping to place corresponding parts of the gaze patterns at corresponding times and then averaged together. This approach enables the measurement of quick changes in cognitive load during visual tasks, in which task components occur at unpredictable times but are identifiable via gaze data. We illustrate the method through example analyses of visual search and map reading. We conclude with a discussion of the scope and limitations of this new method.

The pupillometric precision of a remote video eye tracker

To determine the accuracy and precision of pupil measurements made with the Tobii 1750 remote video eye tracker, we performed a formal metrological study with respect to a calibrated reference instrument, a medical pupillometer. We found that the eye tracker measures mean binocular pupil diameter with precision 0.10 mm and mean binocular pupil dilations with precision 0.15 mm.