Robert Bixler

Toward Fully Automated Person-Independent Detection of Mind Wandering

Mind wandering is a ubiquitous phenomenon where attention involuntary shifts from task-related processing to task-unrelated thoughts. Mind wandering has negative effects on performance, hence, intelligent interfaces that detect mind wandering can intervene to restore attention to the current task. We investigated the use of eye gaze and contextual cues to automatically detect mind wandering during reading with a computer interface. Participants were pseudo-randomly probed to report mind wandering instances while an eye tracker recorded their gaze during a computerized reading task. Supervised machine learning techniques detected positive responses to mind wandering probes from gaze and context features in a user-independent fashion. Mind wandering was predicted with an accuracy of 72% (expected accuracy by chance was 62%) when probed at the end of a page and an accuracy of 59% (chance was 50%) when probed in the midst of reading a page. Possible improvements to the detectors and applications are discussed.

Automated Physiological-Based Detection of Mind Wandering during Learning

Unintentional lapses of attention, or mind wandering, are ubiquitous and detrimental during learning. Hence, automated methods that detect and combat mind wandering might be beneficial to learning. As an initial step in this direction, we propose to detect mind wandering by monitoring physiological measures of skin conductance and skin temperature. We conducted a study in which students physiology signals were measured while they learned topics in research methods from instructional texts. Momentary self-reports of mind wandering were collected with standard probe-based methods. We computed features from the physiological signals in windows leading up to the probes and trained supervised classification models to detect mind wandering. We obtained a kappa, a measurement of accuracy corrected for random guessing, of .22, signaling feasibility of detecting MW in a student-independent manner. Though modest, we consider this result to be an important step towards fully-automated unobtrusive detection of mind wandering during learning.

Automatic Gaze-Based Detection of Mind Wandering with Metacognitive Awareness

Mind wandering (MW) is a ubiquitous phenomenon where attention involuntarily shifts from task-related processing to task-unrelated thoughts. There is a need for adaptive systems that can reorient attention when MW is detected due to its detrimental effects on performance and productivity. This paper proposes an automated gaze-based detector of self-caught MW (i.e., when users become consciously aware that they are MW). Eye gaze data and self-reports of MW were collected as 178 users read four instructional texts from a computer interface. Supervised machine learning models trained on features extracted from users’ gaze fixations were used to detect pages where users caught themselves MW. The best performing model achieved a user-independent kappa of .45 (accuracy of 74% compared to a chance accuracy of 52%); the first ever demonstration of a self-caught MW detector. An analysis of the features revealed that during MW, users made more regression fixations, had longer saccades that crossed lines more often, and had more uniform fixation durations, indicating a violation from normal reading patterns. Applications of the MW detector are discussed.

An automated behavioral measure of mind wandering during computerized reading

Mind wandering is a ubiquitous phenomenon in which attention shifts from task-related to task-unrelated thoughts. The last decade has witnessed an explosion of interest in mind wandering, but research has been stymied by a lack of objective measures, leading to a near-exclusive reliance on self-reports. We addressed this issue by developing an eye-gaze-based, machine-learned model of mind wandering during computerized reading. Data were collected in a study in which 132 participants reported self-caught mind wandering while reading excerpts from a book on a computer screen. A remote Tobii TX300 or T60 eyetracker recorded their gaze during reading. The data were used to train supervised classification models to discriminate between mind wandering and normal reading in a manner that would generalize to new participants. We found that at the point of maximal agreement between the model-based and self-reported mind-wandering means (smallest difference between the group-level means: Mmodel= .310, Mself= .319), the participant-level mind-wandering proportional distributions were similar and were significantly correlated (r = .400). The model-based estimates were internally consistent (r = .751) and predicted text comprehension more strongly than did self-reported mind wandering (rmodel= −.374, rself= −.208). Our results also indicate that a robust strategy of probabilistically predicting mind wandering in cases with poor or missing gaze data led to improved performance on all metrics, as compared to simply discarding these data. Our findings demonstrate that an automated objective measure might be available for laboratory studies of mind wandering during reading, providing an appealing alternative or complement to self-reports.

Identifying Learning Conditions that Minimize Mind Wandering by Modeling Individual Attributes

The propensity to involuntarily disengage by zoning out or mind wandering MW is a common phenomenon that has negative effects on learning. The ability to stay focused while learning from instructional texts involves factors related to the text, to the task, and to the individual. This study explored the possibility that learners could be placed in optimal conditions task and text to reduce MW based on an analysis of individual attributes. Students studied four texts which varied along dimensions of value and difficulty while reporting instances of MW. Supervised machine learning techniques based on a small set of individual difference attributes determined the optimal condition for each participant with some success when considering value and difficulty separately kappas of .16 and .24; accuracy of 59% and 64% respectively. Results are discussed in terms of creating a learning system that prospectively places learners in the optimal condition to increase learning by minimizing MW.

Predicting Individual Action Switching in Covert and Continuous Interactive Tasks Using the Fluid Events Model

The Fluid Events Model is aimed at predicting changes in the actions people take on a moment-by-moment basis. In contrast with other research on action selection, this work does not investigate why some course of action was selected, but rather the likelihood of discontinuing the current course of action and selecting another in the near future. This is done using both task-based and experience-based factors. Prior work evaluated this model in the context of trial-by-trial, independent, interactive events, such as choosing how to copy a figure of a line drawing. In this paper, we extend this model to more covert event experiences, such as reading narratives, as well as to continuous interactive events, such as playing a video game. To this end, the model was applied to existing data sets of reading time and event segmentation for written and picture stories. It was also applied to existing data sets of performance in a strategy board game, an aerial combat game, and a first person shooter game in which a participant’s current state was dependent on prior events. The results revealed that the model predicted behavior changes well, taking into account both the theoretically defined structure of the described events, as well as a person’s prior experience. Thus, theories of event cognition can benefit from efforts that take into account not only how events in the world are structured, but also how people experience those events.

ETGraph: A graph-based approach for visual analytics of eye-tracking data ☆

Abstract Mind wander(ing) (MW) or zoning out is a ubiquitous phenomenon where attention involuntary shifts from task-related processing to task-unrelated thoughts. Unfortunately, MW is a highly internal state so it cannot be readily inferred from overt behaviors and expressions. To help experts investigate mind wanderings, we present a graph-based approach for visual analytics of eye-tracking data, which utilizes the graph representations to illustrate the reading patterns and further help experts detect and verify mind wanderings based on the graph structures and other graph attributes. The input data are collected from multiple participants reading multiple pages of a book on a computer screen. Our approach first clusters fixations into fixation clusters, then creates the eye-tracking graph, i.e., ETGraph, for use in conjunction with the standard page view, time view, and statistics view. The graph view presents a visual representation of the actual reading patterns of a single participant or multiple participants and therefore serves as the main visual interface for exploration and navigation. We design a suite of techniques to help users identify common reading patterns and outliers for analytical reasoning at three different levels of detail: single participant single page, single participant multiple pages, and multiple participants single page. Interactive querying and filtering functions are provided for reducing visual clutter in the visualization and enabling users to answer questions and glean insights. Our tool also facilitates the detection and verification of mind wandering that the experts seek to investigate. We conduct a user study and an expert evaluation to assess the effectiveness of ETGraph in terms of its visual summarization and comparison capabilities.

Towards Automated Detection and Regulation of Affective States During Academic Writing

This project focuses on developing methods to automatically detect and respond to emotions that students experience while developing writing proficiency with computerized environments. We describe progress that we have already made toward detecting affect during writing using keystroke analysis, stable traits, and task appraisals. We were able to distinguish boredom from engagement with an accuracy of 38% above random guessing. Our next goal is to improve the accuracy of our classifier. We plan to accomplish this through an exploration of higher level features such as sequences of character types. Ultimately we hope to develop a system capable of both detecting affect and influencing affect through interventions and experimentally testing this system.