Shamsi T. Iqbal

Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management

Notifications can have reduced interruption cost if delivered at moments of lower mental workload during task execution. Cognitive theorists have speculated that these moments occur at subtask boundaries. In this article, we empirically test this speculation by examining how workload changes during execution of goal-directed tasks, focusing on regions between adjacent chunks within the tasks, that is, the subtask boundaries. In a controlled experiment, users performed several interactive tasks while their pupil dilation, a reliable measure of workload, was continuously measured using an eye tracking system. The workload data was extracted from the pupil data, precisely aligned to the corresponding task models, and analyzed. Our principal findings include (i) workload changes throughout the execution of goal-directed tasks; (ii) workload exhibits transient decreases at subtask boundaries relative to the preceding subtasks; (iii) the amount of decrease tends to be greater at boundaries corresponding to the completion of larger chunks of the task; and (iv) different types of subtasks induce different amounts of workload. We situate these findings within resource theories of attention and discuss important implications for interruption management systems.

Task-evoked pupillary response to mental workload in human-computer interaction

Accurate assessment of a users mental workload will be critical for developing systems that manage user attention (interruptions) in the user interface. Empirical evidence suggests that an interruption is much less disruptive when it occurs during a period of lower mental workload. To provide a measure of mental workload for interactive tasks, we investigated the use of task-evoked pupillary response. Results show that a more difficult task demands longer processing time, induces higher subjective ratings of mental workload, and reliably evokes greater pupillary response at salient subtasks. We discuss the findings and their implications for the design of an attention manager.

Towards an index of opportunity: understanding changes in mental workload during task execution

To contribute to systems that reason about human attention, our work empirically demonstrates how a users mental workload changes during task execution. We conducted a study where users performed interactive, hierarchical tasks while mental workload was measured through the use of pupil size. Results show that (i) different types of subtasks impose different mental workload, (ii) workload decreases at subtask boundaries, (iii) workload decreases more at boundaries higher in a task model and less at boundaries lower in the model, (iv) workload changes among subtask boundaries within the same level of a task model, and (v) effective understanding of why changes in workload occur requires that the measure be tightly coupled to a validated task model. From the results, we show how to map mental workload onto a computational Index of Opportunity that systems can use to better reason about human attention.

Effects of intelligent notification management on users and their tasks

We present a novel system for notification management and report results from two studies testing its performance and impact. The system uses statistical models to realize defer-to-breakpoint policies for managing notifications. The first study tested how well the models detect three types of breakpoints within novel task sequences. Results show that the models detect breakpoints reasonably well, but struggle to differentiate their type. Our second study explored effects of managing notifications with our system on users and their tasks. Results showed that scheduling notifications at breakpoints reduces frustration and reaction time relative to delivering them immediately. We also found that the relevance of notification content determines the type of breakpoint at which it should be delivered. The core concept of scheduling notifications at breakpoints fits well with how users prefer notifications to be managed. This indicates that users would likely adopt the use of notification management systems in practice.

Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption

This work investigates the use of workload-aligned task models for predicting opportune moments for interruption. From models for several tasks, we selected boundaries with the lowest (Best) and highest (Worst) mental workload. We compared effects of interrupting primary tasks at these and Random moments on resumption lag, annoyance, and social attribution. Results show that interrupting at the Best moments consistently caused less resumption lag and annoyance, and fostered more social attribution. Results demonstrate that use of workload-aligned models offers a systematic method for predicting opportune moments.

Leveraging characteristics of task structure to predict the cost of interruption

A challenge in building interruption reasoning systems is to compute an accurate cost of interruption (COI). Prior work has used interface events and other cues to predict COI, but ignore characteristics related to the structure of a task. This work investigates how well characteristics of task structure can predict COI, as objectively measured by resumption lag. In an experiment, users were interrupted during task execution at various boundaries to collect a large sample of resumption lag values. Statistical methods were employed to create a parsimonious model that uses characteristics of task structure to predict COI. A subsequent experiment with different tasks showed that the model can predict COI with reasonably high accuracy. Our model can be expediently applied to many goal-directed tasks, allowing systems to make more effective decisions about when to interrupt.

Understanding and developing models for detecting and differentiating breakpoints during interactive tasks

The ability to detect and differentiate breakpoints during task execution is critical for enabling defer-to-breakpoint policies within interruption management. In this work, we examine the feasibility of building statistical models that can detect and differentiate three granularities (types) of perceptually meaningful breakpoints during task execution, without having to recognize the underlying tasks. We collected ecological samples of task execution data, and asked observers to review the interaction in the collected videos and identify any perceived breakpoints and their type. Statistical methods were applied to learn models that map features of the interaction to each type of breakpoint. Results showed that the models were able to detect and differentiate breakpoints with reasonably high accuracy across tasks. Among many uses, our resulting models can enable interruption management systems to better realize defer-to-breakpoint policies for interactive, free-form tasks.

Mobile taskflow in context: a screenshot study of smartphone usage

The impact of interruptions on workflow and productivity has been extensively studied in the PC domain, but while fragmented user attention is recognized as an inherent aspect of mobile phone usage, little formal evidence exists of its effect on mobile productivity. Using a survey and a screenshot-based diary study we investigated the types of barriers people face when performing tasks on their mobile phones, the ways they follow up with such suspended tasks, and how frustrating the experience of task disruption is for mobile users. From 386 situated samples provided by 12 iPhone and 12 Pocket PC users, we distill a classification of barriers to the completion of mobile tasks. Our data suggest that moving to a PC to complete a phone task is common, yet not inherently problematic, depending on the task. Finally, we relate our findings to prior design guidelines for desktop workflow, and discuss how the guidelines can be extended to mitigate disruptions to mobile taskflow.

Oasis: A framework for linking notification delivery to the perceptual structure of goal-directed tasks

A notification represents the proactive delivery of information to a user and reduces the need to visually scan or repeatedly check an external information source. At the same time, notifications often interrupt user tasks at inopportune moments, decreasing productivity and increasing frustration. Controlled studies have shown that linking notification delivery to the perceptual structure of a users tasks can reduce these interruption costs. However, in these studies, the scheduling was always performed manually, and it was not clear whether it would be possible for a system to mimic similar techniques. This article contributes the design and implementation of a novel system called Oasis that aligns notification scheduling with the perceptual structure of user tasks. We describe the architecture of the system, how it detects task structure on the fly without explicit knowledge of the task itself, and how it layers flexible notification scheduling policies on top of this detection mechanism. The system also includes an offline tool for creating customized statistical models for detecting task structure. The value of our system is that it intelligently schedules notifications, enabling the reductions in interruption costs shown within prior controlled studies to now be realized by users in everyday desktop computing tasks. It also provides a test bed for experimenting with how notification management policies and other system functionalities can be linked to task structure.

Bored mondays and focused afternoons: the rhythm of attention and online activity in the workplace

While distractions using digital media have received attention in HCI, understanding engagement in workplace activities has been little explored. We logged digital activity and continually probed perspectives of 32 information workers for five days in situ to understand how attentional states change with context. We present a framework of how engagement and challenge in work relate to focus, boredom, and rote work. Overall, we find more focused attention than boredom in the workplace. Focus peaks mid-afternoon while boredom is highest in early afternoon. People are happiest doing rote work and most stressed doing focused work. On Mondays people are most bored but also most focused. Online activities are associated with different attentional states, showing different patterns at beginning and end of day, and before and after a mid-day break. Our study shows how rhythms of attentional states are associated with context and time, even in a dynamic workplace environment.