James Fogarty

Predicting human interruptibility with sensors

A person seeking another persons attention is normally able to quickly assess how interruptible the other person currently is. Such assessments allow behavior that we consider natural, socially appropriate, or simply polite. This is in sharp contrast to current computer and communication systems, which are largely unaware of the social situations surrounding their usage and the impact that their actions have on these situations. If systems could model human interruptibility, they could use this information to negotiate interruptions at appropriate times, thus improving human computer interaction.This article presents a series of studies that quantitatively demonstrate that simple sensors can support the construction of models that estimate human interruptibility as well as people do. These models can be constructed without using complex sensors, such as vision-based techniques, and therefore their use in everyday office environments is both practical and affordable. Although currently based on a demographically limited sample, our results indicate a substantial opportunity for future research to validate these results over larger groups of office workers. Our results also motivate the development of systems that use these models to negotiate interruptions at socially appropriate times.

Predicting human interruptibility with sensors: a Wizard of Oz feasibility study

A person seeking someone elses attention is normally able to quickly assess how interruptible they are. This assessment allows for behavior we perceive as natural, socially appropriate, or simply polite. On the other hand, todays computer systems are almost entirely oblivious to the human world they operate in, and typically have no way to take into account the interruptibility of the user. This paper presents a Wizard of Oz study exploring whether, and how, robust sensor-based predictions of interruptibility might be constructed, which sensors might be most useful to such predictions, and how simple such sensors might be.The study simulates a range of possible sensors through human coding of audio and video recordings. Experience sampling is used to simultaneously collect randomly distributed self-reports of interruptibility. Based on these simulated sensors, we construct statistical models predicting human interruptibility and compare their predictions with the collected self-report data. The results of these models, although covering a demographically limited sample, are very promising, with the overall accuracy of several models reaching about 78%. Additionally, a model tuned to avoiding unwanted interruptions does so for 90% of its predictions, while retaining 75% overall accuracy.

Presence versus availability: the design and evaluation of a context-aware communication client

Although electronic communication plays an important role in the modern workplace, the interruptions created by poorly-timed attempts to communicate are disruptive. Prior work suggests that sharing an indication that a person is currently busy might help to prevent such interruptions, because people could wait for a person to become available before attempting to initiate communication. We present a context-aware communication client that uses the built-in microphones of laptop computers to sense nearby speech. Combining this speech detection sensor data with location, computer, and calendar information, our system models availability for communication, a concept that is distinct from the notion of presence found in widely-used systems. In a 4 week study of the system with 26 people, we examined the use of this additional context. To our knowledge, this is the first-field study to quantitatively examine how people use automatically sensed context and availability information to make decisions about when and how to communicate with colleagues. Participants appear to have used the provided context to as an indication of presence, rather than considering availability. Our results raise the interesting question of whether sharing an indication that a person is currently unavailable will actually reduce inappropriate interruptions.

HydroSense: infrastructure-mediated single-point sensing of whole-home water activity

Recent work has examined infrastructure-mediated sensing as a practical, low-cost, and unobtrusive approach to sensing human activity in the physical world. This approach is based on the idea that human activities (e.g., running a dishwasher, turning on a reading light, or walking through a doorway) can be sensed by their manifestations in an environments existing infrastructures (e.g., a homes water, electrical, and HVAC infrastructures). This paper presents HydroSense, a low-cost and easily-installed single-point sensor of pressure within a homes water infrastructure. HydroSense supports both identification of activity at individual water fixtures within a home (e.g., a particular toilet, a kitchen sink, a particular shower) as well as estimation of the amount of water being used at each fixture. We evaluate our approach using data collected in ten homes. Our algorithms successfully identify fixture events with 97.9% aggregate accuracy and can estimate water usage with error rates that are comparable to empirical studies of traditional utility-supplied water meters. Our results both validate our approach and provide a basis for future improvements.

Examining the robustness of sensor-based statistical models of human interruptibility

Current systems often create socially awkward interruptions or unduly demand attention because they have no way of knowing if a person is busy and should not be interrupted. Previous work has examined the feasibility of using sensors and statistical models to estimate human interruptibility in an office environment, but left open some questions about the robustness of such an approach. This paper examines several dimensions of robustness in sensor-based statistical models of human interruptibility. We show that real sensors can be constructed with sufficient accuracy to drive the predictive models. We also create statistical models for a much broader group of people than was studied in prior work. Finally, we examine the effects of training data quantity on the accuracy of these models and consider tradeoffs associated with different combinations of sensors. As a whole, our analyses demonstrate that sensor-based statistical models of human interruptibility can provide robust estimates for a variety of office workers in a range of circumstances, and can do so with accuracy as good as or better than people. Integrating these models into systems could support a variety of advances in human computer interaction and computer-mediated communication.

Assieme: finding and leveraging implicit references in a web search interface for programmers

Programmers regularly use search as part of the development process, attempting to identify an appropriate API for a problem, seeking more information about an API, and seeking samples that show how to use an API. However, neither general-purpose search engines nor existing code search engines currently fit their needs, in large part because the information programmers need is distributed across many pages. We present Assieme, a Web search interface that effectively supports common programming search tasks by combining information from Web-accessible Java Archive (JAR) files, API documentation, and pages that include explanatory text and sample code. Assieme uses a novel approach to finding and resolving implicit references to Java packages, types, and members within sample code on the Web. In a study of programmers performing searches related to common programming tasks, we show that programmers obtain better solutions, using fewer queries, in the same amount of time spent using a general Web search interface.

The angle mouse: target-agnostic dynamic gain adjustment based on angular deviation

We present a novel method of dynamic C-D gain adaptation that improves target acquisition for users with motor impairments. Our method, called the Angle Mouse, adjusts the mouse C-D gain based on the deviation of angles sampled during movement. When angular deviation is low, the gain is kept high. When angular deviation is high, the gain is dropped, making the target bigger in motor-space. A key feature of the Angle Mouse is that, unlike most pointing facilitation techniques, it is target-agnostic, requiring no knowledge of target locations or dimensions. This means that the problem of distractor targets is avoided because adaptation is based solely on the users behavior. In a study of 16 people, 8 of which had motor impairments, we found that the Angle Mouse improved motor-impaired pointing throughput by 10.3% over the Windows default mouse and 11.0% over sticky icons. For able-bodied users, there was no significant difference among the three techniques, as Angle Mouse throughput was within 1.2% of the default. Thus, the Angle Mouse improved pointing performance for users with motor impairments while remaining unobtrusive for able-bodied users.

Examining task engagement in sensor-based statistical models of human interruptibility

The computer and communication systems that office workers currently use tend to interrupt at inappropriate times or unduly demand attention because they have no way to determine when an interruption is appropriate. Sensor?based statistical models of human interruptibility offer a potential solution to this problem. Prior work to examine such models has primarily reported results related to social engagement, but it seems that task engagement is also important. Using an approach developed in our prior work on sensor?based statistical models of human interruptibility, we examine task engagement by studying programmers working on a realistic programming task. After examining many potential sensors, we implement a system to log low?level input events in a development environment. We then automatically extract features from these low?level event logs and build a statistical model of interruptibility. By correctly identifying situations in which programmers are non?interruptible and minimizing cases where the model incorrectly estimates that a programmer is non?interruptible, we can support a reduction in costly interruptions while still allowing systems to convey notifications in a timely manner.

Aesthetic information collages: generating decorative displays that contain information

Normally, the primary purpose of an information display is to convey information. If information displays can be aesthetically interesting, that might be an added bonus. This paper considers an experiment in reversing this imperative. It describes the Kandinsky system which is designed to create displays which are first aesthetically interesting, and then as an added bonus, able to convey information. The Kandinsky system works on the basis of aesthetic properties specified by an artist (in a visual form). It then explores a space of collages composed from information bearing images, using an optimization technique to find compositions which best maintain the properties of the artists aesthetic expression.

A comprehensive study of frequency, interference, and training of multiple graphical passwords

Graphical password systems have received significant attention as one potential solution to the need for more usable authentication, but nearly all prior work makes the unrealistic assumption of studying a single password. This paper presents the first study of multiple graphical passwords to systematically examine frequency of access to a graphical password, interference resulting from interleaving access to multiple graphical passwords, and patterns of access while training multiple graphical passwords. We find that all of these factors significantly impact the ease of authenticating using multiple facial graphical passwords. For example, participants who accessed four different graphical passwords per week were ten times more likely to completely fail to authenticate than participants who accessed a single password once per week. Our results underscore the need for more realistic evaluations of the use of multiple graphical passwords, have a number of implications for the adoption of graphical password systems, and provide a new basis for comparing proposed graphical password systems.