Matthew Kay

Lullaby: a capture & access system for understanding the sleep environment

The bedroom environment can have a significant impact on the quality of a persons sleep. Experts recommend sleeping in a room that is cool, dark, quiet, and free from disruptors to ensure the best quality sleep. However, it is sometimes difficult for a person to assess which factors in the environment may be causing disrupted sleep. In this paper, we present the design, implementation, and initial evaluation of a capture and access system, called Lullaby. Lullaby combines temperature, light, and motion sensors, audio and photos, and an off-the-shelf sleep sensor to provide a comprehensive recording of a persons sleep. Lullaby allows users to review graphs and access recordings of factors relating to their sleep quality and environmental conditions to look for trends and potential causes of sleep disruptions. In this paper, we report results of a feasibility study where participants (N=4) used Lullaby in their homes for two weeks. Based on our experiences, we discuss design insights for sleep technologies, capture and access applications, and personal informatics tools.

Unequal Representation and Gender Stereotypes in Image Search Results for Occupations

Information environments have the power to affect peoples perceptions and behaviors. In this paper, we present the results of studies in which we characterize the gender bias present in image search results for a variety of occupations. We experimentally evaluate the effects of bias in image search results on the images people choose to represent those careers and on peoples perceptions of the prevalence of men and women in each occupation. We find evidence for both stereotype exaggeration and systematic underrepresentation of women in search results. We also find that people rate search results higher when they are consistent with stereotypes for a career, and shifting the representation of gender in image search results can shift peoples perceptions about real-world distributions. We also discuss tensions between desires for high-quality results and broader societal goals for equality of representation in this space.

SleepTight: low-burden, self-monitoring technology for capturing and reflecting on sleep behaviors

Manual tracking of health behaviors affords many benefits, including increased awareness and engagement. However, the capture burden makes long-term manual tracking challenging. In this study on sleep tracking, we examine ways to reduce the capture burden of manual tracking while leveraging its benefits. We report on the design and evaluation of SleepTight, a low-burden, self-monitoring tool that leverages the Androids widgets both to reduce the capture burden and to improve access to information. Through a four-week deployment study (N = 22), we found that participants who used SleepTight with the widgets enabled had a higher sleep diary compliance rate (92%) than participants who used SleepTight without the widgets (73%). In addition, the widgets improved information access and encouraged self-reflection. We discuss how to leverage widgets to help people collect more data and improve access to information, and more broadly, how to design successful manual self-monitoring tools that support self-reflection.

Beyond Weber's Law: A Second Look at Ranking Visualizations of Correlation

Models of human perception - including perceptual “laws” - can be valuable tools for deriving visualization design recommendations. However, it is important to assess the explanatory power of such models when using them to inform design. We present a secondary analysis of data previously used to rank the effectiveness of bivariate visualizations for assessing correlation (measured with Pearsons r) according to the well-known Weber-Fechner Law. Beginning with the model of Harrison et al. [1], we present a sequence of refinements including incorporation of individual differences, log transformation, censored regression, and adoption of Bayesian statistics. Our model incorporates all observations dropped from the original analysis, including data near ceilings caused by the data collection process and entire visualizations dropped due to large numbers of observations worse than chance. This model deviates from Webers Law, but provides improved predictive accuracy and generalization. Using Bayesian credibility intervals, we derive a partial ranking that groups visualizations with similar performance, and we give precise estimates of the difference in performance between these groups. We find that compared to other visualizations, scatterplots are unique in combining low variance between individuals and high precision on both positively- and negatively correlated data. We conclude with a discussion of the value of data sharing and replication, and share implications for modeling similar experimental data.

When (ish) is My Bus?: User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems

Users often rely on realtime predictions in everyday contexts like riding the bus, but may not grasp that such predictions are subject to uncertainty. Existing uncertainty visualizations may not align with user needs or how they naturally reason about probability. We present a novel mobile interface design and visualization of uncertainty for transit predictions on mobile phones based on discrete outcomes. To develop it, we identified domain specific design requirements for visualizing uncertainty in transit prediction through: 1) a literature review, 2) a large survey of users of a popular realtime transit application, and 3) an iterative design process. We present several candidate visualizations of uncertainty for realtime transit predictions in a mobile context, and we propose a novel discrete representation of continuous outcomes designed for small screens, quantile dotplots. In a controlled experiment we find that quantile dotplots reduce the variance of probabilistic estimates by ~1.15 times compared to density plots and facilitate more confident estimation by end-users in the context of realtime transit prediction scenarios.

Cognitive rhythms: unobtrusive and continuous sensing of alertness using a mobile phone

Throughout the day, our alertness levels change and our cognitive performance fluctuates. The creation of technology that can adapt to such variations requires reliable measurement with ecological validity. Our study is the first to collect alertness data in the wild using the clinically validated Psychomotor Vigilance Test. With 20 participants over 40 days, we find that alertness can oscillate approximately 30% depending on time and body clock type and that Daylight Savings Time, hours slept, and stimulant intake can influence alertness as well. Based on these findings, we develop novel methods for unobtrusively and continuously assessing alertness. In estimating response time, our model achieves a root-mean-square error of 80.64 milliseconds, which is significantly lower than the 500ms threshold used as a standard indicator of impaired cognitive ability. Finally, we discuss how such real-time detection of alertness is a key first step towards developing systems that are sensitive to our biological variations.

Mobile manifestations of alertness: connecting biological rhythms with patterns of smartphone app use

Our body clock causes considerable variations in our behavioral, mental, and physical processes, including alertness, throughout the day. While much research has studied technology usage patterns, the potential impact of underlying biological processes on these patterns is under-explored. Using data from 20 participants over 40 days, this paper presents the first study to connect patterns of mobile application usage with these contributing biological factors. Among other results, we find that usage patterns vary for individuals with different body clock types, that usage correlates with rhythms of alertness, that app use features such as duration and switching can distinguish periods of low and high alertness, and that app use reflects sleep interruptions as well as sleep duration. We conclude by discussing how our findings inform the design of biologically-friendly technology that can better support personal rhythms of performance.

How Good is 85%?: A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy

Many HCI and ubiquitous computing systems are characterized by two important properties: their output is uncertain-it has an associated accuracy that researchers attempt to optimize-and this uncertainty is user-facing-it directly affects the quality of the user experience. Novel classifiers are typically evaluated using measures like the F1 score-but given an F-score of (e.g.) 0.85, how do we know whether this performance is good enough? Is this level of uncertainty actually tolerable to users of the intended application-and do people weight precision and recall equally? We set out to develop a survey instrument that can systematically answer such questions. We introduce a new measure, acceptability of accuracy, and show how to predict it based on measures of classifier accuracy. Out tool allows us to systematically select an objective function to optimize during classifier evaluation, but can also offer new insights into how to design feedback for user-facing classification systems (e.g., by combining a seemingly-low-performing classifier with appropriate feedback to make a highly usable system). It also reveals potential issues with the ubiquitous F1-measure as applied to user-facing systems.

Special Interest Group on Transparent Statistics in HCI

Transparent statistics is a philosophy of statistical reporting whose purpose is scientific advancement rather than persuasion. We propose a SIG to discuss problems and limitations in statistical practices in HCI and options for moving the field towards clearer and more reliable ways of writing about experiments.

Moving Transparent Statistics Forward at CHI

Transparent statistics is a philosophy of statistical reporting whose purpose is scientific advancement rather than persuasion. We ran a SIG at CHI 2016 to discuss problems and limitations in statistical practices in HCI and options for moving the field towards clearer and more reliable ways of writing about experiments, and received an overwhelming response. This SIG resulted in rough drafts of reviewer guidelines, resources for authors, and other suggestions for advancing a vision of transparent statistics within the field; this year, we propose a concentrated one-day writing workshop to develop those documents into a polished state with input from a diverse cross-section of the CHI community.