Shaun K. Kane

Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques

Recent advances in touch screen technology have increased the prevalence of touch screens and have prompted a wave of new touch screen-based devices. However, touch screens are still largely inaccessible to blind users, who must adopt error-prone compensatory strategies to use them or find accessible alternatives. This inaccessibility is due to interaction techniques that require the user to visually locate objects on the screen. To address this problem, we introduce Slide Rule, a set of audio-based multi-touch interaction techniques that enable blind users to access touch screen applications. We describe the design of Slide Rule, our interaction techniques, and a user study in which 10 blind people used Slide Rule and a button-based Pocket PC screen reader. Results show that Slide Rule was significantly faster than the button-based system, and was preferred by 7 of 10 users. However, users made more errors when using Slide Rule than when using the more familiar button-based system.

Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities

Mobile devices provide people with disabilities new opportunities to act independently in the world. However, these empowering devices have their own accessibility challenges. We present a formative study that examines how people with visual and motor disabilities select, adapt, and use mobile devices in their daily lives. We interviewed 20 participants with visual and motor disabilities and asked about their current use of mobile devices, including how they select them, how they use them while away from home, and how they adapt to accessibility challenges when on the go. Following the interviews, 19 participants completed a diary study in which they recorded their experiences using mobile devices for one week. Our results show that people with visual and motor disabilities use a variety of strategies to adapt inaccessible mobile devices and successfully use them to perform everyday tasks and navigate independently. We provide guidelines for more accessible and empowering mobile device design.

Usable gestures for blind people: understanding preference and performance

Despite growing awareness of the accessibility issues surrounding touch screen use by blind people, designers still face challenges when creating accessible touch screen interfaces. One major stumbling block is a lack of understanding about how blind people actually use touch screens. We conducted two user studies that compared how blind people and sighted people use touch screen gestures. First, we conducted a gesture elicitation study in which 10 blind and 10 sighted people invented gestures to perform common computing tasks on a tablet PC. We found that blind people have different gesture preferences than sighted people, including preferences for edge-based gestures and gestures that involve tapping virtual keys on a keyboard. Second, we conducted a performance study in which the same participants performed a set of reference gestures. We found significant differences in the speed, size, and shape of gestures performed by blind people versus those performed by sighted people. Our results suggest new design guidelines for accessible touch screen interfaces.

Ability-Based Design: Concept, Principles and Examples

Current approaches to accessible computing share a common goal of making technology accessible to users with disabilities. Perhaps because of this goal, they may also share a tendency to centralize disability rather than ability. We present a refinement to these approaches called ability-based design that consists of focusing on ability throughout the design process in an effort to create systems that leverage the full range of human potential. Just as user-centered design shifted the focus of interactive system design from systems to users, ability-based design attempts to shift the focus of accessible design from disability to ability. Although prior approaches to accessible computing may consider users’ abilities to some extent, ability-based design makes ability its central focus. We offer seven ability-based design principles and describe the projects that inspired their formulation. We also present a research agenda for ability-based design.

Bonfire: a nomadic system for hybrid laptop-tabletop interaction

We present Bonfire, a self-contained mobile computing system that uses two laptop-mounted laser micro-projectors to project an interactive display space to either side of a laptop keyboard. Coupled with each micro-projector is a camera to enable hand gesture tracking, object recognition, and information transfer within the projected space. Thus, Bonfire is neither a pure laptop system nor a pure tabletop system, but an integration of the two into one new nomadic computing platform. This integration (1) enables observing the periphery and responding appropriately, e.g., to the casual placement of objects within its field of view, (2) enables integration between physical and digital objects via computer vision, (3) provides a horizontal surface in tandem with the usual vertical laptop display, allowing direct pointing and gestures, and (4) enlarges the input/output space to enrich existing applications. We describe Bonfires architecture, and offer scenarios that highlight Bonfires advantages. We also include lessons learned and insights for further development and use.

Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces

Using a mobile device while moving limits attention and motor ability and can result in reduced performance. Mobile devices that can sense and adapt to contextual factors such as movement may reduce this performance deficit. We performed two studies evaluating the feasibility of walking user interfaces (WUIs) that adapt their layout when the user is moving. In a pilot study with 6 users, we evaluated the effects of different button sizes on performance when walking while using a portable music player. Results showed significant interactions between size and movement. In the second study, 29 users evaluated the performance of a WUI that dynamically changed button sizes as the user moved. Results show that our dynamic user interface performs at the level of its component static interfaces without any additional penalty due to adaptation. This work adds to our design knowledge about walking user interfaces and provides lessons learned in evaluating mobile devices while walking in public spaces.

Working Overtime: Patterns of Smartphone and PC Usage in the Day of an Information Worker

Research has demonstrated that information workers often manage several different computing devices in an effort to balance convenience, mobility, input efficiency, and content readability throughout their day. The high portability of the mobile phone has made it an increasingly valuable member of this ecosystem of devices. To understand how future technologies might better support productivity tasks as people transition between devices, we examined the mobile phone and PC usage patterns of sixteen information workers across several weeks. Our data logs, together with follow-up interview feedback from four of the participants, confirm that the phone is highly leveraged for digital information needs beyond calls and SMS, but suggest that these users do not currently traverse the device boundary within a given task.

Design patterns for sociality in human-robot interaction

We propose that Christopher Alexanders idea of design patterns can benefit the emerging field of HRI. We first discuss four features of design patterns that appear particularly useful. For example, a pattern should be specified abstractly enough such that many different instantiations of the pattern can be uniquely realized in the solution to specific problems in context. Then, after describing our method for generating patterns, we offer and describe eight possible design patterns for sociality in human robot interaction: initial introduction, didactic communication, in motion together, personal interests and history, recovering from mistakes, reciprocal turn-taking in game context, physical intimacy, and claiming unfair treatment or wrongful harms. We also discuss the issue of validation of design patterns. If a design pattern program proves successful, it will provide HRI researchers with basic knowledge about human robot interaction, and save time through the reuse of patterns to achieve high levels of sociality.

Access overlays: improving non-visual access to large touch screens for blind users

Many touch screens remain inaccessible to blind users, and those approaches to providing access that do exist offer minimal support for interacting with large touch screens or spatial data. In this paper, we introduce a set of three software-based access overlays intended to improve the accessibility of large touch screen interfaces, specifically interactive tabletops. Our access overlays are called edge projection, neighborhood browsing, and touch-and-speak. In a user study, 14 blind users compared access overlays to an implementation of Apples VoiceOver screen reader. Our results show that two of our techniques were faster than VoiceOver, that participants correctly answered more questions about the screens layout using our techniques, and that participants overwhelmingly preferred our techniques. We developed several applications demonstrating the use of access overlays, including an accessible map kiosk and an accessible board game.

Envisioning systemic effects on persons and society throughout interactive system design

The design, development, and deployment of interactive systems can substantively impact individuals, society, and the natural environment, now and potentially well into the future. Yet, a scarcity of methods exists to support long-term, emergent, systemic thinking in interactive design practice. Toward addressing this gap, we propose four envisioning criteria --- stakeholders, time, values, and pervasiveness -- distilled from prior work in urban planning, design noir, and Value Sensitive Design. We characterize how the criteria can support systemic thinking, illustrate the integration of the envisioning criteria into established design practice (scenariobased design), and provide strategic activities to serve as generative envisioning tools. We conclude with suggestions for use and future work. Key contributions include: 1) four envisioning criteria to support systemic thinking, 2) value scenarios (extending scenario-based design), and 3) strategic activities for engaging the envisioning criteria in interactive system design practice.