Tony Stockman

A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research

This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user-friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in general.

Auditory display design-An investigation of a design pattern approach

We present the evaluation of a methodological design framework that supports expert and novice designers in creating auditory artefacts in human-technology interaction. We first motivate the development of our framework by analysing available guidance and the current practice in the field. Subsequently, we recapitulate on the design of the framework-paco, pattern design in the context space-and present its key concepts and methods. The evaluation of paco aimed to investigate how useful this framework is in a real-world environment. It was conducted in two phases: experts in auditory display design first captured successful designs through paco and created a body of design patterns. These patterns were subsequently used in a controlled experiment with novice designers who were given a design task that forced them to use audio. The results demonstrate that paco has facilitated the transfer of design knowledge and good practice from experts to novices through design patterns. The context space, a key concept in paco, improves the contextual awareness of designers and provides an organising principle for problems, patterns and artefacts. We close by reflecting on the results and discussing future lines of research.

Using think aloud protocol with blind users:: a case for inclusive usability evaluation methods

There is a need to assess the applicability of conventional Usability Evaluation Methods to users with disabilities, given the growing importance of involving these users in the usability evaluation process. We found that conventional Think Aloud Protocol cannot be used as is, and will require modification to be useful, when evaluating websites with blind users.

Telematic dinner party: designing for togetherness through play and performance

There is an increasing desire to remain connected when physically distant and computer-mediated communication (CMC) is one means of satisfying this desire. In particular, there is a growing trend for individuals to use commercially available technology to connect with friends and family in social and leisure settings. Drawing on this trend, performative arts and existing telecommunications research, we identify the social practice of sharing a meal together as ripe for reinterpretation within CMC. We explore the opportunities to design a technology platform that supports remote guests in experiencing togetherness and playfulness within the practices of a traditional dinner party. Through both visual and aural channels as well as remote agency, the dinner guests were able to share a holistic telematic dining experience comparable to a traditional co-presence dinner. Based on the findings, we propose that one must consider the social structure and cultural background of users to inform the design of a technological intervention.

Constructing relational diagrams in audio: the multiple perspective hierarchical approach

Although research on non-visual access to visually represented information is steadily growing, very little work has investigated how such forms of representation could be constructed through non-visual means. We discuss in this paper our approach for providing audio access to relational diagrams using multiple perspective hierarchies, and describe the design of two interaction strategies for constructing and manipulating such diagrams through this approach. A comparative study that we conducted with sighted users showed that a non-guided strategy allowed for significantly faster interaction times, and that both strategies supported similar levels of diagram comprehension. Overall, the reported study revealed that using multiple perspective hierarchies to structure the information encoded in a relational diagram enabled users construct and manipulate such information through an audio-only interface, and that combining aspects from the guided and the non-guided strategies could support greater usability.

Designing with and for people living with visual impairments: audio-tactile mock-ups, audio diaries and participatory prototyping

Methods used to engage users in the design process often rely on visual techniques, such as paper prototypes, to facilitate the expression and communication of design ideas. The visual nature of these tools makes them inaccessible to people living with visual impairments. In addition, while using visual means to express ideas for designing graphical interfaces is appropriate, it is harder to use them to articulate the design of non-visual displays. In this article, we present an approach to conducting participatory design with people living with visual impairments incorporating various techniques to help make the design process accessible. We reflect on the benefits and challenges that we encountered when employing these techniques in the context of designing cross-modal interactive tools.

Interactive hierarchy-based auditory displays for accessing and manipulating relational diagrams

An approach to designing hierarchy-based auditory displays that supports non-visual interaction with relational diagrams is presented. The approach is motivated by an analysis of the functional and structural properties of relational diagrams in terms of their role as external representations. This analysis informs the design of a multiple perspective hierarchy-based model that captures modality independent features of a diagram when translating it into an audio accessible form. The paper outlines design lessons learnt from two user studies that were conducted to evaluate the proposed approach.

Participatory Design with Blind Users: A Scenario-Based Approach

Through out the design process, designers have to consider the needs of potential users. This is particularly important, but rather harder, when the designers interact with the artefact to-be-designed using different senses or devices than the users, for example, when sighted designers are designing an artefact for use by blind users. In such cases, designers have to ensure that the methods used to engage users in the design process and to communicate design ideas are accessible. In this paper, we describe a participatory approach with blind users based on the use of a scenario and the use of dialogue-simulated interaction during the development of a search interface. We achieved user engagement in two ways: firstly, we involved a blind user with knowledge of assistive technologies in the design team and secondly, we used a scenario as the basis of a dialogue between the designers and blind users to simulate interaction with the proposed search interface. Through this approach, we were able to verify requirements for the proposed search interface and blind searchers were able to provide formative feedback, to critique design plans and to propose new design ideas based on their experience and expertise with assistive technologies. In this paper, we describe the proposed scenario-based approach and examine the types of feedback gathered from its evaluation with blind users. We also critically reflect on the benefits and limitations of the approach, and discuss practical considerations in its application.

Effect of latency on performer interaction and subjective quality assessment of a digital musical instrument

When designing digital musical instruments the importance of low and consistent action-to-sound latency is widely accepted. This paper investigates the effects of latency (0-20ms) on instrument quality evaluation and performer interaction. We present findings from an experiment conducted with musicians who performed on an percussive digital musical instrument with variable amounts of latency. Three latency conditions were tested against a zero latency condition, 10ms, 20ms and 10ms ± 3ms jitter. The zero latency condition was significantly rated more positively than the 10ms with jitter and 20ms latency conditions in six quality measures, emphasising the importance of not only low, but stable latency in digital musical instruments. There was no significant difference in rating between the zero latency condition and 10ms condition. A quantitative analysis of timing accuracy in a metronome task under latency conditions showed no significant difference in mean synchronisation error. This suggests that the 20ms and 10ms with jitter latency conditions degrade subjective impressions of an instrument, but without significantly affecting the timing performance of our participants. These findings are discussed in terms of control intimacy and instrument transparency.

Design to support interpersonal communication in the special educational needs classroom

This paper describes our Experience-centered Design (ECD) inquiry into the current and potential role of digital photography to support interpersonal communication and expression in a class at a mixed special education needs school. Presented as a case study, we describe five classroom-based Creative Photography workshops that engaged pupils with a broad range of complex special needs, along with classroom staff. We further describe how, from these workshops, we generated a set of qualitative considerations for the design of digital photographic tools to support interpersonal communication and expression in this setting. Additionally, we present the preliminary evaluation of a photo-sorting tool that we developed in response. Our case study demonstrates how an ECD approach can guide an interaction design process in a special education needs setting, supporting interaction designers in understanding and responding pragmatically to the complex and dynamic interactions at play between the stakeholders.