Andrew Johnston

sPeAK-MAN: towards popular gameplay for speech therapy

Current speech therapy treatments are not easily accessible to the general public due to cost and demand. Therapy sessions are also laborious and maintaining motivation of patients is hard. We propose using popular games and speech recognition technology for speech therapy in an individualised and accessible manner. sPeAK-MAN is a Pac-Man-like game with a core gameplay mechanic that incorporates vocalisation of words generated from a pool commonly used in clinical speech therapy sessions. Other than improving engagement, sPeAK-MAN aims to provide real-time feedback on the vocalisation performance of patients. It also serves as an initial prototype to demonstrate the possibilities of using familiar popular gameplay (instead of building one from scratch) for rehabilitation purposes.

Amplifying reflective thinking in musical performance

In this paper we report on the development of tools that encourage both a creative and reflective approach to music-making and musical skill development. A theoretical approach to musical skill development is outlined and previous work in the area of music visualisation is discussed. In addition the characterisation of music performance as a type of design problem is discussed and the implications of this position for the design of tools for musicians are outlined. Prototype tools, the design of which is informed by the theories and previous work, are described and some preliminary evaluation of their effectiveness is discussed.Future directions are outlined.

Keeping Research in Tune with Practice

This chapter examines the relationship between practice, research and evaluation with reference to the design and development of interactive systems for use in a large-scale dance work developed in collaboration with Stalker Theatre, Encoded. Strategies for keeping creative practice and the associated research aligned with the concerns of practicing artists are presented. These strategies include working with experienced, high-calibre artists, applying user-centred, iterative design and development approaches, and carefully examining the impact of new technologies and techniques on performers’ practices and experiences. Findings from an examination of Stalker Theatre’s experiences with the Encoded systems indicate that the use of interactive systems in live performance has a significant impact on the way performances are developed, staged and structured.

Storytelling with Interactive Physical Theatre: A case study of Dot and the Kangaroo

This paper examines the way movement based interactive visuals were used as a storytelling device in the physical theatre production of Creature: Dot and the Kangaroo. A number of performers and artists involved in the production were interviewed and their perceptions of the interactive technology have been contrasted against a similar study into abstract dance. The animated backgrounds and interactive animal graphics projected onto the stage were found to reduce the density of script by describing the location of action and spirit of the character, reducing the necessity for this to be spoken. Peak moments of the show were identified by those interviewed and a scene analysis revealed that the most successful scenes featured a more integrated storytelling where the interaction between performers and the digital projections portrayed a key narrative message.

Remote Control of Complex Interactive Art Installations

Movement based interactive artworks are capable of instantly engaging audiences by reacting to physical motion consistently with real-world physics. Sustaining this engagement, however, requires a constant alteration of both the output and interaction aesthetics. Mobile devices (such as the iPad or iPhone) can be used to control the often-overwhelming plethora of parameters found in many interactive systems. The effect that mobile control of these parameters has on the inception, refinement and live performance of two separate art works is examined. An open-source dynamic remote control system is being developed to further facilitate the creative development and performance of interactive artwork as demonstrated by these case studies.

Designing for technicians working in the field: 8 usability heuristics for mobile application design

Mobile applications are frequently used by technicians and logistics personnel to access documentation and communicate and log information about the work they do in the field. Currently, however, there are no context-specific usability heuristics for use by designers who are building mobile applications for this sector. By conducting contextual inquiries with technicians and logistics personnel who use mobile applications for their day to day work, we identified specific usability issues affecting the use of these applications. From this research, we propose a set of eight heuristics for use by designers and developers creating mobile applications for users in this area.

Opportunities for Practice-Based Research in Musical Instrument Design

This paper considers the relationship between design, practice and research in the area of New Interfaces for Musical Expression (NIME). The author argues that NIME practitioner-researchers should embrace the instability and dynamism inherent in digital musical interactions in order to explore and document the evolving processes of musical expression.

Sound stream

Sound Stream is a work for oboe and interactive system which blurs the boundaries between instrumental performance, gestural interaction and dance. Largely improvised within a predetermined high-level structure, the work explores the relationships between acoustic sounds, instrumental gestures and expressive gestures and between sound and image. The interactive system uses motion tracking, real-time fluid simulation and fluid-controlled sound synthesis to produce an immersive performance environment.

Visual feedback of acoustic data for speech therapy: model and design parameters

Feedback, usually of a verbal nature, is important for speech therapy sessions. Some disadvantages exist however with traditional methods of speech therapy, and visual feedback of acoustic data is a useful alternative that can be used to complement typical clinical sessions. Visual feedback has been investigated before, and in this paper we propose several new prototypes. From these prototypes we develop an iterative model of analysing the design of feedback systems by examining the feedback process. From this iterative model, we then extract methods to inform design of visual feedback systems for speech therapy.

Fleeting Film: Using Story to Seek Archival Permanence in the Transitory and Globalized Digital Visual Effects Industry

Abstract Archiving is a long-standing vocation, founded on principles such as provenance, original order, truth, evidence, preservation and permanence. A far cry from the visual spectacle and movable feast of film visual effects (VFX)—a transitory and globalized industry of disposable firms, ever-advancing technologies and a roving workforce which craft digital animations and seamless effects for the big screen. In this paper we utilize the concept of “story” as a premise to bring together the seemingly different vocations of archival science and film VFX. Through an exploration of digital film production and archival practice under the context of storytelling, we aim to highlight the need for archivists to work with the VFX industry to ensure evidence of this culturally significant aspect of filmmaking and cinema discourse is preserved into the future. As well present the argument that archives are more than collections of historical evidence. Archives are story—and archivists are storytellers.