Brian Bridges

Sound, Ecological Affordances and Embodied Mappings in Auditory Display

The third wave of HCI has seen the widespread adoption of design principles borrowed from and informed by breakthroughs in the field of embodied cognitive science. These developments have taken place primarily in the contexts of visual media and interaction, but they are also of importance to the design of auditory displays and interactive systems in which sounds plays a dominant role, where they open up new affordances by which information might be communicated to a listener. This chapter examines the relationship between auditory display, sonic interaction design and embodied cognition and explores frameworks from embodied cognition that might inform the design of more informative auditory displays in a variety of contexts. It will do so by addressing these issues from an interdisciplinary perspective, bringing together insights from cognitive science and philosophy, general HCI and computer science, along with music theory and practice.

SceneMaker: Creative Technology for Digital StoryTelling

The School of Creative Arts & Technologies at Ulster University (Magee) has brought together the subject of computing with creative technologies, cinematic arts (film), drama, dance, music and design in terms of research and education. We propose here the development of a flagship computer software platform, SceneMaker, acting as a digital laboratory workbench for integrating and experimenting with the computer processing of new theories and methods in these multidisciplinary fields. We discuss the architecture of SceneMaker and relevant technologies for processing within its component modules. SceneMaker will enable the automated production of multimodal animated scenes from film and drama scripts or screenplays. SceneMaker will highlight affective or emotional content in digital storytelling with particular focus on character body posture, facial expressions, speech, non-speech audio, scene composition, timing, lighting, music and cinematography. Applications of SceneMaker include automated simulation of productions and education and training of actors, screenwriters and directors.