Koray Tahiroglu

Perceived physicality in audio-enhanced force input

This paper investigates how the perceived physicality of the action of applying force with a finger on a rigid surface (such as on a force-sensing touch screen) can be enhanced using real-time synthesized audio feedback. A selection of rich and evocative audio designs was used. Additionally, audio-tactile cross-modal integration was encouraged, by observing that the main rules of multisensory integration were supported. The study conducted showed that richness of perceived physicality increased considerably, mostly in its auditory expression (what pressing sounded like). In addition, in many instances it was observed that the haptic expression of physicality also increased (what pressing felt like), including some perception of compliance. This last result was particularly interesting as it showed that audio-tactile cross-modal integration might be present.

Design and evaluation of human-computer rhythmic interaction in a tutoring system

of Human–Computer Rhythmic Interaction in a Tutoring System Antti Jylha,∗ Inger Ekman,† Cumhur Erkut,∗ and Koray Tahiroglu∗∗ ∗ Department of Signal Processing and Acoustics Aalto University School of Electrical Engineering P.O. Box 13000, FI-00076 Aalto, Finland {antti.jylha, cumhur.erkut}@aalto.fi †Center for Knowledge and Innovation Research (CKIR) Aalto University School of Economics P.O. Box 21255, FI-00076 Aalto, Finland inger.ekman@aalto.fi ∗∗Media Lab, Department of Media Aalto University School of Art and Design P.O. Box 31000, FI-00076 Aalto, Finland koray.tahiroglu@aalto.fi

NOISA: A Novel Intelligent System Facilitating Smart Interaction

In this paper, we describe NOISA (Network of Intelligent Sonic Agents). NOISA is an intelligent system that acts to maintain and deepen the users engagement with digital artefacts by learning from the users actions and behavioural patterns in the moment of interaction. It facilitates a smart interaction by monitoring users bodily movements, facial expressions and control inputs. We present our model and system in a musical context, interfaced with our digital musical instrument (DMI). Our concept can be further extended to possible application areas in Human Computer Interaction (HCI) research field.

AHNE: a novel interface for spatial interaction

In this paper we describe AHNE (Audio-Haptic Navigation Environment). It is a three-dimensional user interface (3D UI) for manipulating virtual sound objects with natural gestures in a real environment. AHNE uses real-time motion tracking and custom-made glove controllers as input devices, and auditory and haptic feedback as the output. We present the underlying system and a possible use for the interface as a musical controller.

User experience and usage scenarios of audio-tactile interaction with virtual objects in a physical environment

Augmented reality (AR) systems enable new user experiences while the user is interacting with virtual objects in the physical space. The virtual objects have mostly been presented visually, overlaid on the physical world. In this paper, we present an explorative user study of a prototype system AHNE with the aim to understand the user experience and usage potential of non-visual audio-tactile augmented reality interactions. The qualitative results reveal that non-visual interaction was experienced confusing at first, but also fun, playful, and exciting. Even though audio feedback clearly dominates the spatial interaction experience, tactile feedback makes the virtual objects feel more concrete. The ideated and evaluated set of 23 usage scenarios includes both game-like and pragmatic application ideas for individual and social usage. The results can be used to inform the design of novel AR applications that utilise human senses beyond vision.

MARSUI: malleable audio-reactive shape-retaining user interface

MARSUI is a hardware deformable prototype exhibiting plastic (shape-retaining) behavior. It can track the shape that the user creates when deforming it. We envision that a set of predefined shapes could be mapped onto particular applications and functions. In its current implementation, we present three shapes that MARSUI can be deformed into: circular band, flat surface and sharp bent. These shapes map respectively onto the following applications: wristwatch, mobile phone and media player. Since the malleable interface can also take other forms, feedback plays an important role in guiding the user towards the predefined shapes. In this paper, we focus on investigating the possibilities that auditory feedback could offer in guiding the user towards reaching the intended shapes.

Investigating tangential access for location-based digital cultural heritage applications

Abstract As location-based digital cultural heritage applications are beginning to be applied outside ‘traditional’ cultural heritage sites, there is an increased need to consider their use amongst people who may be interested in the cultural heritage of a site, but have that interest as a tangential, opportunistic, aspect of their visit. We outline why this is important, and present issues in evaluating it. We then introduce Explore: a mobile, location-based, digital cultural heritage application for the Finnish recreational island of Seurasaari. By considering how the common visitor trajectory of a museum visit can be mapped onto a visit to the island, we were able to evaluate Explore with participants who were not primarily there to access cultural heritage. Evaluation with 45 participants over 5 days identified how the low immersion techniques used allowed participants to fit accessing cultural heritage around their visit, how participants managed group tension between those more and those less interested in Explore, as well as issues around considering the use of cultural heritage applications as a tangential purpose of a visit.

Facilitating the Musician's Engagement with New Musical Interfaces: Counteractions in Music Performance

The level of engagement of a musician performing on an instrument is related to the degree of satisfaction derived from that activity. With our work, we aim to assist musicians performing live on a new musical instrument, Network of Interactive Sonic Agents (NOISA), by helping them maintain or increase their level of engagement with the activity. The NOISA system can learn from performers through observation and estimate their engagement level in real time. The new response module, which includes new sound design, comparison of gestures, and audio-analysis features, can also decide what action to take, and when to implement it, to help the performer recover from lowering engagement levels. We report on a formative user study that evaluates the impact of this response module.

Sound in New Media and Design Studies

Introduction The effect of new media on the computerization of cultural practices has moved to a new level and has become more open, common, and prevailing as we deal with new forms of artistic creation and design processes in supporting higher education. Massive changes have occurred in the way frameworks for new media studies have been constructed, and they have had a significant effect on higher education in which sound studies are taught. In the past, sound has not received much attention in new media research and studies. Instead, the emphasis has been on visual processing and representations. Today, imagining an auditory paradigm that targets new modes of reflection in sound domain activities—activities that result in new forms of interactions in cultural, aesthetic, urban, and media contexts—is easy. Computational tools and new media play an increasing role in this process because they have become ubiquitous in people’s use of computers and new technologies to express thoughts and ideas.1 From an auditory media perspective, digitization of sound has engendered various opportunities in the production and distribution of art and design work. This new form in auditory media has resulted in a growing scholarly interest in interdisciplinary research and studies, just as it brings up further discussions about why sound should be considered a part of design practice and studies. Not just in terms of sound as a design practice, but design practices in general have begun to address demands for products, services, art, and entertainment—designed as part of everyday culture—that has emotive, functional, intriguing, and beautiful qualities.2 Sound as a design and mediating element can manifest the qualities of the design object and open up new ways for us to consider our interactions with it. With increased knowledge and understanding of sound as a design element in design studies, students become better equipped to enrich the usability, attractiveness, and communicative qualities of designed artifacts. 1 Lev Manovich, The Language of the New Media (Cambridge, MA: The MIT Press, 2001). 2 Bill Moggridge, Designing Interactions (Cambridge, MA: The MIT Press, 2007).