Jari Kleimola

Worlds of information: designing for engagement at a public multi-touch display

In designing for engagement at a public multi-touch installation, we identified supporting multiple users and allowing for gradual discovery as challenges. In this paper, we present Worlds of Information, a multi-touch application featuring 3D Worlds, which provide access to different content. These 3D widgets gradually unfold and allow for temporal navigation of multimedia in parallel, while also providing a 2D plane where media can be shared. We report on a field trial at an exhibition using questionnaires and video ethnography. We studied engagement through questions adapted from Flow, Presence and Intrinsic Motivation questionnaires, which showed that users, overall, had a positive and social experience with the installation. The worlds effectively invited multiple users and provided for parallel interaction. While functionality was discovered gradually through social learning, the study demonstrates the challenges of designing multi-touch applications for walk-up-and-use displays.

Interactive sonification of synchronisation of motoric behaviour in social active listening to music with mobile devices

This paper evaluates three different interactive sonifications of dyadic coordinated human rhythmic activity. An index of phase synchronisation of gestures was chosen as coordination metric. The sonifications are implemented as three prototype applications exploiting mobile devices: Sync’n’Moog, Sync’n’Move, and Sync’n’Mood. Sync’n’Moog sonifies the phase synchronisation index by acting directly on the audio signal and applying a nonlinear time-varying filtering technique. Sync’n’Move intervenes on the multi-track music content by making the single instruments emerge and hide. Sync’n’Mood manipulates the affective features of the music performance. The three sonifications were also tested against a condition without sonification.

User-centric context-aware mobile applications for embodied music listening

This paper surveys a collection of sample applications for networked user-centric context-aware embodied music listening. The applications have been designed and developed in the framework of the EU-ICT Project SAME (www.sameproject.eu) and have been presented at Agora Festival (IRCAM, Paris, France) in June 2009. All of them address in different ways the concept of embodied, active listening to music, i.e., enabling listeners to interactively operate in real-time on the music content by means of their movements and gestures as captured by mobile devices. In the occasion of the Agora Festival the applications have also been evaluated by both expert and non-expert users.

Reducing Aliasing from Synthetic Audio Signals Using Polynomial Transition Regions

Sampling of discontinuous audio signals with rich spectra is a valuable asset in subtractive synthesis, but results in aliasing distortion. This letter proposes an aliasing-reduction technique, which is cost-effective, transient-free, and extensible to various discontinuities. It replaces the samples on a finite region around each discontinuity with values taken from a smooth polynomial, based on a novel interpretation of the differentiated polynomial waveform (DPW) method. In the musical pitch range, the number of operations in the proposed method is at least 40% smaller than in the DPW algorithm. The method is widely usable in sound synthesis applications.

Feedback Amplitude Modulation Synthesis

A recently rediscovered sound synthesis method, which is based on feedback amplitude modulation (FBAM), is investigated. The FBAM system is interpreted as a periodically linear time-varying digital filter, and its stability, aliasing, and scaling properties are considered. Several novel variations of the basic system are derived and analyzed. Separation of the input and the modulation signals in FBAM structures is proposed which helps to create modular sound synthesis and digital audio effects applications. The FBAM is shown to be a powerful and versatile sound synthesis principle, which has similarities to the established distortion synthesis methods, but which is also essentially different from them.

Discrete-time modelling of the moog sawtooth oscillator waveform

Discrete-time modelling strategies of analogue Moog sawtooth oscillator waveforms are presented. Two alternative approaches suitable for real-time implementation are proposed, one modelling the analogue waveform in time domain using phase distortion synthesis and another matching the spectrum of an existing antialiasing sawtooth oscillator to the corresponding analogue spectrum using a first-order IIR post-equalising filter. A parameter estimation procedure for both approaches is explained and performed. Performance evaluation using polynomial fits for the estimated parameters is carried out, and good matches between the model outputs and recorded waveforms are obtained. The best match of the tested algorithms is produced by the phase distortion model and by post-equalising the fourth-order B-spline bandlimited step function sawtooth oscillator.

Human movement expressivity for mobile active music listening

In this paper we describe the SAME networked platform for context-aware, experience-centric mobile music applications, and we present an implementation of the SAME active music listening paradigm: the Mobile Conductor. It allows the user to express herself in conducting a virtual ensemble playing a MIDI piece of music by means of her mobile phone. The mobile phone detects the user’s hand movement and molds the music performance style by modulating its speed, volume, and intonation.

Web browser as platform for audiovisual performances

The present study aims to address the following research question: how to create a tool for audiovisual performance, allowing for real-time usage of shared online visual resources, which can be customizable, and used across a variety of different hardware platforms? To address this issue, we have developed AVVX (AudioVisual Vector eXchange), a novel application for audiovisual performances, based on open web technologies such as HTML5, JavaScript and SVG (Scalable Vector Graphics). This paper contextualizes AVVX with related work and technologies, and then presents the design and development of the software. Taking as a starting point a workshop conducted with AVVX, the project has been evaluated by means of a questionnaire and user tests. The results of the tests indicate that the web browser, together with open web technologies, can provide a foundation for a customizable, content-sharing and multi-platform approach to audiovisual performance.

TouchModifier: enriched multi-touch gestures for tablet browsers

Touch-based web browsing with tablet devices is not yet utilizing its full potential. This paper introduces an asymmetric bimanual interaction technique that makes browser-based multi-touch gestures more expressive. In the proposed TouchModifier technique, a semi-transparent panel with modifier controls is docked to the side of the screen. The non-dominant hand operates the side panel, while the dominant hand interacts with the application content as usual. The controls on the side panel operate as fluid mode selectors that enrich and override the semantics of the dominant hand gestures. This opens novel interaction possibilities in browser applications, while remaining interoperable with existing web pages. In this paper, we describe the proposed concept and present its prototype implementation with a use case.

UniQue: An Approach for Unified and Efficient Querying of Heterogeneous Web Data Sources

Governments, organizations, and people are publishing open data on the Web more than ever before. To consume the data, however, requires substantial effort from web mashup developers, as they have to familiarize themselves with a diversity of data formats and query techniques specific to each data source. While several solutions have been proposed to improve web querying, none of them covers aforementioned aspects in a developer friendly and efficient manner. Therefore, we devised a unified querying (UniQue) approach and a proxy-based implementation that provides a uniform and declarative interface for querying heterogeneous data sources across the Web. Besides hiding the differences between the underlying data formats and query techniques, UniQue heavily embraces open W3C standards to minimize the learning effort required by developers. Pursuing this further, we propose Unified Query Language (UQL) that combines the expressiveness of CSS Selectors and XPath into a single and flexible selector language. We show that the adoption of UniQue and UQL can effectively streamline web querying, leverage developers’ existing knowledge, and reduce generated network traffic compared to the current state-of-the-art approach.