Antti Jylhä

Real-time recognition of percussive sounds by a model-based method

Interactive musical systems require real-time, low-latency, accurate, and reliable event detection and classification algorithms. In this paper, we introduce a model-based algorithm for detection of percussive events and test the algorithm on the detection and classification of different percussive sounds. We focus on tuning the algorithm for a good compromise between temporal precision, classification accuracy and low latency. The model is trained offline on different percussive sounds using the expectation maximization approach for learning spectral templates for each sound and is able to run online to detect and classify sounds from audio stream input by a Hidden Markov Model. Our results indicate that the approach is promising and applicable in design and development of interactive musical systems.

A hand clap interface for sonic interaction with the computer

We present a hand clapping interface for sonic interaction with the computer. The current implementation has been built on the Pure Data (PD) software. The interface makes use of the cyclic nature of hand clapping and recognition of the clap type, and enables interactive control over different applications. Three prototype applications for the interface are presented: a virtual crowd of clappers, controlling the tempo of music, and a simple sampler. Preliminary tests indicate that rather than having total control via the interface, the user negotiates with the computer to control the tempo.

Design and evaluation of human-computer rhythmic interaction in a tutoring system

of Human–Computer Rhythmic Interaction in a Tutoring System Antti Jylha,∗ Inger Ekman,† Cumhur Erkut,∗ and Koray Tahiroglu∗∗ ∗ Department of Signal Processing and Acoustics Aalto University School of Electrical Engineering P.O. Box 13000, FI-00076 Aalto, Finland {antti.jylha, cumhur.erkut}@aalto.fi †Center for Knowledge and Innovation Research (CKIR) Aalto University School of Economics P.O. Box 21255, FI-00076 Aalto, Finland inger.ekman@aalto.fi ∗∗Media Lab, Department of Media Aalto University School of Art and Design P.O. Box 31000, FI-00076 Aalto, Finland koray.tahiroglu@aalto.fi

Rhythmic walking interactions with auditory feedback: an exploratory study

Walking is a natural rhythmic activity that has become of interest as a means of interacting with software systems such as computer games. Therefore, designing multimodal walking interactions calls for further examination. This exploratory study presents a system capable of different kinds of interactions based on varying the temporal characteristics of the output, using the sound of human walking as the input. The system either provides a direct synthesis of a walking sound based on the detected amplitude envelope of the users footstep sounds, or provides a continuous synthetic walking sound as a stimulus for the walking human, either with a fixed tempo or a tempo adapting to the human gait. In a pilot experiment, the different interaction modes are studied with respect to their effect on the walking tempo and the experience of the subjects. The results tentatively outline different user profiles in interacting with such a system.

Auditory feedback in an interactive rhythmic tutoring system

We present the recent developments in the design of audio-visual feedback in iPalmas, the interactive Flamenco rhythm tutor. Based on evaluation of the original implementation, we have re-designed the interface to better support the user in learning and performing rhythmic patterns. The system measures the performance parameters of the user and provides auditory feedback on the performance with different sounds corresponding to different performance attributes. The design of these sounds is informed by several attributes derived from the evaluation. We propose informative, non-intrusive. and archetypal sounds to be used in the system.

Rhythmic blueprints: a tutorial for design and evaluation of rhythmic interaction

Rhythm, the temporal organization of sound, triggers our sensation, perception, and cognition. It is a conveyor of meaning and emotion. Neuroscientists suggest that rhythmicity is a key element that makes us human, tapping essentially in all levels of organization in our brains [10].

Simulation of rhythmic learning: a case study

Simulation of human interaction with computational systems can inform their design and provide means for designing new, intelligent systems capturing some of the essence of human behavior. We describe a system simulating a situation, where a virtual tutor is teaching rhythms to a human learner. In this simulation, we virtualize the human behavior related to the learning of new rhythms. We inform the design of the system based on an experiment, in which a virtual tutor taught Flamenco hand clapping patterns to human subjects. Based on the findings on interaction with the system and learning of the patterns, we are simulating this learning situation with a virtual learning clapper. We also discuss the future work to be undertaken for more realistic, agent-based simulation of rhythmic interaction.

Interacting with virtual musical instruments at the junction nodes

Sound synthesis by block‐based physical modeling of musical instruments separates the tasks of component modeling and managing their interactions. The components are the exciters or the resonators, and their interactions are managed by explicit interaction blocks, which are obtained from the physical continuity and energy conservation rules. Well‐known examples of the interactors include the wave‐ digital adaptors and the digital waveguide scattering junctions. When the virtual instruments need to be interfaced to the outside environment with sensors and actuators for bidirectional interaction, it is advantageous to reformulate the interactors to accept and provide signal inputs and outputs, respectively. In this contribution, we refer to these elements as nodes, introduce different types of nodes, and discuss their interconnection.