Stefano Baldan

Mobile rhythmic interaction in a sonic tennis game

This paper presents a game for mobile devices which simulates a tennis match between two players. It is an audio-based game, so the majority of information and feedback to the user is given through sound instead of being displayed on a screen. As users are not requested to keep their eyes on the display, the device can be used as a motion-based controller, exploiting its internal motion sensors to their full potential. The game aims to be useful for both entertainment and educational purposes, and enjoyable both by visually-impaired (the main target audience for audio-based games nowadays) and sighted users.

Algorithms for an automatic transcription of live music performances into symbolic format

This paper addresses the problem of the real-time automatic transcription of a live music performance into a symbolic format. The source data are given by any music instrument or other device able to communicate through a performance protocol. During a performance, music events are parsed and their parameters are evaluated thanks to rhythm and pitch detection algorithms. The final step is the creation of a well-formed XML document, validated against the new international standard known as IEEE 1599. This work will shortly describe both the software environment and the XML format, but the main analysis will involve the real-time recognition of music events. Finally, a case study will be presented: PureMX, a set of Pure Data externals, able to perform the automatic transcription of MIDI events.

Auditory depth map representations with a sensory substitution scheme based on synthetic fluid sounds

A novel sensory substitution algorithm based on the sonification of depth maps into physically based fluid flow sounds is described. Spatial properties are extracted from depth maps and mapped into parameters of an empirical phenomenological model of bubble statistics, which manages the generation of the corresponding synthetic fluid flow sound. Following minimal training, the proposed approach was tested in a preliminary experiment with 20 normally sighted participants and compared against the well-known vOICe sensory substitution algorithm. Although the accuracy in recognizing visual sequences based on the corresponding sonification is comparable between the two systems, an overwhelming support for the fluid sounds compared to the vOICe output in terms of pleasantness was recorded. Collected data further suggests that ample margins of performance improvement are achievable following thorough training procedures.

Physically informed car engine sound synthesis for virtual and augmented environments

The richness of crossmodal feedback in car driving makes it an engaging, complex, yet “natural” activity. Audition plays an important role, as the engine sound, perceived in the cabin, conveys relevant cues about the vehicle motion. In this paper, we introduce a procedural and physically informed model for synthetic combustion engine sound, as an effective, flexible and computationally efficient alternative to sample-based and analysis/resynthesis approaches. The sound model, currently being developed as Max/MSP external, has been integrated in GeneCars, a driving simulator environment for industrial sound design, and SkAT Studio, a demonstration framework for the rapid creation of audio processing workflows.

Embodied sound design

Abstract Embodied sound design is a process of sound creation that involves the designer’s vocal apparatus and gestures. The possibilities of vocal sketching were investigated by means of an art installation. An artist–designer interpreted several vocal self-portraits and rendered the corresponding synthetic sketches by using physics-based and concatenative sound synthesis. Both synthesis techniques afforded a broad range of artificial sound objects, from concrete to abstract, all derived from natural vocalisations. The vocal-to-synthetic transformation process was then automated in SEeD, a tool allowing to set and play interactively with physics- or corpus-based sound models. The voice-driven process and tool, developed and evaluated through design exercises, show how an embodied sound sketching system can work in supporting the externalisation of sonic concepts.