Graziano Bertini

Giving expression to multimedia performance

In this paper we describe the experience of researchers and artists involved in the activities of the “computer ART lab” (cART lab) of the italian National Council of Research (C.N.R.) in Pisa, regarding the “wireless technology” developed for controlling in real-time and giving expression to interactive multimedia performances. Due to the daily increase in computers power and electronics systems able to sense the presence, the shape, the distance and the position of objects, a new field of investigation and implementation has been started in the last few years: computer recognition of human gesture [1][2]. As a result, the human body itself can now be considered as a natural and powerful expressive “interface” to give feeling to performances based on computer generated electro-acoustic music and computer generated visual-art. Modern human computer interfaces are extremely rich, incorporating traditional interface devices such as keyboard and mouse and a wealth of advanced media types: sound, video, animated graphics. The term multi-modal is often associated with such interfaces to emphasize that the combined use of multiple modes of perception is relevant to the users interface [3][4]. The most relevant devices and systems developed at cART lab for gesture recognition to be used in interactive multimedia performances are here reported.

Light baton system: A system for conducting computer music performance

Abstract We describe an interface designed and developed to allow an automatic synthesis system to be controlled by a human conductor. The aim is to endow that electronic music with the expressiveness and feeling which are characteristic of live performances. The system recognises those movements and gestures made by a conductor during a live performance which conform with international standards. The system consists of a special conductors baton and a personal computer with an image acquisition board connected to a camera. The movements of a small lamp placed on the tip of the baton are detected by the camera, digitised as light point position changes and stored. A best‐fitting interpolating method is used to perform the corresponding path, increase time resolution and predict the beat points. The data acquired by the system, i.e., beat point prediction, order number of the beats, amplitude of gesture, end of movement and absence of light points, are applied as input parameters of an algorithmic composi...

About the Role of Mapping in Gesture-Controlled Live Computer Music

Interactive computer music proposes a number of considerations about what the audience experiences in relationship of what-is-going-on-on-stage and the overall musical result. While a traditional music instrument is a compact tool and “to play an instrument” has a precise meaning for everybody, the new electro-acoustic instrument is a system consisting of a number of spread out components: sensors and controllers, computer and electronic sound generators, amplifiers and loudspeakers. How to link information between the various parts of this exploded instrument is deeply correlated to new modalities of composing and performing in relationship with how the audience perceives and accepts these new paradigm. We here report our point of view and considerations about the role of “mapping” derived from our experience both in developing original controllers and in the realization of interactive electro-acoustic performances.

An interactive musical exhibit based on infrared sensors

This paper deals with the description of the design of an exhibit for controlling real-time audio synthesis with a wireless, IR-based interface. Researching new way for playing and real-time controlling electronic music is todays hot topic in the computer music field. The goal of this specific project consists of an enjoyable, robust and reliable exhibit that gives the possibility to constantly operate with young users (especially children) and, more in general, non-expert people. Our effort has been focused to carefully design the hardware/software project, in a way that the final user will interact only with non-critical parts of the system.