Sylvain Le Beux

REALTIME AND ACCURATE MUSICAL CONTROL OF EXPRESSION IN SINGING SYNTHESIS

In this paper, we describe a full computer-based musical instrument allowing realtime synthesis of expressive singing voice. The expression results from the continuous action of an interpreter through a gestural control interface. In this context, expressive features of voice are discussed. New real-time implementations of a spectral model of glottal flow (CALM) are described. These interactive modules are then used to identify and quantify voice quality dimensions. Experiments are conducted in order to develop a first framework for voice quality control. The representation of vocal tract and the control of several vocal tract movements are explained and a solution is proposed and integrated. Finally, some typical controllers are connected to the system and expressivity is evaluated.

Drawing melodies: Evaluation of chironomic singing synthesis

Cantor Digitalis, a real-time formant synthesizer controlled by a graphic tablet and a stylus, is used for assessment of melodic precision and accuracy in singing synthesis. Melodic accuracy and precision are measured in three experiments for groups of 20 and 28 subjects. The task of the subjects is to sing musical intervals and short melodies, at various tempi, using chironomy (hand-controlled singing), mute chironomy (without audio feedback), and their own voices. The results show the high accuracy and precision obtained by all the subjects for chironomic control of singing synthesis. Some subjects performed significantly better in chironomic singing compared to natural singing, although other subjects showed comparable proficiency. For the chironomic condition, mean note accuracy is less than 12 cents and mean interval accuracy is less than 25 cents for all the subjects. Comparing chironomy and mute chironomy shows that the skills used for writing and drawing are used for chironomic singing, but that the audio feedback helps in interval accuracy. Analysis of blind chironomy (without visual reference) indicates that a visual feedback helps greatly in both note and interval accuracy and precision. This study demonstrates the capabilities of chironomy as a precise and accurate mean for controlling singing synthesis.

The Glass Organ: Musical Instrument Augmentation for Enhanced Transparency

The Organ and Augmented Reality (ORA) project has been presented to public audiences at two immersive concerts, with both visual and audio augmentations of an historic church organ. On the visual side, the organ pipes displayed a spectral analysis of the music using visuals inspired by LED-bar VU-meters. On the audio side, the audience was immersed in a periphonic sound field, acoustically placing listeners inside the instrument. The architecture of the graphical side of the installation is made of acoustic analysis and calibration, mapping from sound levels to animation, visual calibration, real-time multi-layer graphical composition and animation. It opens new perspectives to musical instrument augmentation where the purpose is to make the instrument more legible while offering the audience enhanced artistic content.

Organ Augmented Reality: Audio-Graphical Augmentation of a Classical Instrument

This paper discusses the Organ Augmented Reality ORA project, which considers an audio and visual augmentation of an historical church organ to enhance the understanding and perception of the instrument through intuitive and familiar mappings and outputs. ORA has been presented to public audiences at two immersive concerts. The visual part of the installation was based on a spectral analysis of the music. The visuals were projections of LED-bar VU-meters on the organ pipes. The audio part was an immersive periphonic sound field, created from the live capture of the organ sounds, so that the listeners had the impression of being inside the augmented instrument. The graphical architecture of the installation is based on acoustic analysis, mapping from sound levels to synchronous graphics through visual calibration, real-time multi-layer graphical composition and animation. The ORA project is a new approach to musical instrument augmentation that combines enhanced instrument legibility and enhanced artistic content.