Nicolas Bouillot

A high-fidelity orchestra simulator for individual musicians' practice

We developed the Open Orchestra system to provide individual musicians with a high-fidelity experience of ensemble rehearsal or performance, combined with the convenience and flexibility of solo study. This builds on the theme of an immersive orchestral simulator that also supports the pedagogical objective of instructor feedback on individual recordings, as needed to improve ones performance. Unlike previous systems intended for musical rehearsal, Open Orchestra attempts to offer both an auditory and a visual representation of the rest of the orchestra, spatially rendered from the perspective of the practicing musician. We review the objectives and architecture of our system, describe the functions of our digital music stand, discuss the challenges of generating the media content needed for this system, and describe provisions for offering feedback during rehearsal.

nJam user experiments: enabling remote musical interaction from milliseconds to seconds

Remote real-time musical interaction is a domain where end-to-end latency is a well known problem. Today, the main explored approach aims to keep it below the musicians perception threshold. In this paper, we explore another approach, where end-to-end delays rise to several seconds, but computed in a controlled (and synchronized) way depending on the structure of the musical pieces. Thanks to our fully distributed prototype called nJam, we perform user experiments to show how this new kind of interactivity breaks the actual end-to-end latency bounds.

Monitoring the Distributed Virtual Orchestra with a CORBA Based Object Oriented Real-Time Data Distribution Service

This paper presents a CORBA based object oriented real-time data distribution service and its use to monitor our distributed virtual orchestra over a campus LAN. This service offers two types of data management policies: periodic exchanges and condition based exchanges. The distributed virtual orchestra uses the jMax engine to generate automatically sounds (virtual player), play and transform the music from real musicians (real player). Each player is connected to a PC and sends its high quality audio stream through the network. Real players hear each other, including the sounds from virtual players, and are self-synchronized. The jMax engine is modeled by a virtual sound automaton remotely monitored through our real-time data distribution service. The monitoring service is effective and has been used for jMax management during musical performance. Future works deal with a CORBA-CCM based monitoring service.

In situ editing (EiS) for fulldomes

The creation workflow for fulldome consists today of many back and forth between the immersive space for visualization and desktop computers for editing. The EiS platform is an interactive creation tool that enables editing while being inside the immersive environment, allowing for coarse-grained 3D content editing. In this paper we present EiS, a platform geared toward editing inside fulldomes, but also possibly other immersive spaces. The core of EiS is a set of Blender (the popular 3D modeling tool) plugins adding support for VR controllers, fulldome rendering and interfacing with an external audio spatialization engine.