Zack Settel

Real-time timbral transformation: FFT-based resynthesis

This paper presents real-time musical applications using the IRCAM Signal Processing Workstation which make use of FFT/IFFT-based resynthesis for timbral transformation in a compositional context. An intuitive and straightforward user interface, intended for use by musicians, has been developed by the authors in the Max programming environment. Techniques for filtering, cross-synthesis, noise reduction, and dynamic spectral shaping are presented along with control structures that allow for both fine timbral modification and control of complex sound transformations using few parameters.

Real-time musical applications using frequency domain signal processing

This paper presents real-time musical applications using the IRCAM signal processing workstation which make use of FFT/IFFT-based resynthesis for timbral transformation in a musical context. An intuitive and straightforward user interface, intended for use by musicians, has been developed by the authors in the Max programming environment. Techniques for high quality time-stretching, filtering, cross-synthesis, dynamic range processing, and spectrum shaping are presented along with dynamic control structures that allow for both fine timbral modification and control of complex sound transformations using few parameters.

Low-dimensional audio-rate control of FFT-based processing

While the use of the fast Fourier transform (FFT) for signal processing in music applications has been widespread, applications in real-time systems for dynamic spectral transformation have been quite limited. The limitations have been largely due to the amount of computation required for the operations. With faster machines, and with suitable implementation for frequency-domain processing, real-time dynamic control of high-quality spectral processing can be accomplished with great efficiency and a simple approach. This paper describe some previous work in dynamic real-time control of frequency-domain-based signal processing. Since the implementation of the FFT/IFFT is central to the approach and methods discussed, the authors provide a description of this implementation, as well as of the development environment used in their work.

Ménagerie imaginaire

Our work leading to this piece began with the search for a better way to interact with electronic sound during performance. Taking the current trend towards performing on stage with laptop computers, we have conceived of a radically different arrangement where performer motion is not only unrestricted, but actually serves a principal role in the performance interface. We abandon conventional electronic audio devices, and use physically modelled virtual simulation as the performance medium. This allows for the use of natural gestures to control sophisticated audio processing, while still allowing traditional musical instruments to be played. In our approach, motions and sounds are captured and modelled within a virtual 3D environment. A combination of microphones and sensors allows multiple performers to input their audio into the scene and steer their signal with great precision, allowing for interaction with digital audio effects located throughout the virtual world. For example, when a performer wishes to send sound through a reverberator, they simply point their instrument toward the reverb unit in the 3D space. The performers may also travel different sonic regions that contain varying types of musical accompaniment and effects. The scene is rendered graphically in realtime, allowing the audience to watch the performance on a large screen situated above the performers on stage. The audience viewpoint provides a subjective rendering, with proper spatialization of all virtual audio sources. Live video from webcams provide textures which are mapped onto 3D avatars, allowing the audience to see close-up views of the performers.