Adrian Freed

A force sensitive multi-touch array supporting multiple 2-D musical control structures

We describe the design, implementation, and evaluation with musical applications of force sensitive multi-touch arrays of touchpads. Each of the touchpads supports a three dimensional representation of musical material: two spatial dimensions plus a force measurement we typically use to control dynamics. We have developed two pad systems, one with 24 pads and a second with 2 arrays of 16 pads each. We emphasize the treatment of gestures as sub-sampled audio signals. This tight coupling of gesture with audio provides for a high degree of control intimacy. Our experiments with the pad arrays demonstrate that we can efficiently deal with large numbers of audio encoded gesture channels - 72 for the 24 pad array and 96 for the two 16 pad arrays.

Neural networks for simultaneous classification and parameter estimation in musical instrument control

In this report we present our tools for prototyping adaptive user interfaces in the context of real-time musical instrument control. Characteristic of most human communication is the simultaneous use of classified events and estimated parameters. We have integrated a neural network object into the MAX language to explore adaptive user interfaces that considers these facets of human communication. By placing the neural processing in the context of a flexible real-time musical programming environment, we can rapidly prototype experiments on applications of adaptive interfaces and learning systems to musical problems. We have trained networks to recognize gestures from a Mathews radio baton, Nintendo Power GloveTM, and MIDI keyboard gestural input devices. In one experiment, a network successfully extracted classification and attribute data from gestural contours transduced by a continuous space controller, suggesting their application in the interpretation of conducting gestures and musical instrument control. We discuss network architectures, low-level features extracted for the networks to operate on, training methods, and musical applications of adaptive techniques.

Sound design as human matter interaction

Recently, terms like material computation or natural computing in foundations of computer science and engineering, and new materiality in cultural studies signal a broader turn to conceptions of the world that are not based on solely human categories. While respecting the values of human-centered design, how can we begin to think about the design of responsive environments and computational media while paying as much attention to material qualities like elasticity, density, wear, and tension as to social and cognitive phenomena? This question understands computation as a potential property of matter in a non-reductive way that plausibly spans formal divides between symbolic-semiotic, social, and physical processes. Full investigation greatly exceeds one brief paper. But we open this question in the concrete practices of computational sound and sound design.

A fixed-point recursive digital oscillator for additive synthesis of audio

This paper summarizes our work adapting a recursive digital resonator for use on sixteen-bit fixed-point hardware. Our modified oscillator is a two-pole filter that maintains frequency precision at a cost of two additional operations per filter sample. The new filters error properties are expressly matched to use in the range of frequencies relevant to additive synthesis of digital audio and sinusoidal modelling of speech in order to minimize the additional computational overhead. We present the algorithm, an error analysis, a performance analysis, and measurements of an implementation on a fixed-point vector microprocessor system.

Colocated surface sound interaction

We present three related schemes for colocating sensing and sound actuation on flat surfaces. One uses conductive paper to create a musical instrument, another uses magnets mounted in gloves and printed conductors to form planar loudspeaker arrays. Finally we show how conductive and resistive fabrics can be integrated with loudspeaker drivers.

Flexible surfaces for interactive audio

We present here flat flexible audio speaker surface arrays, which are transparent, formed to various environments, and allow for user interaction. These speaker arrays provide an alternative to traditional models of sound reproduction, which often involve discreet point source systems and bulky hardware passively received by the user. The surface array system opens up new possibilities for acoustic spaces, creativity, and sound interactivity.

Volumetric modeling of acoustic fields in CNMAT's sound spatialization theatre

A new tool for real time visualization of acoustic sound fields has been developed for a new sound spatialization theatre. The theatre is described and several applications of the acoustic and volumetric modeling software are presented. The visualization system described is a valuable tool for spatial sound researchers, sound engineers and composers using CNMATs sound spatialization theatre. Further work is in progress on the adaptation of better acoustic simulation methods (M. Monks et al., 1996) for more accurate display of the quality of the reverberant field. The room database will be automatically extracted from a model built with 3D modeling software. Volume visualization strategies are being explored to display sounds in spectral and impulse response form.

o.OM: structured-functional communication between computer music systems using OSC and Odot

O.—odot—is a portable media programming framework based on the OSC data encoding. It embeds a small expression language which allows writing and executing programs in OSC structures. The integration of programming and declarative functional descriptions within data transfer protocols enables structured and expressive communication in media systems: program snippets can be distributed in OSC messages, which evaluate to further OSC messages in the different communicating software. We present experiments using this framework in the OpenMusic computer-aided composition environment, and illustrate via case studies some advantages of such integrated system.

The Fingerphone: a Case Study of Sustainable Instrument Redesign

The Fingerphone, a reworking of the Stylophone in conductive paper, is presented as an example of new design approaches for sustainability and playability of electronic musical instruments.

2003: OpenSound Control: State of the Art 2003

OpenSound Control (“OSC”) is a protocol for communication among computers, sound synthesizers, and other multimedia devices that is optimized for modern networking technology. OSC has achieved wide use in the field of computer-based new interfaces for musical expression for wide-area and local-area networked distributed music systems, inter-process communication, and even within a single application.