David Wessel

Timbre Space as a Musical Control Structure

Research on musical timbre typically seeks representations of the perceptual structure inherent in a set of sounds that have implications for expressive control over the sounds in composition and performance. With digital analysis-based sound synthesis and with experiments on tone quality perception, we can obtain representations of sounds that suggest ways to provide low-dimensional control over their perceptually important properties. In this paper, we will describe a system for taking subjective measures of perceptual contrast between sound objects and using this data as input to some computer programs. The computer programs use multidimensional scaling algorithms to generate geometric representations from the input data. In the timbral spaces that result from the scaling programs, the various tones can be represented as points and a good statistical relationship can be sought between the distances in the space and the contrast judgments between the corresponding tones. The spatial representation is given a psychoacoustical interpretation by relating its dimensions to the acoustical properties of the tones. Controls are then applied directly to these properties in synthesis. The control schemes to be described are for additive synthesis and allow for the manipulation of the evolving spectral energy distribution and various temporal features of the tones. Tests of the control schemes have been carried out in musical contexts.* Particular emphasis will be given here to the construction of melodic lines in which the timbre is manipulated on a note-to-note basis. Implications for the design of human control interfaces and of software for real-time digital sound synthesizers will be discussed.

Problems and prospects for intimate musical control of computers

In this paper we describe our efforts towards the development of live performance computer-based musical instrumentation. Our design criteria include initial ease of use coupled with a long term potential for virtuosity, minimal and low variance latency, and clear and simple strategies for programming the relationship between gesture and musical result. We present custom controllers and unique adaptations of standard gestural interfaces, a programmable connectivity processor, a communications protocol called Open Sound Control (OSC), and a variety of metaphors for musical control. We further describe applications of our technology to a variety of real musical performances and directions for future research.

Exploration of timbre by analysis and synthesis

Publisher Summary This chapter explains the exploration of timbre by analysis and synthesis. Timber is referred as the quality of sound. It is the perceptual attribute that helps in distinguishing among orchestral instruments that are playing the same pitch and are equally loud. But, unlike loudness and pitch, timbre is not a well-defined perceptual attribute. The chapter explains that timber is that attribute of auditory sensation in terms of which a listener can judge that two sounds similarly presented and having the same loudness and pitch are dissimilar. It seems that a form of timbral constancy is implied by the common observation that a sound source can be reliably identified over a wide variety of circumstances. For periodic tones, timbre depends upon spectrum. It has long been thought that musical tones are periodic, at least for most of their duration. Musical tones are often thought of as comprising three sections: attack, steady state, and decay. Helmholtz and his followers considered timbre to be determined by the spectrum of the steady state. However, this conception suffers from serious difficulties. Thus, the chapter briefly discusses the shortcomings of the classical conception.

The Swarm at the Edge of the Cloud

Mobile devices such as laptops, netbooks, tablets, smart phones and game consoles have become our de facto interface to the vast amount of information delivery and processing capabilities of the cloud. The move to mobility has been enabled by the dual forces of ubiquitous wireless connectivity combined with the increasing energy efficiency offered by Moores law.

A Generalized Mechanism for Perception of Pitch Patterns

Surviving in a complex and changeable environment relies on the ability to extract probable recurring patterns. Here we report a neurophysiological mechanism for rapid probabilistic learning of a new system of music. Participants listened to different combinations of tones from a previously unheard system of pitches based on the Bohlen-Pierce scale, with chord progressions that form 3:1 ratios in frequency, notably different from 2:1 frequency ratios in existing musical systems. Event-related brain potentials elicited by improbable sounds in the new music system showed emergence over a 1 h period of physiological signatures known to index sound expectation in standard Western music. These indices of expectation learning were eliminated when sound patterns were played equiprobably, and covaried with individual behavioral differences in learning. These results demonstrate that humans use a generalized probability-based perceptual learning mechanism to process novel sound patterns in music.

Humans Use Summary Statistics to Perceive Auditory Sequences

In vision, humans use summary statistics (e.g., the average facial expression of a crowd) to efficiently perceive the gist of groups of features. Here, we present direct evidence that ensemble coding is also important for auditory processing. We found that listeners could accurately estimate the mean frequency of a set of logarithmically spaced pure tones presented in a temporal sequence (Experiment 1). Their performance was severely reduced when only a subset of tones from a given sequence was presented (Experiment 2), which demonstrates that ensemble coding is based on a substantial number of the tones in a sequence. This precise ensemble coding occurred despite very limited representation of individual tones from the sequence: Listeners were poor at identifying specific individual member tones (Experiment 3) and at determining their positions in the sequence (Experiment 4). Together, these results indicate that summary statistical coding is not limited to visual processing and is an important auditory mechanism for extracting ensemble frequency information from sequences of sounds.

A force sensitive multi-touch array supporting multiple 2-D musical control structures

We describe the design, implementation, and evaluation with musical applications of force sensitive multi-touch arrays of touchpads. Each of the touchpads supports a three dimensional representation of musical material: two spatial dimensions plus a force measurement we typically use to control dynamics. We have developed two pad systems, one with 24 pads and a second with 2 arrays of 16 pads each. We emphasize the treatment of gestures as sub-sampled audio signals. This tight coupling of gesture with audio provides for a high degree of control intimacy. Our experiments with the pad arrays demonstrate that we can efficiently deal with large numbers of audio encoded gesture channels - 72 for the 24 pad array and 96 for the two 16 pad arrays.

Harmonic expectation and affect in Western music : Effects of attention and training

We investigated the effects of selective attention and musical training on the processing of harmonic expectations. In Experiment 1, participants with and without musical training were required to respond to the contour of melodies as they were presented with chord progressions that were highly expected, slightly unexpected, or extremely unexpected. Reaction time and accuracy results showed that when attention was focused on the melody, musically trained participants were still sensitive to different harmonic expectations, whereas participants with no musical training were undifferentiated across expectation conditions. In Experiment 2, participants were required to listen holistically to the entire chord progression and to rate their preference for each chord progression. Results from preference ratings showed that all the participants, with or without musical training, were sensitive to manipulations of harmonic expectations. Experiments 3 and 4 showed that changing the speed of presentation of chord progressions did not affect the pattern of results. The four experiments together highlight the importance of attentional focus in musical training, especially as it relates to the processing of harmonic expectations.

The ZIPI Music Parameter Description Language

ZIPIs Music Parameter Description Language (MPDL) is a new language for describing music. It delivers musical parameters, such as articulation and brightness, to notes or groups of notes. It includes parameters that are well understood and universally implemented, such as loudness and pitch, and supports parameters that should be more common in the future, such as brightness and noise amount. A large number of parameters remain unspecified, thus ensuring expandability and flexibility. The MPDL is just one of ZIPIs application layers; others will include MIDI, data dumps, and digital audio. This article does not address any of the lowlevel networking issues associated with ZIPI. We will assume that ZIPIs lower levels deliver arbi-

Neural networks for simultaneous classification and parameter estimation in musical instrument control

In this report we present our tools for prototyping adaptive user interfaces in the context of real-time musical instrument control. Characteristic of most human communication is the simultaneous use of classified events and estimated parameters. We have integrated a neural network object into the MAX language to explore adaptive user interfaces that considers these facets of human communication. By placing the neural processing in the context of a flexible real-time musical programming environment, we can rapidly prototype experiments on applications of adaptive interfaces and learning systems to musical problems. We have trained networks to recognize gestures from a Mathews radio baton, Nintendo Power GloveTM, and MIDI keyboard gestural input devices. In one experiment, a network successfully extracted classification and attribute data from gestural contours transduced by a continuous space controller, suggesting their application in the interpretation of conducting gestures and musical instrument control. We discuss network architectures, low-level features extracted for the networks to operate on, training methods, and musical applications of adaptive techniques.