Doug Van Nort

Multidimensional Scratching, Sound Shaping and Triple Point

ABSTRACT The author discusses performance utilizing his greis software system, which is built around the principle of a scrubbing interaction with roots in the recording industry and the paradigm of scrubbing tape across a magnetic head.

Electro/Acoustic Improvisation and Deeply Listening Machines

In this paper we discuss our approach to designing improvising music systems whose intelligence is centred around careful listening, particularly to qualities related to timbre and texture. Our interest lies in systems that can make contextual decisions based on the overall character of the sound field, as well as the specific shape and contour created by each player. We describe the history and paradigm of ‘expanded instrument’ systems, which has led to one instrumental system (GREIS) focused on manual sculpting of sound with machine assistance, and one improvising system (FILTER) which introduces the ability to listen, recognize and transform a performer’s sound in a contextually relevant fashion. We describe the different modules of these improvising performance systems, as well as specific musical performances as examples of their use. We also describe our free improvisation trio, in order to describe the musical context that situates and informs our research.

A vibrotactile music system based on sensory substitution.

The idea of the project reported here was to design a system that builds on touch to enable people with severe hearing impairments to “listen” to music through a process called sensory substitution. The goal was to transform the auditory parameter space into one that is adequate for haptic perception. The approach reported here builds on (i) the design of a haptic display, a tabletop device with 8–24 actuators that can be driven individually, (ii) machine learning algorithms, and (iii) a psychophysical study to determine which music cues can be perceived through touch. The latter was necessary because vibrotactile perception is not yet well understood in the context of music perception. The double‐blind study analyzes how vibrotactile stimuli contribute to the perception, cognition, and distinction of sounds in human participants who have been trained versus those who have not. In order to ensure that normal‐hearing participants could not hear sounds radiated from the haptic display, sound isolating headp...

A Collaborative Approach to Teaching Sound Sculpting, Embodied Listening and the Materiality of Sound

This paper presents recent work in engaging both studentsand working professionals from a variety of disciplines andbackgrounds with the practice of collective and site-specificelectroacoustic music creation. The emphasis is placed onembodied, deep listening in tandem with a manual approachto sonic art creation that bridges an understanding of theinterplay between digital sound manipulation, largercomposed structures and the physical presentation of a workin a given space. Through a practice-oriented approach,participants gain insights into areas such as the abstractworld of digital sound recording and representation, theextreme influence on this content enacted by a given sounddelivery system and a given space, and the subjectiveexperience of listening to sounds from a variety oforientations and postures, and with varying levels ofunderstanding of the original source recordings. Finally,through a group approach to composing larger structures,participants begin to understand the often mysterious andunsaid processes involved in the normally solitary act ofcomposing electroacoustic music.

Creativity and conducting: handle in the CAIRA project

After providing some context via (i) earlier work on literary creativity carried out by Bringsjord et al., and (ii) an account of creativity espoused by Cope, which stands in rather direct opposition to Bringsjords account, we summarize our nascent attempt to engineer an artificial conductor: Handle. Handle is a microcosmic version of part of a larger, much more ambitious system: CAIRA. Both are under development courtesy of a three-year CreativeIT grant from the National Science Foundation (PI Braasch, Co-PIs Oliveros & Bringsjord).

Human:machine:human: Gesture, Sound and Embodiment

Purpose – The purpose of this paper is to present an embodied view on human/machine co‐creation in general, and musical improvisation in particular.Design/methodology/approach – Questions and propositions are formed by examining personal work in intelligent, interactive system design.Findings – Proper consideration of gestural representation and intentionality leads to enhanced potential for collective expression in human/machine interaction.Originality/value – This approach extends ideas of conversation theory to improvisational contexts based on spontaneous, collective expression.

Extending the acoustic ensemble through spectral and temporal transformations in real‐time.

The paradigm of live performance mixing acoustics and electronics has predominantly focused on simple background “tape music,” human players performing highly structured sample‐based music (e.g., using the ABLETON LIVE software), or reactive systems that respond to player qualities such as timing, pitch, and so on. In this talk I will present my approach to improvised “laptop performance” that focuses on the transformation of acoustic players in real‐time. Rather than simply altering the acoustic content in the manner of an effect processor, the goal is to capture notes and phrases in short‐term memory and to re‐articulate the material so that it presents a new gestural inflection and timbral content that can be completely novel or suggestive of other players’ sound. The system presented utilizes a hybrid system combining spectral analysis and feature extraction with block‐based temporal processing and a feedback delay network. The interaction paradigm of “scrubbing” the intermediate time/frequency repres...

Instrumental analysis of extended saxophone techniques for live electronics.

The development of automated music transcription systems focuses predominantly on polyphonic musical instruments. At the same time, the analysis of a monophonic instrument is usually much simpler wherein pitch, loudness, and duration of individual notes may be tracked robustly. When using extended techniques, however, many more parameters than the aforementioned three can be meaningful for the performed music. This paper explores the challenges that extended techniques pose for music recognition systems using the example of the saxophone. The goal is to correctly identify extended techniques over the whole range of the instrument, including subtones, multiphonics, growl, and other voice‐enhanced tones, as well as tones where the reed is supported by the lower teeth. The feature analysis is based on cepstrum, spectral moments, pitch, and roughness, among other features. A hidden‐Markov model is used to recognize the trajectory of the various extended techniques based on the given feature space. Finally, it...

Creating systems for collaborative network‐based digital music performance.

The internet has proven to be an important catalyst in bringing together musicians for remote collaboration and performance. Existing technologies for network audio streaming possess varying degrees of technological transparency with regard to allowable bandwidth, latency, and software interface constraints, among other factors. In another realm of digital audio, the performance of “laptop music” presents a set of challenges with regard to human‐computer and interperformer interaction—particularly in the context of improvisation. This paper discusses the limitations as well as newfound freedoms that can arise in the construction of musical performance systems that merge the paradigms of laptop music and network music. Several such systems are presented from personal work created over the past several years that consider the meaning of digital music collaboration, the experience of sound‐making in remote physical spaces, and the challenge of improvising across time and space with limited visual feedback. T...