Jason Freeman

Extreme sight-reading, mediated expression, and audience participation: Real-time music notation in live performance

Rossini would often wait until the last possible moment to compose overtures. He wrote the overture to Otello the evening before the opera’s 1816 premiere. For La gazza ladra the following year, Rossini waited until the day of the premiere to score the overture, working, as he later wrote, “up under the roof of La Scala in Milan.” He also noted: “Nothing is better for inspiration than necessity, the presence of a copyist waiting for your work, sheet by sheet” (Hughes 1956, p. 247). Real-time music notation systems take Rossini’s strategy to new extremes, waiting to create the score until during the performance. Unlike most live computer-music performance environments, these software algorithms do not produce digital audio or control data. Instead, they produce a dynamic musical score that may contain conventional Western notation or a range of graphical representations, which is interpreted by human musicians to create sound.

Engaging underrepresented groups in high school introductory computing through computational remixing with EarSketch

In this paper, we describe a pilot study of EarSketch, a computational remixing approach to introductory computer science, in a formal academic computing course at the high school level. The EarSketch project provides an integrated curriculum, Python API, digital audio workstation (DAW), audio loop library, and social sharing site. The goal for EarSketch is to broaden participation in computing, particularly by traditionally underrepresented groups, through a thickly authentic learning environment that has personal and industry relevance in both computational and artistic domains. The pilot results show statistically significant gains in computing attitudes across multiple constructs, with particularly strong results for female and minority participants.

Web-based collaboration, live musical performance and open-form scores

ABSTRACT As Internet-based social listening, composition and improvisation applications become increasingly important cultural mechanisms through which to experience, create and share digital music, the author seeks to create comparable systems to engage audiences within the context of live musical performances. Real-time collaboration with audiences during performance can lead to exciting and rewarding musical experiences. But the technical, logistical and design constraints at live venues, including simultaneous participation and single-stream output, make such live participation systems challenging to create and deploy effectively. This article instead focuses on linking web-based collaborative experiences with more traditional live musical performances. The article explores the motivations behind such hybrid performance works and the history of them, and it then discusses and evaluates two recent works by the author, Graph Theory (2006) and Piano Etudes (2009), both of which use open-form musical scores as a structural framework for participation.

Soundscape composition and field recording as a platform for collaborative creativity

In this paper, the authors describe and discuss UrbanRemix, a platform consisting of mobile-device applications and web-based tools to facilitate collaborative field recording, sound exploration, and soundscape creation. Reflecting on its use at workshops, festivals and community events, they evaluate the project in terms of its ability to enable participants to engage with their aural environments and to uncover their own creativity in the process.

Tackling engagement in computing with computational music remixing

In this paper, we describe EarSketch, an integrated curriculum, software toolset, and social media website, grounded in constructionist principles, that targets introductory high school computing education. We hypothesize that the use of collaborative computational music composition and remixing may avoid some of the engagement and culture-specific issues that other approaches, both in music and other media, have had. We discuss the design of EarSketch, its use in a pilot summer camp, and the evaluation results from that pilot.

Creative collaboration between audiences and musicians in Flock

Flock is a full-evening work for saxophone quartet, dancers, audience participation, video and electronic sound. A computer vision system tracks the location of participants as they move around an open performance space, and custom software then uses that data to generate music notation, video animation and electronic sound. In this paper, we discuss the projects connection to previous work in large-audience creativity and multi-player gaming, and we outline the conception, design and implementation of its computer vision system, real-time notation architecture and collaborative structure. We evaluate the projects success as measured by audience surveys from performances at the Adrienne Arsht Center for the Performing Arts in Miami and the 01SJ Festival in San Jose, and we outline revisions to the work based on that feedback.

Tools for Real-Time Music Notation

This article assesses the current state of hardware and software architectures and tools for real-time notation rendering and display in concert performance. The article is intended as a practical resource for composers wishing to explore real-time notation in their works, as a survey of available tools and their capabilities and limitations, and as a consideration of the relationships between technical architectures and artistic designs in real-time notation systems.

Glimmer: Creating New Connections

Glimmer, a composition for chamber orchestra and audience, uses novelty light sticks, video cameras, computer software, multi-colored stand lights, and projected video animation to create a continuous feedback loop in which audience activities, software algorithms, and orchestral performance together create the music. This paper establishes the aesthetic background and motivations behind Glimmer and describes the conceptual framework and technical realization of the piece in detail. Performances of the work by the American Composers Orchestra at Carnegie Hall in New York and by musicians at the Hamabada Art Center in Jerusalem are evaluated with respect to the audience, the musicians, and the resulting music that was created.

EarSketch: An integrated approach to teaching introductory computer music

EarSketch is an all-in-one approach to supporting a holistic introductory course to computer music as an artistic pursuit and a research practice. Targeted to the high school and undergraduate levels, EarSketch enables students to acquire a strong foundation in electroacoustic composition, computer music research and computer science. It integrates a Python programming environment with a commercial digital audio workstation program (Cockos’ Reaper) to provide a unified environment within which students can use programmatic techniques in tandem with more traditional music production strategies to compose music. In this paper we discuss the context and goals of EarSketch, its design and implementation, and its use in a pilot summer camp for high school students.

EarSketch: A STEAM-Based Approach for Underrepresented Populations in High School Computer Science Education

This article presents EarSketch, a learning environment that combines computer programming with sample-based music production to create a computational remixing environment for learning introductory computing concepts. EarSketch has been employed in both formal and informal settings, yielding significant positive results in student content knowledge and attitudes toward computing as a discipline, especially in ethnic and gender populations that are currently underrepresented in computing fields. This article describes the rationale and components of EarSketch, the evaluation design, and lessons learned to apply to future environment design and development.