Morwaread Farbood

Hyperscore: a graphical sketchpad for novice composers

The Hyperscore graphical computer-assisted composition system for users with limited or no musical training takes freehand drawing as input, letting users literally sketch their pieces. Designing an intelligent, intuitive system that enables novices-particularly children-to compose music is a difficult task. We can view the problem as a spectrum of tasks that range from the development of musical algorithms for automating the compositional process to designing an appropriate interface for humans to interact with the machine. The Hyperscore software tool attempts to address both of these issues.

The neural processing of hierarchical structure in music and speech at different timescales

Music, like speech, is a complex auditory signal that contains structures at multiple timescales, and as such is a potentially powerful entry point into the question of how the brain integrates complex streams of information. Using an experimental design modeled after previous studies that used scrambled versions of a spoken story (Lerner et al., 2011) and a silent movie (Hasson et al., 2008), we investigate whether listeners perceive hierarchical structure in music beyond short (~6 s) time windows and whether there is cortical overlap between music and language processing at multiple timescales. Experienced pianists were presented with an extended musical excerpt scrambled at multiple timescales—by measure, phrase, and section—while measuring brain activity with functional magnetic resonance imaging (fMRI). The reliability of evoked activity, as quantified by inter-subject correlation of the fMRI responses, was measured. We found that response reliability depended systematically on musical structure coherence, revealing a topographically organized hierarchy of processing timescales. Early auditory areas (at the bottom of the hierarchy) responded reliably in all conditions. For brain areas at the top of the hierarchy, the original (unscrambled) excerpt evoked more reliable responses than any of the scrambled excerpts, indicating that these brain areas process long-timescale musical structures, on the order of minutes. The topography of processing timescales was analogous with that reported previously for speech, but the timescale gradients for music and speech overlapped with one another only partially, suggesting that temporally analogous structures—words/measures, sentences/musical phrases, paragraph/sections—are processed separately.

Interpreting expressive performance through listener judgments of musical tension.

This study examines listener judgments of musical tension for a recording of a Schubert song and its harmonic reduction. Continuous tension ratings collected in an experiment and quantitative descriptions of the pieces musical features, include dynamics, pitch height, harmony, onset frequency, and tempo, were analyzed from two different angles. In the first part of the analysis, the different processing timescales for disparate features contributing to tension were explored through the optimization of a predictive tension model. The results revealed the optimal time windows for harmony were considerably longer (~22 s) than for any other feature (~1–4 s). In the second part of the analysis, tension ratings for the individual verses of the song and its harmonic reduction were examined and compared. The results showed that although the average tension ratings between verses were very similar, differences in how and when participants reported tension changes highlighted performance decisions made in the interpretation of the score, ambiguity in tension implications of the music, and the potential importance of contrast between verses and phrases. Analysis of the tension ratings for the harmonic reduction also provided a new perspective for better understanding how complex musical features inform listener tension judgments.

Temporal dynamics and the identification of musical key.

A central process in music cognition involves the identification of key; however, little is known about how listeners accomplish this task in real time. This study derives from work that suggests overlap between the neural and cognitive resources underlying the analyses of both music and speech and is the first, to our knowledge, to explore the timescales at which the brain infers musical key. We investigated the temporal psychophysics of key-finding over a wide range of tempi using melodic sequences with strong structural cues, where statistical information about overall key profile was ambiguous. Listeners were able to provide robust judgments within specific limits, at rates as high as 400 beats per minute (bpm; ∼7 Hz) and as low as 30 bpm (0.5 Hz), but not outside those bounds. These boundaries on reliable performance show that the process of key-finding is restricted to timescales that are closely aligned with beat induction and speech processing.

The contribution of timbre attributes to musical tension

Timbre is an auditory feature that has received relatively little attention in empirical work examining musical tension. In order to address this gap, an experiment was conducted to explore the contribution of several specific timbre attributes-inharmonicity, roughness, spectral centroid, spectral deviation, and spectral flatness-to the perception of tension. Listeners compared pairs of sounds representing low and high degrees of each attribute and indicated which sound was more tense. Although the response profiles showed that the high states corresponded with increased tension for all attributes, further analysis revealed that some attributes were strongly correlated with others. When qualitative factors, attribute correlations, and listener responses were all taken into account, there was fairly strong evidence that higher degrees of roughness, inharmonicity, and spectral flatness elicited higher tension. On the other hand, evidence that higher spectral centroid and spectral deviation corresponded to increases in tension was ambiguous.

Determining Feature Relevance in Subject Responses to Musical Stimuli

This paper presents a method that determines the relevance of a set of signals (musical features) given listener judgments of music in an experimental setting. Rather than using linear correlation methods, we allow for nonlinear relationships and multi-dimensional feature vectors. We first provide a methodology based on polynomial functions and the least-mean-square error measure. We then extend the methodology to arbitrary nonlinear function approximation techniques and introduce the Kullback-Leibler Distance as an alternative relevance metric. The method is demonstrated first with simple artificial data and then applied to analyze complex experimental data collected to examine the perception of musical tension.

The mutability of pitch memory in a tonal context.

An experiment that investigates how a tonal context affects pitch recognition is presented. Melodic sequences that were composed to invoke varying degrees of tonality were rated by musicians (N = 34) for perceived strength of tonality. The sequences were then used in a pitch memory test based on a delayed-tone recognition paradigm. Listeners (N = 48) were asked to compare the first note of each melody (the standard) with a final, appended comparison tone that was either the same pitch or transposed by one semitone. The results showed that various factors including the presence of an interference tone one semitone away from the standard tone, the degree of tonality of the melodic sequence, and the tonal fitness of the standard and comparison tones predicted listener responses. In particular, the fitness of the comparison tone was a key factor in how listeners performed in the recognition task: comparison tones with higher fitness values increased performance when the comparison and standard were the same but decreased performance when they were different. These results illustrate how tonality can both facilitate and interfere with pitch encoding and recognition, providing a detailed and definitive perspective on how pitch memory is influenced by tonal contexts.

Musical training and the perception of phonetic detail in a shadowing task

Phonetic convergence, or shadowing, is the phenomenon in which people unintentionally and temporarily change phonetic details of their speech to sound more similar to another talker. Previous research has examined effects of the shadower and of the target speaker. In the current study, we examine listener contributions on the perception of phonetic convergence. Two hundred and sixty listeners completed an AXB perception task in which they were asked to determine whether the first or third stimulus was a better imitation of the middle stimulus. Listeners provided information about previous instrumental and vocal training. Preliminary examination of the results reveals that listeners with more musical training (in the form of instrumental or vocal experience) lead to better accuracy. Results from the study suggest that there is a link between musical training and detection of fine-grained phonetic details in speech.

Evaluating Hierarchical Structure in Music Annotations

Music exhibits structure at multiple scales, ranging from motifs to large-scale functional components. When inferring the structure of a piece, different listeners may attend to different temporal scales, which can result in disagreements when they describe the same piece. In the field of music informatics research (MIR), it is common to use corpora annotated with structural boundaries at different levels. By quantifying disagreements between multiple annotators, previous research has yielded several insights relevant to the study of music cognition. First, annotators tend to agree when structural boundaries are ambiguous. Second, this ambiguity seems to depend on musical features, time scale, and genre. Furthermore, it is possible to tune current annotation evaluation metrics to better align with these perceptual differences. However, previous work has not directly analyzed the effects of hierarchical structure because the existing methods for comparing structural annotations are designed for “flat” descriptions, and do not readily generalize to hierarchical annotations. In this paper, we extend and generalize previous work on the evaluation of hierarchical descriptions of musical structure. We derive an evaluation metric which can compare hierarchical annotations holistically across multiple levels. sing this metric, we investigate inter-annotator agreement on the multilevel annotations of two different music corpora, investigate the influence of acoustic properties on hierarchical annotations, and evaluate existing hierarchical segmentation algorithms against the distribution of inter-annotator agreement.

Hyperscore [graphics software package]

Hyperscore software was developed by Mary Farbood and Egon Pasztor. Hyperscore is a research prototype and is still in progress. There are two versions of Hyperscore: one that needs a display with a resolution of at least 1024/spl times/768, and one that works with displays that are only 800/spl times/600. Hyperscore needs DirectX to run. This paper reviews the Hyperscore software.