Nori Jacoby

Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants

Human language, as well as birdsong, relies on the ability to arrange vocal elements in new sequences. However, little is known about the ontogenetic origin of this capacity. Here we track the development of vocal combinatorial capacity in three species of vocal learners, combining an experimental approach in zebra finches (Taeniopygia guttata) with an analysis of natural development of vocal transitions in Bengalese finches (Lonchura striata domestica) and pre-lingual human infants. We find a common, stepwise pattern of acquiring vocal transitions across species. In our first study, juvenile zebra finches were trained to perform one song and then the training target was altered, prompting the birds to swap syllable order, or insert a new syllable into a string. All birds solved these permutation tasks in a series of steps, gradually approximating the target sequence by acquiring new pairwise syllable transitions, sometimes too slowly to accomplish the task fully. Similarly, in the more complex songs of Bengalese finches, branching points and bidirectional transitions in song syntax were acquired in a stepwise fashion, starting from a more restrictive set of vocal transitions. The babbling of pre-lingual human infants showed a similar pattern: instead of a single developmental shift from reduplicated to variegated babbling (that is, from repetitive to diverse sequences), we observed multiple shifts, where each new syllable type slowly acquired a diversity of pairwise transitions, asynchronously over development. Collectively, these results point to a common generative process that is conserved across species, suggesting that the long-noted gap between perceptual versus motor combinatorial capabilities in human infants may arise partly from the challenges in constructing new pairwise vocal transitions.

Sensorimotor synchronization with tempo-changing auditory sequences: Modeling temporal adaptation and anticipation

The current study investigated the human ability to synchronize movements with event sequences containing continuous tempo changes. This capacity is evident, for example, in ensemble musicians who maintain precise interpersonal coordination while modulating the performance tempo for expressive purposes. Here we tested an ADaptation and Anticipation Model (ADAM) that was developed to account for such behavior by combining error correction processes (adaptation) with a predictive temporal extrapolation process (anticipation). While previous computational models of synchronization incorporate error correction, they do not account for prediction during tempo-changing behavior. The fit between behavioral data and computer simulations based on four versions of ADAM was assessed. These versions included a model with adaptation only, one in which adaptation and anticipation act in combination (error correction is applied on the basis of predicted tempo changes), and two models in which adaptation and anticipation were linked in a joint module that corrects for predicted discrepancies between the outcomes of adaptive and anticipatory processes. The behavioral experiment required participants to tap their finger in time with three auditory pacing sequences containing tempo changes that differed in the rate of change and the number of turning points. Behavioral results indicated that sensorimotor synchronization accuracy and precision, while generally high, decreased with increases in the rate of tempo change and number of turning points. Simulations and model-based parameter estimates showed that adaptation mechanisms alone could not fully explain the observed precision of sensorimotor synchronization. Including anticipation in the model increased the precision of simulated sensorimotor synchronization and improved the fit of model to behavioral data, especially when adaptation and anticipation mechanisms were linked via a joint module based on the notion of joint internal models. Overall results suggest that adaptation and anticipation mechanisms both play an important role during sensorimotor synchronization with tempo-changing sequences. This article is part of a Special Issue entitled SI: Prediction and Attention.

Parameter Estimation of Linear Sensorimotor Synchronization Models: Phase Correction, Period Correction, and Ensemble Synchronization

Linear models have been used in several contexts to study the mechanisms that underpin sensorimotor synchronization. Given that their parameters are often linked to psychological processes such as phase correction and period correction, the fit of the parameters to experimental data is an important practical question. We present a unified method for parameter estimation of linear sensorimotor synchronization models that extends available techniques and enhances their usability. This method enables reliable and efficient analysis of experimental data for single subject and multi-person synchronization. In a previous paper (Jacoby et al., 2015), we showed how to significantly reduce the estimation error and eliminate the bias of parameter estimation methods by adding a simple and empirically justified constraint on the parameter space. By applying this constraint in conjunction with the tools of matrix algebra, we here develop a novel method for estimating the parameters of most linear models described in the literature. Through extensive simulations, we demonstrate that our method reliably and efficiently recovers the parameters of two influential linear models: Vorberg and Wing (1996), and Schulze et al. (2005), together with their multi-person generalization to ensemble synchronization. We discuss how our method can be applied to include the study of individual differences in sensorimotor synchronization ability, for example, in clinical populations and ensemble musicians.

Both Isochronous and Non-Isochronous Metrical Subdivision Afford Precise and Stable Ensemble Entrainment: A Corpus Study of Malian Jembe Drumming.

Most approaches to musical rhythm, whether in music theory, music psychology, or musical neuroscience, presume that musical rhythms are based on isochronous (temporally equidistant) beats and/or beat subdivisions. However, rhythms that are based on non-isochronous, or unequal patterns of time are prominent in the music of Southeast Europe, the Near East and Southern Asia, and in the music of Africa and the African diaspora. The present study examines one such style found in contemporary Malian jembe percussion music. A corpus of 15 representative performances of three different pieces (“Manjanin,” “Maraka,” and “Woloso”) containing ~43,000 data points was analyzed. Manjanin and Woloso are characterized by non-isochronous beat subdivisions (a short IOI followed by two longer IOIs), while Maraka subdivisions are quasi-isochronous. Analyses of onsets and asynchronies show no significant differences in timing precision and coordination between the isochronously timed Maraka vs. the non-isochronously timed Woloso performances, though both pieces were slightly less variable than non-isochronous Manjanin. Thus, the precision and stability of rhythm and entrainment in human music does not necessarily depend on metric isochrony, consistent with the hypothesis that isochrony is not a biologically-based constraint on human rhythmic behavior. Rather, it may represent a historically popular option within a variety of culturally contingent options for metric organization.

Musically puzzling I: Sensitivity to overall structure in the sonata form?

Previous studies have suggested that listeners are not sensitive to the overall tonal structure of musical pieces. This assumption is reexamined in the current study in an active musical puzzle task, with no time constraints, focusing on the presumably most directional musical form – the sonata form. In our first study (reported here, and referred to as “the Mozart study”), participants with varying levels of musical training were presented with disordered sections of Mozart’s piano sonata K. 570/I in B flat major and asked to rearrange the ten sections into a musically logical coherent whole. A second study (to be reported in Musicae Scientiae issue 16[1]) replicated the task in a different group of participants who listened to Haydn’s piano sonata, Hob: XVI-34/I in E minor. In contrast with previous studies, we do not focus on listeners’ ability to recover the original sonatas. Rather, we explore emergent patterns in their responses using new types of analysis. Our results indicate that listeners show: (1) Some sensitivity to the overall structure of A-B-A’ around the non-stable B section; (2) Non- trivial sensitivity to overall “directionality” through a new type of analysis (“distance score”); (3) Correct grouping and placement of developmental sections possibly related to listener’s sensitivity to musical tension; (4) Sensitivity to opening and closing gestures, thematic similarity and surface cues and; (5) No sensitivity to global harmonic structure.

Rhythm histograms and musical meter: A corpus study of Malian percussion music

Studies of musical corpora have given empirical grounding to the various features that characterize particular musical styles and genres. Palmer & Krumhansl (1990) found that in Western classical music the likeliest places for a note to occur are the most strongly accented beats in a measure, and this was also found in subsequent studies using both Western classical and folk music corpora (Huron & Ommen, 2006; Temperley, 2010). We present a rhythmic analysis of a corpus of 15 performances of percussion music from Bamako, Mali. In our corpus, the relative frequency of note onsets in a given metrical position does not correspond to patterns of metrical accent, though there is a stable relationship between onset frequency and metrical position. The implications of this non-congruence between simple statistical likelihood and metrical structure for the ways in which meter and metrical accent may be learned and understood are discussed, along with importance of cross-cultural studies for psychological research.

An auditory illusion reveals the role of streaming in the temporal misallocation of perceptual objects.

This study investigates the neural correlates and processes underlying the ambiguous percept produced by a stimulus similar to Deutschs ‘octave illusion’, in which each ear is presented with a sequence of alternating pure tones of low and high frequencies. The same sequence is presented to each ear, but in opposite phase, such that the left and right ears receive a high–low–high … and a low–high–low … pattern, respectively. Listeners generally report hearing the illusion of an alternating pattern of low and high tones, with all the low tones lateralized to one side and all the high tones lateralized to the other side. The current explanation of the illusion is that it reflects an illusory feature conjunction of pitch and perceived location. Using psychophysics and electroencephalogram measures, we test this and an alternative hypothesis involving synchronous and sequential stream segregation, and investigate potential neural correlates of the illusion. We find that the illusion of alternating tones arises from the synchronous tone pairs across ears rather than sequential tones in one ear, suggesting that the illusion involves a misattribution of time across perceptual streams, rather than a misattribution of location within a stream. The results provide new insights into the mechanisms of binaural streaming and synchronous sound segregation. This article is part of the themed issue ‘Auditory and visual scene analysis’.

Lower bound on the accuracy of parameter estimation methods for linear sensorimotor synchronization models

The mechanisms that support sensorimotor synchronization — that is, the temporal coordination of movement with an external rhythm — are often investigated using linear computational models. The main method used for estimating the parameters of this type of model was established in the seminal work of Vorberg and Schulze (2002), and is based on fitting the model to the observed autocovariance function of asynchronies between movements and pacing events. Vorberg and Schulze also identified the problem of parameter interdependence, namely, that different sets of parameters might yield almost identical fits, and therefore the estimation method cannot determine the parameters uniquely. This problem results in a large estimation error and bias, thereby limiting the explanatory power of existing linear models of sensorimotor synchronization. We present a mathematical analysis of the parameter interdependence problem. By applying the Cramer–Rao lower bound, a general lower bound limiting the accuracy of any parameter estimation procedure, we prove that the mathematical structure of the linear models used in the literature determines that this problem cannot be resolved by any unbiased estimation method without adopting further assumptions. We then show that adding a simple and empirically justified constraint on the parameter space — assuming a relationship between the variances of the noise terms in the model — resolves the problem. In a follow-up paper in this volume, we present a novel estimation technique that uses this constraint in conjunction with matrix algebra to reliably estimate the parameters of almost all linear models used in the literature.

Assessing the applied benefits of perceptual training: Lessons from studies of training working-memory

In the 1980s to 1990s, studies of perceptual learning focused on the specificity of training to basic visual attributes such as retinal position and orientation. These studies were considered scientifically innovative since they suggested the existence of plasticity in the early stimulus-specific sensory cortex. Twenty years later, perceptual training has gradually shifted to potential applications, and research tends to be devoted to showing transfer. In this paper we analyze two key methodological issues related to the interpretation of transfer. The first has to do with the absence of a control group or the sole use of a test-retest group in traditional perceptual training studies. The second deals with claims of transfer based on the correlation between improvement on the trained and transfer tasks. We analyze examples from the general intelligence literature dealing with the impact on general intelligence of training on a working memory task. The re-analyses show that the reports of a significantly larger transfer of the trained group over the test-retest group fail to replicate when transfer is compared to an actively trained group. Furthermore, the correlations reported in this literature between gains on the trained and transfer tasks can be replicated even when no transfer is assumed.

An Information Theoretic Approach to Chord Categorization and Functional Harmony

We present new tools for categorizing chords based on corpus data, applicable to a variety of representations from Roman numerals to MIDI notes. Using methods from information theory, we propose that harmonic theories should be evaluated by at least two criteria, accuracy (how well the theory describes the musical surface) and complexity (the efficiency of the theory according to Occam’s razor). We use our methods to consider a range of approaches in music theory, including function theory, root functionality, and the figured-bass tradition. Using new corpus data as well as eleven datasets from five published works, we argue that our framework produces results consistent both with musical intuition and previous work, primarily by recovering the tonic/subdominant/dominant categorization central to traditional music theory. By showing that functional harmony can be analysed as a clustering problem, we link machine learning, information theory, corpus analysis and music theory.