Petri Toiviainen

A Matlab Toolbox for Music Information Retrieval

We present MIRToolbox, an integrated set of functions written in Matlab, dedicated to the extraction from audio files of musical features related, among others, to timbre, tonality, rhythm or form. The objective is to offer a state of the art of computational approaches in the area of Music Information Retrieval (MIR). The design is based on a modular framework: the different algorithms are decomposed into stages, formalized using a minimal set of elementary mechanisms, and integrating different variants proposed by alternative approaches — including new strategies we have developed —, that users can select and parametrize. These functions can adapt to a large area of objects as input.

Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm

We investigated the neural underpinnings of timbral, tonal, and rhythmic features of a naturalistic musical stimulus. Participants were scanned with functional Magnetic Resonance Imaging (fMRI) while listening to a stimulus with a rich musical structure, a modern tango. We correlated temporal evolutions of timbral, tonal, and rhythmic features of the stimulus, extracted using acoustic feature extraction procedures, with the fMRI time series. Results corroborate those obtained with controlled stimuli in previous studies and highlight additional areas recruited during musical feature processing. While timbral feature processing was associated with activations in cognitive areas of the cerebellum, and sensory and default mode network cerebrocortical areas, musical pulse and tonality processing recruited cortical and subcortical cognitive, motor and emotion-related circuits. In sum, by combining neuroimaging, acoustic feature extraction and behavioral methods, we revealed the large-scale cognitive, motor and limbic brain circuitry dedicated to acoustic feature processing during listening to a naturalistic stimulus. In addition to these novel findings, our study has practical relevance as it provides a powerful means to localize neural processing of individual acoustical features, be it those of music, speech, or soundscapes, in ecological settings.

Cross-cultural music cognition: cognitive methodology applied to North Sami yoiks

This article is a study of melodic expectancy in North Sami yoiks, a style of music quite distinct from Western tonal music. Three different approaches were taken. The first approach was a statistical style analysis of tones in a representative corpus of 18 yoiks. The analysis determined the relative frequencies of tone onsets and two- and three-tone transitions. It also identified style characteristics, such as pentatonic orientation, the presence of two reference pitches, the frequency of large consonant intervals, and a relatively large set of possible melodic continuations. The second approach was a behavioral experiment in which listeners made judgments about melodic continuations. Three groups of listeners participated. One group was from the Sami culture, the second group consisted of Finnish music students who had learned some yoiks, and the third group consisted of Western musicians unfamiliar with yoiks. Expertise was associated with stronger veridical expectations (for the correct next tone) than schematic expectations (based on general style characteristics). Familiarity with the particular yoiks was found to compensate for lack of experience with the musical culture. The third approach simulated melodic expectancy with neural network models of the self-organizing map (SOM) type (Kohonen, T. (1997). Self-organizing maps (2nd ed.). Berlin: Springer). One model was trained on the excerpts of yoiks used in the behavioral experiment including the correct continuation tone, while another was trained with a set of Finnish folk songs and Lutheran hymns. The convergence of the three approaches showed that both listeners and the SOM model are influenced by the statistical distributions of tones and tone sequences. The listeners and SOM models also provided evidence supporting a core set of psychological principles underlying melody formation whose relative weights appear to differ across musical styles.

Measuring and modeling real-time responses to music: The dynamics of tonality induction

We examined a variety of real-time responses evoked by a single piece of music, the organ Duetto BWV 805 by J S Bach. The primary data came from a concurrent probe-tone method in which the probe-tone is sounded continuously with the music. Listeners judged how well the probe tone fit with the music at each point in time. The process was repeated for all probe tones of the chromatic scale. A self-organizing map (SOM) [Kohonen 1997 Self-organizing Maps (Berlin: Springer)] was used to represent the developing and changing sense of key reflected in these judgments. The SOM was trained on the probe-tone profiles for 24 major and minor keys (Krumhansl and Kessler 1982 Psychological Review 89 334–368). Projecting the concurrent probe-tone data onto the map showed changes both in the perceived keys and in their strengths. Two dynamic models of tonality induction were tested. Model 1 is based on pitch class distributions. Model 2 is based on the tone-transition distributions; it tested the idea that the order of tones might provide additional information about tonality. Both models contained dynamic components for characterizing pitch strength and creating pitch memory representations. Both models produced results closely matching those of the concurrent probe-tone data. Finally realtime judgments of tension were measured. Tension correlated with distance away from the predominant key in the direction of keys built on the dominant and supertonic tones, and also correlated with dissonance.

Melodic Expectation in Finnish Spiritual Folk Hymns: Convergence of Statistical, Behavioral, and Computational Approaches

This study of Finnish spiritual folk hymns combined three approaches to understanding melodic expectation. The first approach was a statistical style analysis of a representative corpus of 18 hymns, which determined the relative frequencies of tone onsets and two- and three-tone transitions. The second approach was a behavioral experiment in which listeners, either familiar (experts) or unfamiliar (nonexperts) with the hymns, made judgments about melodic continuations. The third approach simulated melodic expectation with neural network models of the self-organizing map (SOM) type (Kohonen, 1997). One model was trained on a corpus of Finnish folk songs and Lutheran hymns (Finnish SOM), while another was trained with the hymn contexts used in the experiment with the correct continuation tone (Hymn SOM). The three approaches converged on the following conclusions: (1) Listeners appear to be sensitive to the distributions of tones and tone transitions in music, (2) The nonexperts9 responses more strongly reflected the general distribution of tones, whereas the experts9 responses more strongly reflected the tone transitions and the correct continuations, (3) The SOMs produced results similar to listeners and also appeared sensitive to the distributions of tones and tone transitions, (4) The Hymn SOM correlated more strongly with the experts9 judgments than the Finnish SOM, and (5) the principles of the implication-realization model (Narmour, 1990) were weighted similarly by the behavioral data and the Hymn SOM. /// Tassa suomalaisia hengellisia kansansavelmia kasittelevassa tutkimuksessa pyrittiin selvittamaan melodisia odotuksia kolmen tutkimusmenetelman avulla. Ensimmainen menetelma oli kyseista tyylia edustavien 18 savelman tilastollinen analyysi, jossa maaritelteltiin savelkorkeuksien seka kahden ja kolmen savelen siirtymien tilastolliset jakaumat. Toinen menetelma oli behavioraalinen koe, jossa kuulijat arvioivat savelmien jatkoja. Kuulijat jakaantuivat kahteen ryhmaan: savelmat tunteviin (asiantuntijoihin) ja savelmia tuntemattomiin (ei-asiantuntijoihin). Kolmannessa menetelmassa simuloitiin melodisia odotuksia itsejarjestaytyvaan karttaan (Kohonen, 1997) perustuvalla keinotekoisella hermoverkkomallilla. Ensimmaiselle mallille opetettiin joukko suomalaisia kansanlauluja ja luterilaisia virsia (suomalainen verkko), toiselle kokeessa kaytettyja hengellisia kansansavelmia (hengellinen verkko). Kaytetyt menetelmat tuottivat yhtenevia tuloksia ja antoivat aihetta seuraaviin johtopaatoksiin: (1) kuulijat nayttavat olevan vastaanottavaisia musiikin saveljakaumille ja savelsiirtymille, (2) ei-asiantuntijoiden vastaukset noudattivat enemman savelten yleista jakaumaa, kun taas asiantuntijoiden vastaukset heijastivat enemman savelsiirtymia ja savelmien oikeita jatkoja, (3) hermoverkot tuottivat tuloksia, jotka olivat samankaltaisia kuulijoiden arvioiden kanssa ja jotka noudattivat savelten ja savelsiirtymien jakaumia, (4) hengellisen verkon tulokset korreloivat suomalaisen verkon tuloksia voimakkaammin asiantuntijoiden arvioiden kanssa, ja (5) behavioraaliset tulokset ja hengellinen verkko painottavat implikaatio-realisaatio-mallin (Narmour, 1990) periaatteita samalla tavalla.

Perceived complexity of western and African folk melodies by western and African listeners

Stylistic knowledge and enculturation play a significant role in music perception, although the importance of psychophysical cues in perception of emotions in music has been acknowledged. The psychophysical cues, such as melodic complexity, are assumed to be independent of musical experience. A cross-cultural comparison was used to investigate the ratings of melodic complexity of western and African participants for western (Experiment 1) and African folk songs (Experiment 2). A range of melodic complexity measures was developed to discover what factors contribute to complexity. On the whole, the groups gave similar patterns of responses in both experiments. In Experiment 1, western folk songs represented a style that was familiar for both groups and the results portrayed the differences in stylistic knowledge and high predictive rate of melodic variables. In Experiment 2, African folk songs were stylistically familiar only for the African group and the results illustrated a lower predictive rate of variables and differences between the groups in rhythm and structural variables. These results suggest that the melodic complexity ratings are influenced by musical enculturation.

Influences of rhythm- and timbre-related musical features on characteristics of music-induced movement.

Music makes us move. Several factors can affect the characteristics of such movements, including individual factors or musical features. For this study, we investigated the effect of rhythm- and timbre-related musical features as well as tempo on movement characteristics. Sixty participants were presented with 30 musical stimuli representing different styles of popular music, and instructed to move along with the music. Optical motion capture was used to record participants’ movements. Subsequently, eight movement features and four rhythm- and timbre-related musical features were computationally extracted from the data, while the tempo was assessed in a perceptual experiment. A subsequent correlational analysis revealed that, for instance, clear pulses seemed to be embodied with the whole body, i.e., by using various movement types of different body parts, whereas spectral flux and percussiveness were found to be more distinctly related to certain body parts, such as head and hand movement. A series of ANOVAs with the stimuli being divided into three groups of five stimuli each based on the tempo revealed no significant differences between the groups, suggesting that the tempo of our stimuli set failed to have an effect on the movement features. In general, the results can be linked to the framework of embodied music cognition, as they show that body movements are used to reflect, imitate, and predict musical characteristics.

Timbre Similarity: Convergence of Neural, Behavioral, and Computational Approaches

The present study compared the degree of similarity of timbre representations as observed with brain recordings, behavioral studies, and computer simulations. To this end, the electrical brain activity of subjects was recorded while they were repetitively presented with five sounds differing in timbre. Subjects read simultaneously so that their attention was not focused on the sounds. The brain activity was quantified in terms of a change-specific mismatch negativity component. Thereafter, the subjects were asked to judge the similarity of all pairs along a five-step scale. A computer simulation was made by first training a Kohonen self-organizing map with a large set of instrumental sounds. The map was then tested with the experimental stimuli, and the distance between the most active artificial neurons was measured. The results of these methods were highly similar, suggesting that timbre representations reflected in behavioral measures correspond to neural activity, both as measured directly and as simulated in self-organizing neural network models.

From Vivaldi to Beatles and back: Predicting lateralized brain responses to music

We aimed at predicting the temporal evolution of brain activity in naturalistic music listening conditions using a combination of neuroimaging and acoustic feature extraction. Participants were scanned using functional Magnetic Resonance Imaging (fMRI) while listening to two musical medleys, including pieces from various genres with and without lyrics. Regression models were built to predict voxel-wise brain activations which were then tested in a cross-validation setting in order to evaluate the robustness of the hence created models across stimuli. To further assess the generalizability of the models we extended the cross-validation procedure by including another dataset, which comprised continuous fMRI responses of musically trained participants to an Argentinean tango. Individual models for the two musical medleys revealed that activations in several areas in the brain belonging to the auditory, limbic, and motor regions could be predicted. Notably, activations in the medial orbitofrontal region and the anterior cingulate cortex, relevant for self-referential appraisal and aesthetic judgments, could be predicted successfully. Cross-validation across musical stimuli and participant pools helped identify a region of the right superior temporal gyrus, encompassing the planum polare and the Heschls gyrus, as the core structure that processed complex acoustic features of musical pieces from various genres, with or without lyrics. Models based on purely instrumental music were able to predict activation in the bilateral auditory cortices, parietal, somatosensory, and left hemispheric primary and supplementary motor areas. The presence of lyrics on the other hand weakened the prediction of activations in the left superior temporal gyrus. Our results suggest spontaneous emotion-related processing during naturalistic listening to music and provide supportive evidence for the hemispheric specialization for categorical sounds with realistic stimuli. We herewith introduce a powerful means to predict brain responses to music, speech, or soundscapes across a large variety of contexts.

Musical Timbre : Similarity Ratings Correlate With Computational Feature Space Distances

Abstract Tarahe paper compares mappings of a set of synthetic timbre stimuli on a Kohonen self‐organizing topology and the distance matrix derived from subjects’ similarity ratings concerning the same stimuli. Significant correlation is reported between the two domains. This result is interpreted to provide support for a more general hypothesis implicit in many current connectionist studies, namely that to represent is to project complexes of multidimensional environmental‐perceptual conditions onto less dimensionally defined, ordered neural‐mental responses.