Nicholas J. Bailey

Inferring musical structure through bodily gestures

Despite the idiosyncrasies present in bodily motion during musical performances, it is suggested that these movements are not incidental but instead contribute to the meaning of performed music. Nine pianists performing two Chopin Preludes are recorded using Vicon motion capture, in synchrony with audio and MIDI recordings. Using principal components analysis (PCA) on the recorded upper body motion, motion profiles for each pianist are formed from the weighted combinations of these components. This measure of global body movement produces a comparable measure across performers and pieces, whilst keeping the performers’ interpretative gestures in view of their whole concept of the piece intact. This study proposes that performers use these motions as a way of corporeally manifesting their interpretative choices. It is suggested that their overall body motion is underscored by such structures as interpreted from the written music. Results show evident underlying musical structure across performers’ motion profiles. These profiles are seen to be constructed from functions of harmonic and melodic relationships in the score. These findings hold significant implications for research into music perception, pedagogy and development of computational technologies for the analysis of music performance.

Efficient Tracking of Pianists’ Finger Movements

Abstract Pianist finger movement contains information about note preparation, accent information as well as personal identifiers. However, no system for detailed finger tracking presently exists that can also be used ecologically: at an acoustic piano whilst also incurring acceptably low disruption to the performance. An optical motion capture system is described using a monocular camera setup to track passive UV paint markers placed directly onto pianists’ fingers. Tracking accuracy and 3D estimation algorithms are evaluated. This low-cost, efficient system maximizes its uses in performance analysis experiments which are most importantly not confined to laboratory environments.