Jan Van Balen

Designing Games with a Purpose for Data Collection in Music Research. Emotify and Hooked: Two Case Studies

Collecting ground truth data for music research requires large amounts of time and money. To avoid these costs, researchers are now trying to collect information through online multiplayer games with the underlying purpose of collecting scientific data. In this paper we present two case studies of such games created for data collection in music information retrieval (MIR): Emotify, for emotional annotation of music, and Hooked, for studying musical catchiness. In addition to the basic requirement of scientific validity, both applications address essential development and design issues, for example, acquiring licensed music or employing popular social frameworks. As such, we hope that they may serve as blueprints for the development of future serious games, not only for music but also for other humanistic domains. The pilot launch of these two games showed that their models are capable of engaging participants and supporting large-scale empirical research.

Automatic Segmentation and Deep Learning of Bird Sounds

We present a study on automatic birdsong recognition with deep neural networks using the birdclef2014 dataset. Through deep learning, feature hierarchies are learned that represent the data on several levels of abstraction. Deep learning has been applied with success to problems in fields such as music information retrieval and image recognition, but its use in bioacoustics is rare. Therefore, we investigate the application of a common deep learning technique deep neural networks in a classification task using songs from Amazonian birds. We show that various deep neural networks are capable of outperforming other classification methods. Furthermore, we present an automatic segmentation algorithm that is capable of separating bird sounds from non-bird sounds.

Sample Identification in Hip Hop Music

Sampling is a creative tool in composition that is widespread in popular music production and composition since the 1980s. However, the concept of sampling has for a long time been unaddressed in Music Information Retrieval. We argue that information on the origin of samples has a great musicological value and can be used to organise and disclose large music collections. In this paper we introduce the problem of automatic sample identification and present a first approach for the case of hip hop music. In particular, we modify and optimize an existing fingerprinting approach to meet the necessary requirements of a realworld sample identification task. The obtained results show the viability of such an approach, and open new avenues for research, especially with regard to inferring artist influences and detecting musical reuse.

Tonic: Combining Ranking and Clustering Dynamics for Music Discovery

This paper describes the design of Tonic, a novel web interface for music discovery and playlist creation. Tonic maps songs into a two dimensional space using a combination of free tags, metadata, and audio-derived features. Search results are presented in this two dimensional space using a combination of clustering and ranking visualization strategies. Tonic was ranked first in the 2014 MIREX User Experience Grand Challenge, where it was evaluated in terms of learnability, robustness and overall user satisfaction, amongst others.