Cyril Laurier

Multimodal Music Mood Classification Using Audio and Lyrics

In this paper we present a study on music mood classification using audio and lyrics information. The mood of a song is expressed by means of musical features but a relevant part also seems to be conveyed by the lyrics. We evaluate each factor independently and explore the possibility to combine both, using natural language processing and music information retrieval techniques. We show that standard distance-based methods and latent semantic analysis are able to classify the lyrics significantly better than random, but the performance is still quite inferior to that of audio-based techniques. We then introduce a method based on differences between language models that gives performances closer to audio-based classifiers. Moreover, integrating this in a multimodal system (audio+text) allows an improvement in the overall performance. We demonstrate that lyrics and audio information are complementary, and can be combined to improve a classification system.

Indexing music by mood: design and integration of an automatic content-based annotator

In the context of content analysis for indexing and retrieval, a method for creating automatic music mood annotation is presented. The method is based on results from psychological studies and framed into a supervised learning approach using musical features automatically extracted from the raw audio signal. We present here some of the most relevant audio features to solve this problem. A ground truth, used for training, is created using both social network information systems (wisdom of crowds) and individual experts (wisdom of the few). At the experimental level, we evaluate our approach on a database of 1,000 songs. Tests of different classification methods, configurations and optimizations have been conducted, showing that Support Vector Machines perform best for the task at hand. Moreover, we evaluate the algorithm robustness against different audio compression schemes. This fact, often neglected, is fundamental to build a system that is usable in real conditions. In addition, the integration of a fast and scalable version of this technique with the European Project PHAROS is discussed. This real world application demonstrates the usability of this tool to annotate large-scale databases. We also report on a user evaluation in the context of the PHAROS search engine, asking people about the utility, interest and innovation of this technology in real world use cases.

Music Mood Annotator Design and Integration

A robust and efficient technique for automatic music mood annotation is presented. A songs mood is expressed by a supervised machine learning approach based on musical features extracted from the raw audio signal. A ground truth, used for training, is created using both social network information systems and individual experts. Tests of 7 different classification configurations have been performed, showing that Support Vector Machines perform best for the task at hand. Moreover, we evaluate the algorithm robustness to different audio compression schemes. This fact, often neglected, is fundamental to build a system that is usable in real conditions. In addition, the integration of a fast and scalable version of this technique with the European Project PHAROS is discussed.

Pharos: an audiovisual search platform

1. THE PHAROS PLATFORM AND DEMO PHAROS [1] is an Integrated Project aimed at building a platform for advanced audiovisual search applications. The Consortium comprises 12 partners from 9 European countries. PHAROS unbundles the functionalities of an audiovisual search engine into an open service-based ecosystem, where content can be submitted to customized analysis pipelines, third-party annotation components can be plugged-in, and content based search engines can be registered. PHAROS enables a variety of application scenarios, from content acquisition and enrichment, to annotation fusion, to multi-modal queries. Figure 1 shows the architecture of PHAROS, which supports two main process: Content Caption and Refinement (CCR) executes flow of operators on the captured content and produces XML metadata (subsequently indexed by a core XML search engine) and derived artifacts(used for similarity querying and result presentation); Query Execution and Result Presentation (QUIRP) accepts a user’s query (by keyword, by image similarity, by audio similarity, by video similarity), expands it with user’s profile and social information, brokers its execution on the registered search engines, and presents results in a Rich Internet Interface. The demo exploits the online access to the PHAROS platform for an in-depth tour of: content acquisition, design and