Anna Xambó

Semantic audio content-based music recommendation and visualization based on user preference examples

Preference elicitation is a challenging fundamental problem when designing recommender systems. In the present work we propose a content-based technique to automatically generate a semantic representation of the users musical preferences directly from audio. Starting from an explicit set of music tracks provided by the user as evidence of his/her preferences, we infer high-level semantic descriptors for each track obtaining a user model. To prove the benefits of our proposal, we present two applications of our technique. In the first one, we consider three approaches to music recommendation, two of them based on a semantic music similarity measure, and one based on a semantic probabilistic model. In the second application, we address the visualization of the users musical preferences by creating a humanoid cartoon-like character - the Musical Avatar - automatically inferred from the semantic representation. We conducted a preliminary evaluation of the proposed technique in the context of these applications with 12 subjects. The results are promising: the recommendations were positively evaluated and close to those coming from state-of-the-art metadata-based systems, and the subjects judged the generated visualizations to capture their core preferences. Finally, we highlight the advantages of the proposed semantic user model for enhancing the user interfaces of information filtering systems.

Let's jam the reactable: Peer learning during musical improvisation with a tabletop tangible interface

There has been little research on how interactions with tabletop and Tangible User Interfaces (TUIs) by groups of users change over time. In this article, we investigate the challenges and opportunities of a tabletop tangible interface based on constructive building blocks. We describe a long-term lab study of groups of expert musicians improvising with the Reactable, a commercial tabletop TUI for music performance. We examine interaction, focusing on interface, tangible, musical, and social phenomena. Our findings reveal a practice-based learning between peers in situated contexts, and new forms of participation, all of which is facilitated by the Reactables tangible interface, if compared to traditional musical ensembles. We summarise our findings as a set of design considerations and conclude that construction processes on interactive tabletops support learning by doing and peer learning, which can inform constructivist approaches to learning with technology.

TOUCHtr4ck: democratic collaborative music

When electronic musicians compose collaboratively, they typically use their own single-user musical controllers. It may, therefore, be useful to develop novel controllers that support collaborative workflows and democratic principles. After describing the design principles for developing such controllers, we present TOUCHtr4ck, a prototype multi-touch system designed to facilitate such democratic relationships. Informal testing has revealed that this approach does facilitate democratic and collaborative music making, and can produce creative musical results.

Exploring methodological innovation in the social sciences: the body in digital environments and the arts

Abstract In this paper we examine methodological innovation in the social sciences through a focus on researching the body in digital environments. There are two strands to our argument as to why this is a useful site to explore methodological innovation in the social sciences. First, researching the body in digital environments places new methodological demands on social science. Second, as an area of interest at the intersection of the social sciences and the arts, it provides a focus for exploring how social science innovation can be informed by engagement with the arts, in this instance how the arts work with the body in digital environments and take up social science ideas in novel ways. We argue that social science engagement with the arts and the relatively unmapped terrain of the body in digital environments has the potential to open up spaces for innovative social science questions and methods: spaces, questions and methods that have potential for more general social science methodological innovation. We draw on the findings of the Methodological Innovation in Digital Arts and Social Sciences (MIDAS) project a multi-site ethnography of the research ecologies of the social sciences and the arts related to the body in digital environments. We propose a continuum of methodological innovation that attends to how methods are moved across research contexts and disciplines, in this instance the social sciences and the digital arts. We illustrate and discuss the innovative potential of expanding and re-situating methods across the social sciences and the arts, the transfer of methods and concepts across disciplinary borders and the interdisciplinary generation of new methods. We discuss the catalysts and challenges for social science methodological innovation in relation to the digital and the arts, with attention to how the social sciences might engage with the arts towards innovative research.

Exploring Social Interaction With a Tangible Music Interface

This article presents a video-based field study of the Reactable, a tabletop tangible user interface (TUI) for music performance, in a hands-on science centre. The goal was to investigate visitors’ social interactions in a public setting. We describe liminality and cross-group interaction, both synchronous with fluid transitions and overlaps in use between groups and asynchronous. Our findings indicate the importance of: (i) facilitating smooth transitions and overlaps between groups and (ii) supporting not only synchronous but also asynchronous group interaction. We discuss the lessons learned on how best to enable liminal situations in the design of interactive tabletops and TUIs for social interaction and particularly collaborative tangible music in public museum settings. • Presentation of a video-based field study describing cross-group interaction. • Description of exemplar vignettes of video for illustrating liminal situations in social interaction. • Discussion of the nature of different levels of cross-group interaction and their relevance for social interaction and collaboration. • Lessons learned in how best to enable liminal situations in the design of collaborative tangible user interfaces for promoting social interaction and collaboration.

EarSketch: A STEAM approach to broadening participation in computer science principles

The EarSketch computer science learning environment and curriculum (http://earsketch.gatech.edu) seeks to increase and broaden participation in computing using a STEAM (STEM + Arts) approach. EarSketch creates an authentic learning environment in that it is both personally meaningful and industry relevant in terms of its STEM component (computing) and its artistic domain (music remixing). Students learn to code in JavaScript or Python, tackling learning objectives in the Computer Science Principles curricular framework as they simultaneously learn core concepts in music technology. They create music through code by uploading their own audio content or remixing loops in popular genres created by music industry veterans. No prior experience in music or computer science is required. EarSketch is entirely browser-based and free.

Embodied Interactions with E-Textiles and the Internet of Sounds for Performing Arts

This paper presents initial steps towards the design of an embedded system for body-centric sonic performance. The proposed prototyping system allows performers to manipulate sounds through gestural interactions captured by textile wearable sensors. The e-textile sensor data control, in real-time, audio synthesis algorithms working with content from Audio Commons, a novel web-based ecosystem for re-purposing crowd-sourced audio. The system enables creative embodied music interactions by combining seamless physical e-textiles with web-based digital audio technologies.

Handwaving: Gesture Recognition for Participatory Mobile Music

This paper describes handwaving, a system for participatory mobile music based on accelerometer gesture recognition. The core of the system is a library that can be used to recognize and map arbitrary gestures to sound synthesizers. Such gestures can be quickly learnt by mobile phone users in order to produce sounds in a musical context. The system is implemented using web standards, so it can be used with most current smartphones without the need of installing specific software.

Beacon: Exploring Physicality in Digital Performance

Live performances which involve digital technology often strive toward clear correspondences between distinct media modes, particularly those works which combine audio and video. Often, the process of creating and executing such performances involves mapping schemes which are encased within the digital system, producing content which is tightly synchronized but with relationships which can feel rigid and unexpressive. Within this paper we present a collaborative process between visualist and musician, which builds toward a method for promoting co-creativity in multimedia performance and prioritizes the performers physical presence and interaction with digital content. Through the development of two autonomous systems, a novel physical interface and an interactive music system, we summarize our creative process of co-exploration of system capabilities, and extended periods of experimentation and exploration. From this experience, we offer an early-stage framework for approaching engaging digital audiovisual relationships in live performance settings.

Turn-Taking and Chatting in Collaborative Music Live Coding

Co-located collaborative live coding is a potential approach to network music and to the music improvisation practice known as live coding. A common strategy to support communication between live coders and the audience is the use of a chat window. However, paying attention to simultaneous multi-user actions, such as chat texts and code, can be too demanding to follow. In this paper, we explore collaborative music live coding (CMLC) using the live coding environment and pedagogical tool EarSketch. In particular, we examine the use of turn-taking and a customized chat window inspired by the practice of pair programming, a team-based strategy to efficiently solving computational problems. Our approach to CMLC also aims at facilitating the understanding of this practice to the audience. We conclude discussing the benefits of this approach in both performance and educational settings.