Music-Circles: Can Music Be Represented With Numbers?
Seokgi Kim, Jihye Park, Kihong Seong, Namwoo Cho, Junho Min, Hwajung Hong
MMusic-Circles : Can Music Be Represented With Numbers?
Seokgi Kim, Jihye Park, Kihong Seong, Namwoo Cho, Junho Min, Hwajung Hong *† Figure 1: Overview of the three pages of
Music-Circles . Left: First Page (Circular Barplot, Table, Radar Chart); Middle: Second Page(Survey); Right: Third Page (Cluster Visualization - Bubble Chart, Radar Chart, Table) A BSTRACT
The world today is experiencing an abundance of music like no othertime, and attempts to group music into clusters have become increas-ingly prevalent. Common standards for grouping music were songs,artists, and genres, with artists or songs exploring similar genresof music seen as related. These clustering attempts serve criticalpurposes for various stakeholders involved in the music industry.For end users of music services, they may want to group their musicso that they can easily navigate inside their music library; for musicstreaming platforms like Spotify, companies may want to establish asolid dataset of related songs in order to successfully provide person-alized music recommendations and coherent playlists to their users.Due to increased competition in the streaming market, platforms aretrying their best to find novel ways of learning similarities betweenaudio to gain competitive advantage. Our team, comprised of musiclovers with different tastes, was interested in the same issue, andcreated
Music-Circles , an interactive visualization of music fromthe Billboard.
Music-Circles links audio feature data offered bySpotify to popular songs to create unique vectors for each song, andcalculate similarities between these vectors to cluster them. Throughinteracting with
Music-Circles , users can gain understandings ofaudio features, view characteristic trends in popular music, and findout which music cluster they belong to.
Index Terms:
Music Clustering—K-Means Clustering—MusicRecommendation; Data Visualization—Interactive Visualization * Corresponding author. † Date of current manuscript version January 4, 2021. (All authors con-tributed equally to this work.)
S. Kim, K. Seong, J. Min are with the Ambient NLP Laboratory, GraduateSchool of Data Science, Seoul National University, South Korea (e-mail:[email protected]; [email protected]; [email protected]).J. Park is with the Data Mining Laboratory, Department of Indus-trial Engineering, Seoul National University, South Korea (e-mail: [email protected]).N. Cho is with with the Human Interface Systems Laboratory, Departmentof Industrial Engineering, Seoul National University, South Korea (e-mail:[email protected]).Code is available at: https://github.com/chonamwoo/snuDxd
NTRODUCTION
The music industry went through significant adjustments wheneverthe prominent medium used to listen to music changed. Threemajor changes in medium for music are notable; Vinyls to CDs,CDs to digital downloads, digital downloads to online streaming.The second change from CDs to digital downloads introduced amonumental shift of music from tangible forms to digital forms,and the third transition to online streaming led to a demand forenhanced analytics of the digitized audio content. This was becauseonline streaming platforms offered the complete music library tolisten to by a subscription basis, whereas in the era of downloads,users would find songs that they like and download them to listen.Given a sea of audio contents, users might get lost, so the streamingplatforms decided to use personalized music recommendations as aguide provided to the users.To make this guide, platforms had to invent an algorithm that findssimilar audio contents to a song that a certain user likes. Therefore,through various methods such as collaborative filtering and contentbased filtering, users are recommended songs that they might like.However, in general, the algorithm itself isn’t explained to the users,and the recommendations take forms of song, artist and album nameswith album covers provided if available. If the user has no contextabout the recommended song, artist and album, there is no visualaid to help the user understand that this song is actually related toother songs that the user prefers.In this paper, we present
Music-Circles , a system that supportsuser comprehension of audio features used to cluster music, andoffers visualizations of different music clusters.
Music-Circles bringtogether two music related data sources with several audio featuresand billboard chart song information into integrated visual interfacesto help users understand, and offer coordinated visualizations thatchange simultaneously as the user interacts. The coordinated viewsare expected to give the users macro and micro pictures of differentfeatures and clusters. This multiview style look was selected sincewe wanted to create a functional system that can help clearly under-stand the data. Fig. 1 shows an overview of the system. The firstpage, which we will refer to as the discover-screen in the sectionsbelow, contains annotations and multiple visualizations that willgive the user an understanding of the audio features of songs. Thenthe system moves on to the second page (will be referred to as thesurvey page), which contains a survey that will help the users find amusic cluster that matches their taste. After completing the survey,users are directed to the third page (will be referred to as the cluster-view), which shows annotations and multiple visualizations about a r X i v : . [ c s . H C ] F e b he personalized cluster results. Music-Circles uses songs from theBillboard hot 100 songs chart so that there will be a high chanceof songs being familiar to users, and allows users to analyze thesesongs in ways that serve the users interest. For example, the user cansee what audio features are similar between two songs or find out acertain feature value for a familiar song to get an understanding ofwhich musical feelings/vibes lead to which features.Main contributions of our work are as follows:• A survey research in the field of music consumption;•
Music-Circles , an interactive visual analytics system for ex-ploring audio feature trends of popular songs and discoveringcharacteristics of personalized song clusters;
ELATED W ORK
Primary purpose of music is entertainment and activities surround-ing music including music discovery should be engaging and en-tertaining; the main purpose of our work is to create an interestingvisualization system that music enthusiasts can enjoy [4]. Thus, wehave referred to several works to effectively visualize music data.To visualize individual characteristics of each song, a few issuesshould be taken into account. First, each song should be displayedin an equal relationship rather than a vertical relationship. Second, itshould be possible to put numerous songs on one screen. AlthoughGuedes and Freitas [2] solved these issues by arranging songs in acircular shape, there still seemed to be room for improvement sincethey did not include sorting buttons that can sort the features.The main data of our work are features of the songs. Film Flow-ers visualizes their movie data in terms of the size of flowers,number of petals, and colors. Inspired by this, we came up witha visualization to simultaneously show the data for each song. Byusing the radar chart, we succeeded in simultaneously expressingfeatures of data for songs and data for each cluster.You can see the multiview style in both the discover-screen andthe cluster-view. We referred to PlayerView and confirmed thatmore information can be easily obtained by connecting each chartinside a multiview system. The discover-screen includes a circularbarplot, radar chart, and a top songs table to form a multiview ofthe feature information of each song. In the cluster-view, a bubblechart, radar chart, and an interactive table containing songs abouteach cluster form a multiview of the information about the cluster. ASK A BSTRACTION
Eliciting tasks and system requirements are not easy in visualiz-ing music since users usually cannot articulate their needs directly.Therefore, rather than simply introspecting about the needs and ac-tions taken in exploring information within the music, interviewswere utilized, since an interview is the arsenal of ethnographicallyinspired methods used in human computer interaction [6].After interviewing five people who listen to music on a daily basiswith great interests in music, we identified some of their difficultiesand needs. Music enthusiasts were often interested in the informationabout the songs that they like. Yet, the current practice is that theyarbitrarily receive one-sided information of relevant songs chosen bythe algorithms used in the streaming services unless they proactivelysearch for the information themselves. They wanted an easier wayto find more about the patterns and features within the songs thatthey like and within the popular songs.We then set two goals of our visualizations; enabling the usersto readily see the trends by different characteristics of the songsand song clusters, and allowing them to freely explore the givenvisualizations to find out patterns of their interest with interactionsand annotations provided as a guide for these interactions. We also https://sxywu.com/filmflowers/ https://buckets.peterbeshai.com/ determined the actions the user could take: sorting the features andthus analyzing trends across all features, spotting outlier values,finding out songs matching one’s taste, etc. Furthermore, the teamdecided to enhance engagement and enjoyment for users during theseactions. These analyses became the rationales behind determiningappropriate things to visualize, settling the strategies for handlingthe complexity within the data, and choosing effective visual idioms. ATA A BSTRACTION
We collected two datasets from Kaggle. One contains data onartists and songs that appeared on Billboard Hot 100 from 1999 to2019; and the other offered by Spotify contains audio features of160,000 or more songs released in between 1921 and 2020. Theseaudio features were acquired by Spotify through surveys with 10,000or more listeners on their affective experience of the songs; suchlexical Kansei parameters are very useful for sentimental analysisand affective attribute classfication [9]. There are hot 100 ranking columns in the Billboard dataset, andnumerous features in the Spotify dataset. We merged these twotables, linking audio features in the Spotify dataset to Billboardsongs by song name, to form one combined dataset containing songsfrom the Billboard and their audio feature values.To identify groups of similar songs, we chose the K-means clus-tering method. Prior to applying clustering method, we performedseveral preprocessing steps on the audio features. First, we identi-fied duplicate songs based on the title and artist and removed songsexcept for songs with the highest weekly rank among the duplicatesongs. The total number of songs used in this study was 4,314. Sec-ond, we excluded four binary-features among the 14 audio features,since they were deemed unfit for clustering. Third, we normalizedthe range of three features (key, loudness, tempo) ranging from 0 to1 in order to match the range of other features.After exploring different clustering options in terms of featuresand number of clusters, we carefully chose six features (acousticness,danceability, energy, key, tempo, valence) for five clusters. The finalclusters seemed to be easily distinguished from one another withouthaving extremely low or high feature values.
ISUAL D ESIGN
After the data was transformed to suit our needs, it was visualizedas several idioms to serve the purpose of
Music-Circles . The discover-screen is the initial page presented to the user, and isintended to help users gain insight about what the audio featuresmean, and possibly discover trends and relations between the fea-tures. Annotations shown in the left image of Fig. 1 are included asa guide to help users understand our intentions, and easily interactwith the visualizations.
Circular barplots represent data by arranging bars in a circle. Asnapshot of the discover-screen, Fig. 2, shows such visualizationimplemented in our work. With the intention of examining thecharacteristics of popular songs, we visualized 168 songs that rankedNo.1 on the Billboard chart. The length of each bar means the valueof the selected feature of the song, and the title of each song ismarked at the end of the bar. igure 2: Discover-Screen: Circular Barplot, Top 5 Table, Radar Chart Users can select 5 kinds of features (Acousticness, Danceability,Energy, Tempo, and Valence) under the ’Which feature do you wantto see?’ annotation in the top. When the feature button is pressed,the new circular barplot appears as a feature corresponding to thebutton. We designed the color of the bars to change when a newfeature is selected. This animation was included in order to make theinteraction process more interesting. Users can also see the ’Top 5Songs Table’ displayed at the top right, which shows the 5 songs ofthe highest value for the selected feature including the album images.With this table, users are expected to relate characteristics of songsthey know to certain features to get a better understanding of whatthe features mean. For example, the table in Fig. 2 shows that thesong ’Despacito’ was among the songs with the highest tempo value.Users who are familiar with this song can get a better feel of what ahigh tempo song would sound like.
With the intention of offering users the capability to examine thetrend of relationships between one feature and others, we imple-mented a sorting button in the form of dropdown options, which cansort the bars according to the 5 different features. When a sortingoption is selected, sorting is performed so that the bars are sortedin clockwise direction, from songs with the highest selected featurevalues to songs with the lowest values. The reason why we chose tosort from the highest to the lowest in clockwise direction is becausewe assumed that users would probably want to see the song with thehighest value first when they select a feature. Since we marked thehighest and lowest point, users can easily recognize which song hasthe highest or lowest value.
All bars are hoverable, and enables users to see information of thesong (title, artist, release year, and the selected feature value) in themiddle when the cursor its hovered on them. A radar chart whichsimultaneously reacts to this hovering action in the bottom right ofFig. 2 shows the 5 feature values of the selected song, so that theoverall feature distribution of each song can be seen.
Radar charts are one of the useful ways to display multivariateobservations with an arbitrary number of variables; thus, we usea radar chart to display various audio features of each song. Toprevent users from being overwhelmed by the visual clutter, we
Figure 3: Survey PageFigure 4: Cluster-View: Visualizations of information about the clusterassigned to each user after survey display less than 7 features, which abides by the Miller’s magicnumber seven [5].With dots spotted on each radial position along the axes repre-senting each attribute of the component, users can easily grasp theselected song’s characteristics. As mentioned in the explanation ofhoverable bars, the values in the radar chart change as users hoverover the bars and explore different songs. Furthermore, the color ofthe chart changes as users click on the feature buttons to examinedifferent features; the colors were carefully selected to not hinderthe apprehension of the features, and be easily distinguishable.
When users are done with exploring the overall trends and charac-teristics of the audio features in the discover-screen, a button at theend of annotations shown in the bottom left of the left most image inFig.1 guides them to a survey page. This survey page is intended toenhance user involvement and thus make our project more interest-ing. By completing the survey, users can discover what clusters theirtastes belong to, and explore the clusters’ characteristics. The surveypage design is a simple survey outline, with one question askingto choose one favorite song, five song options, and a ’next/finish’button to move on to the next question. The five songs are eachselected from one of the five clusters that we defined in Fig.4, andsongs deemed as most renowned, like Coldplay’s ’Viva La Vida’ orDrake’s ’One Dance’ are presented so there would be a high chancethat a user is familiar with the song. If the user does not recognize asong, a youtube link in the form of an icon is presented for the userto visit and listen to the song.The background is a left to right color gradient with 5 colors, eachcorresponding to one of the five predefined clusters. For example, theorange color (far left) in Fig.3 corresponds to cluster 0 in Fig.4. Theolors are selected are from a rainbow palette. Although we initiallywanted to use a different palette, after some experimentation wefound the rainbow palette to be the most effective in distinguishingbetween clusters. To make the colors easing to the eye, we choselight, pastel versions of the colors. When the user chooses a songin a certain cluster, the color corresponding to that cluster increasesin opacity. This animation was meant to intrigue the users whilethey are taking the survey, and make the process less dull. Theuser is not informed about the nature of the background colors, andonly told to pay attention to the background through an annotationin the discover-screen. This was for the user to wonder about theanimations in the background and be more immersed in the wholeprocess.After choosing four songs from a series of four questions, theuser is presented with a ’View Results’ button that will lead to apage containing the visualizations of a certain cluster. We choseto include four questions only because we did not want the surveyprocess to take too much time, and wanted to focus on the actualvisualizations. With the chosen four song names, a Javascript codeis designed to fetch the matching rows from the dataset containingsong feature vectors, and compute one mean vector. Then, the meanvector will be compared to the mean feature vectors for the fiveclusters with cosine similarity (a common way of deriving similaritybetween vectors), to find out which cluster the user belongs to. Suchsimilarity models basing on closeness of distance function is widelyused in generating playlists [3].
After completing the survey, users are presented with the cluster-view, where they can explore the cluster assigned to them throughthree visualization idioms: bubble chart, radar chart, and table.The three visualizations simultaneously change as users click on thelegends in the center, showing the difference between clusters. Theselinked visualization idioms provide a coordinated view of differentperspectives. An annotation is provided in the top left of the page,containing explanations about the cluster based on characteristics offeature values, and a guide on what the visualizations mean and howto interact with them. A name is assigned to each cluster to makethe engagement process more entertaining and cluster characteristicseasier to understand. The five names are; Karaoke Please, All AboutThat BASS, All About That Treble, Acoustic Life, Positive VibesOnly. One fun fact per cluster is also added to make the readingmuch more enjoyable. The overview of the visualization for clusterinformation for the ’Positive Vibes Only’ cluster is illustrated inFig. 4.
Bubble charts, which represent data by circles of different sizes andcolors, are one of the best ways to simultaneously visualize trendsin three or more dimensions [8]. As shown in Fig. 4, we propose tovisualize the trend of mega-hit songs in multiple dimensions throughan interactive bubble chart. Each bubble represents a mega-hit song,which was not only ranked inside the Billboard Top 10 at least oncebut also included more than 50 weeks in the Billboard Hot 100.The x-axis, y-axis, size, and color represent the year the song wasreleased, peak position at Billboard Hot 100, number of weeks listedon Billboard Hot 100, and the category of a cluster, respectively.Users can also identify the details of the song with a tooltip byhovering over each bubble. Furthermore, the legends are clickable,allowing the users to explore the identities of clusters other than thecluster assigned to the user.
As discussed in Section 5.1.5, radar charts are one of the usefulgraphical methods to display multivariate values of the audio fea-tures. The radar chart next to the bubble chart displays the average of the audio feature values of the songs in each cluster, allowingusers to grasp the identity of each cluster.
The table at the bottom of the page enables effective browsing forthe songs. Users can not only sort the table in lexicographic order foreach column by selecting the table column heading but also searchfor a specific song title or artist.
ESULTS
We developed
Music-Circles to make exploring the features of pop-ular songs possible for the entertainment of music enthusiasts. Toachieve this goal,
Music-Circles consists of three parts.In the first part, discover-screen, through the circular barplot inFig. 2, 5 kinds of features of 168 songs can be compared through thelength of bars. There are feature buttons and a sorting button thatmake users able to see different feature information or to sort thesongs by some given feature. Additionally, a top 5 songs table wascreated to make it easier for users to see the songs with high valuesof a feature at one glance.After discovering the features of these popular songs, the secondpart, survey page is presented. This page contains a survey todiscover a users’ musical tendencies as shown in Fig. 3. Duringthe survey, the background color changes according to the songthe user chooses so the process would not be dull. Users can alsoget a chance to listen to the music when they find unfamiliar songsthrough the youtube links attached to each song.After the survey, the last part is the cluster-view. When the usersclick the ’View Results’ button, the cluster-view screen, as shown inFig. 4, appears differently according to the users. Users can resonateas they read the explanations of the selected cluster, and also seewhich songs are included in the cluster. Also, it can help them decidewhich song clusters to check out in the future by looking at the radarchart displayed differently depending on the cluster.
ISCUSSION AND F UTURE W ORK
Our work presents a new interactive way to explore the identity ofmusic through its attributes. Users may obtain additional knowledgethan what they would initially obtain from just the Billboard rank-ings or Spotify audio features. Our work encourages users to behighly involved through interactive circular barplot, survey assigningdifferent clusters to users, and annotations with interesting facts. Animportant finding in our study is that annotation plays an essentialrole in conveying key points in visual data-driven storytelling; ithelps presenters explain and emphasize core messages and specificdata [7].Without access to the raw audio signals, the scope for the audiofeatures used in this work is limited to those that can be derivedfrom the Spotify API. Our work could be improved by future inves-tigations and comparisons on audio features from other sources suchas the Million Data Song dataset [1].
ONCLUSION
This paper introduces
Music-Circles , an interactive system with an-notations, visualizations, and surveys to keep users entertained whilethey engage with the system to explore audio features and clusters ofpopular songs. Idioms of visualizations were original ideas from theteam and influenced by related works, and a focus group interviewwas conducted to define our task. Data from Spotify and Billboardwere transformed to well suit the purpose of our project, and thetransformed dataset was used to provide a coordinated view of mul-tiple visualizations like the circular barplot, bubble chart, and radarchart to allow users to well analyze the given data.
Music-Circles is mainly meant for entertainment of music lovers,but can be extended to other domains for different users. Producerscan use our system to discover feature trends in popular songs andain better insight in developing a hit song, and journalists canuse various analyses obtained from our system to write articles onpop culture. Most importantly,
Music-Circles will fill the lack ofvisualizations offered in music recommendation platforms, and helpusers better understand characteristics between similar songs. A CKNOWLEDGMENTS
The authors wish to thank Prof. Hwajung Hong for her feedbackson our work. From Hwajung’s Data Visualization course, we gainedknowledge about the process of visualization design and validationthat was utilized in this project. R EFERENCES [1] T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere. The millionsong dataset. 2011.[2] L. Guedes and C. M. D. S. Freitas. Exploring music rankings withinteractive visualization. In
Proceedings of the Symposium on AppliedComputing , pp. 214–219, 2017.[3] P. Knees, M. Schedl, T. Pohle, and G. Widmer. An innovative three-dimensional user interface for exploring music collections enriched. pp.17–24, 01 2006. doi: 10.1145/1180639.1180652[4] P. Lamere and D. Eck. Using 3d visualizations to explore and discovermusic. pp. 173–174, 01 2007.[5] G. A. Miller. The magic number seven plus or minus two: Some limitson our capacity for processing information.
Psychological review , 63:91–97, 1956.[6] T. Munzner.
Visualization Analysis and Design . CRC Press, 2014.[7] D. Ren, M. Brehmer, B. Lee, T. H¨ollerer, and E. K. Choe. Chartac-cent: Annotation for data-driven storytelling. In , pp. 230–239. Ieee, 2017.[8] G. Robertson, R. Fernandez, D. Fisher, B. Lee, and J. Stasko. Ef-fectiveness of animation in trend visualization.
IEEE transactions onvisualization and computer graphics , 14(6):1325–1332, 2008.[9] W. Wang, J. Wang, Z. Li, Z. Tian, and E. Tsui. Multiple affectiveattribute classification of online customer product reviews a heuristicdeep learning method for supporting kansei engineering.