Mehrdad Yazdani

Combining audio content and social context for semantic music discovery

When attempting to annotate music, it is important to consider both acoustic content and social context. This paper explores techniques for collecting and combining multiple sources of such information for the purpose of building a query-by-text music retrieval system. We consider two representations of the acoustic content (related to timbre and harmony) and two social sources (social tags and web documents). We then compare three algorithms that combine these information sources: calibrated score averaging (CSA), RankBoost, and kernel combination support vector machines (KC-SVM). We demonstrate empirically that each of these algorithms is superior to algorithms that use individual information sources.

A simple control policy for achieving minimum jerk trajectories

Point-to-point fast hand movements, often referred to as ballistic movements, are a class of movements characterized by straight paths and bell-shaped velocity profiles. In this paper we propose a bang-bang optimal control policy that can achieve such movements. This optimal control policy is accomplished by minimizing the L∞ norm of the jerk profile of ballistic movements with known initial position, final position, and duration of movement. We compare the results of this control policy with human motion data recorded with a manipulandum. We propose that such bang-bang control policies are inherently simple for the central nervous system to implement and also minimize wear and tear on the bio-mechanical system. Physiological experiments support the possibility that some parts of the central nervous system use bang-bang control policies. Furthermore, while many computational neural models of movement control have used a bang-bang control policy without justification, our study shows that the use of such policies is not only convenient, but optimal.

Using machine learning to identify major shifts in human gut microbiome protein family abundance in disease

Inflammatory Bowel Disease (IBD) is an autoimmune condition that is observed to be associated with major alterations in the gut microbiome taxonomic composition. Here we classify major changes in microbiome protein family abundances between healthy subjects and IBD patients. We use machine learning to analyze results obtained previously from computing relative abundance of ∼10,000 KEGG orthologous protein families in the gut microbiome of a set of healthy individuals and IBD patients. We develop a machine learning pipeline, involving the Kolomogorv-Smirnov test, to identify the 100 most statistically significant entries in the KEGG database. Then we use these 100 as a training set for a Random Forest classifier to determine ∼5% the KEGGs which are best at separating disease and healthy states. Lastly, we developed a Natural Language Processing classifier of the KEGG description files to predict KEGG relative over-or under-abundance. As we expand our analysis from 10,000 KEGG protein families to one million proteins identified in the gut microbiome, scalable methods for quickly identifying such anomalies between health and disease states will be increasingly valuable for biological interpretation of sequence data.

Determining feature extractors for unsupervised learning on satellite images

Advances in satellite imagery presents unprecedented opportunities for understanding natural and social phenomena at global and regional scales. Although the field of satellite remote sensing has evaluated imperative questions to human and environmental sustainability, scaling those techniques to very high spatial resolutions at regional scales remains a challenge. Satellite imagery is now more accessible with greater spatial, spectral and temporal resolution creating a data bottleneck in identifying the content of images. Because satellite images are unlabeled, unsupervised methods allow us to organize images into coherent groups or clusters. However, the performance of unsupervised methods, like all other machine learning methods, depends on features. Recent studies using features from pre-trained networks have shown promise for learning in new datasets. This suggests that features from pre-trained networks can be used for learning in temporally and spatially dynamic data sources such as satellite imagery. It is not clear, however, which features from which layer and network architecture should be used for learning new tasks. In this paper, we present an approach to evaluate the transferability of features from pre-trained Deep Convolutional Neural Networks for satellite imagery. We explore and evaluate different features and feature combinations extracted from various deep network architectures, and systematically evaluate over 2,000 network-layer combinations. In addition, we test the transferability of our engineered features and learned features from an unlabeled dataset to a different labeled dataset. Our feature engineering and learning are done on the unlabeled Draper Satellite Chronology dataset, and we test on the labeled UC Merced Land dataset to achieve near state-of-the-art classification results. These results suggest that even without any or minimal training, these networks can generalize well to other datasets. This method could be useful in the task of clustering unlabeled images and other unsupervised machine learning tasks.

Quantifying the development of user-generated art during 2001–2010

One of the main questions in the humanities is how cultures and artistic expressions change over time. While a number of researchers have used quantitative computational methods to study historical changes in literature, music, and cinema, our paper offers the first quantitative analysis of historical changes in visual art created by users of a social online network. We propose a number of computational methods for the analysis of temporal development of art images. We then apply these methods to a sample of 270,000 artworks created between 2001 and 2010 by users of the largest social network for art—DeviantArt (www.deviantart.com). We investigate changes in subjects, techniques, sizes, proportions and also selected visual characteristics of images. Because these artworks are classified by their creators into two general categories—Traditional Art and Digital Art—we are also able to investigate if the use of digital tools has had a significant effect on the content and form of artworks. Our analysis reveals a number of gradual and systematic changes over a ten-year period in artworks belonging to both categories.

The exceptional and the everyday: 144 Hours in Kiev

How can we use computational analysis and visualization of content and interactions on social media network to write histories? Traditionally, historical timelines of social and political upheavals give us only distant views of the events, and singular interpretation of a person constructing the timeline. However, using social media as our source, we can potentially present many thousands of individual views of the events. We can also include representation of the everyday life next to the accounts of the exceptional events. This paper explores these ideas using a particular case study - images shared by people in Kiev on Instagram during 2014 Ukranian Revolution. Using Instagram public API we collected 13208 geo-coded images shared by 6165 Instagram users in the central part of Kiev during February 17-22, 2014. We used open source and our own custom software tools to analyze the images along with upload dates and times, geo locations, and tags, and visualize them in different ways.

An Unsupervised Deep Learning Approach for Satellite Image Analysis with Applications in Demographic Analysis

High resolution satellite imagery is a growing source of data with potential applications in many diverse domains. Efficient large scale analysis of this rich data can lead to unprecedented discoveries with societal impact. We present a new framework for organizing collections of satellite images into demographically relevant categories using unsupervised learning techniques. Our framework first extracts features using pre-trained Convolutional Neural Networks from tiles of high resolution satellite images of a city. The k-means algorithm is then applied to these features to organize images into visually similar groups. The resulting clustered images are validated using demographic data. The cluster model is then applied to six different cities around the world to test the transferability of our methods. Finally, the discovered image clusters are visualized in our customized web interface to enable demographers, social scientists, and economists to understand the organization of a city.

Using pairwise difference features to measure temporal changes in the microbial ecology

The exponential affordability of DNA sequencing technologies has enabled not only the accessible and rapid sequencing of the human genome, but the opportunity to sequence vast numbers of multi-cellular organism and the tiniest microbes. Studying the microbes in the human body has led us to the view that the human body is not a single organism but rather a symbiotic ecology of microbes and human cells. The microbiome is referred to the collection of microorganisms in our body (consisting mostly of bacteria, archaea, and some eukarya) and outnumber our human cells ten to one. Numerous studies suggest that the microbiome may play a critical role in a number of autoimmune diseases, with most research on inflammatory bowel disease. The microbiome ecology can be thought of as a composition of the relative abundance of a large number of organisms that reside in our bodies. Numerous studies suggest that disruptions in the relative abundance of a healthy microbial ecology leads to a disease state. The challenge remains, however, in understanding how and what changes in the composition of the ecology leads to a disease state.