Benjamin Markines

Evaluating similarity measures for emergent semantics of social tagging

Social bookmarking systems are becoming increasingly important data sources for bootstrapping and maintaining Semantic Web applications. Their emergent information structures have become known as folksonomies. A key question for harvesting semantics from these systems is how to extend and adapt traditional notions of similarity to folksonomies, and which measures are best suited for applications such as community detection, navigation support, semantic search, user profiling and ontology learning. Here we build an evaluation framework to compare various general folksonomy-based similarity measures, which are derived from several established information-theoretic, statistical, and practical measures. Our framework deals generally and symmetrically with users, tags, and resources. For evaluation purposes we focus on similarity between tags and between resources and consider different methods to aggregate annotations across users. After comparing the ability of several tag similarity measures to predict user-created tag relations, we provide an external grounding by user-validated semantic proxies based on WordNet and the Open Directory Project. We also investigate the issue of scalability. We find that mutual information with distributional micro-aggregation across users yields the highest accuracy, but is not scalable; per-user projection with collaborative aggregation provides the best scalable approach via incremental computations. The results are consistent across resource and tag similarity.

Friendship prediction and homophily in social media

Social media have attracted considerable attention because their open-ended nature allows users to create lightweight semantic scaffolding to organize and share content. To date, the interplay of the social and topical components of social media has been only partially explored. Here, we study the presence of homophily in three systems that combine tagging social media with online social networks. We find a substantial level of topical similarity among users who are close to each other in the social network. We introduce a null model that preserves user activity while removing local correlations, allowing us to disentangle the actual local similarity between users from statistical effects due to the assortative mixing of user activity and centrality in the social network. This analysis suggests that users with similar interests are more likely to be friends, and therefore topical similarity measures among users based solely on their annotation metadata should be predictive of social links. We test this hypothesis on several datasets, confirming that social networks constructed from topical similarity capture actual friendship accurately. When combined with topological features, topical similarity achieves a link prediction accuracy of about 92%.

Folks in Folksonomies: social link prediction from shared metadata

Web 2.0 applications have attracted a considerable amount of attention because their open-ended nature allows users to create lightweight semantic scaffolding to organize and share content. To date, the interplay of the social and semantic components of social media has been only partially explored. Here we focus on Flickr and Last.fm, two social media systems in which we can relate the tagging activity of the users with an explicit representation of their social network. We show that a substantial level of local lexical and topical alignment is observable among users who lie close to each other in the social network. We introduce a null model that preserves user activity while removing local correlations, allowing us to disentangle the actual local alignment between users from statistical effects due to the assortative mixing of user activity and centrality in the social network. This analysis suggests that users with similar topical interests are more likely to be friends, and therefore semantic similarity measures among users based solely on their annotation metadata should be predictive of social links. We test this hypothesis on the Last.fm data set, confirming that the social network constructed from semantic similarity captures actual friendship more accurately than Last.fms suggestions based on listening patterns.

Social spam detection

The popularity of social bookmarking sites has made them prime targets for spammers. Many of these systems require an administrators time and energy to manually filter or remove spam. Here we discuss the motivations of social spam, and present a study of automatic detection of spammers in a social tagging system. We identify and analyze six distinct features that address various properties of social spam, finding that each of these features provides for a helpful signal to discriminate spammers from legitimate users. These features are then used in various machine learning algorithms for classification, achieving over 98% accuracy in detecting social spammers with 2% false positives. These promising results provide a new baseline for future efforts on social spam. We make our dataset publicly available to the research community.

GiveALink: mining a semantic network of bookmarks for web search and recommendation

GiveALink is a public site where users donate their bookmarks to the Web community. Bookmarks are analyzed to build a new generation of Web mining techniques and new ways to search, recommend, surf, personalize and visualize the Web. We present a semantic similarity measure for URLs that takes advantage both of the hierarchical structure of the bookmark files of individual users, and of collaborative filtering across users. We analyze the social bookmark network induced by the similarity measure. A search and recommendation system is built from a number of ranking algorithms based on prestige, generality, and novelty measures extracted from the similarity data.

A scalable, collaborative similarity measure for social annotation systems

Collaborative annotation tools are in widespread use. The metadata from these systems can be mined to induce semantic relationships among Web objects (sites, pages, tags, concepts, users), which in turn can support improved search, recommendation, and otherWeb applications. We build upon prior work by extracting relationships among tags and among resources from two social bookmarking systems, Bibsonomy.org and GiveALink.org. We introduce a scalable and collaborative measure that we name maximum information path (MIP) similarity. Our analysis shows that MIP outperforms the best scalable similarity measures in the literature. We are currently integrating MIP similarity into a number of applications under development in the GiveALink project, including search and recommendation, Web navigation maps, bookmark management, social networks, spam detection, and a tagging game to create incentives for collaborative annotations.

Efficient assembly of social semantic networks

Social bookmarks allow Web users to actively annotate individual Web resources. Researchers are exploring the use of these annotations to create implicit links between online resources. We define an implicit link as a relationship between two online resources established by the Web community. An individual may create or reinforce a relationship between two resources by applying a common tag or organizing them in a common folder. This has led to the exploration of techniques for building networks of resources, categories, and people using the social annotations. In order for these techniques to move from the lab to the real world, efficient building and maintenance of these potentially large networks remains a major obstacle. Methods for assembling and indexing these large networks will allow researchers to run more rigorous assessments of their proposed techniques. Toward this goal we explore an approach from the sparse matrix literature and apply it to our system, GiveALink.org. We also investigate distributing the assembly, allowing us to grow the network with the body of resources, annotations, and users. Dividing the network is effective for assembling a global network where the implicit links are dependent on global properties. Additionally, we explore alternative implicit link measures that remove global dependencies and thus allow for the global network to be assembled incrementally, as each participant makes independent contributions. Finally we evaluate three scalable similarity measures, two of which require a revision of the data model underlying our social annotations.

Incentives for social annotation

Researchers are exploring the use of folksonomies, such as in social bookmarking systems, to build implicit links between online resources. Users create and reinforce links between resources through applying a common tag to those resources. The effectiveness of using such community-driven annotation depends on user participation to provide the critical information. However, the participation of many users is motivated by selfish reasons. An effective way to encourage these users is to create useful or entertaining applications. We demo two such tools -- a browser extension for bookmark management and navigation and a game.Researchers are exploring the use of folksonomies, such as in social bookmarking systems, to build implicit links between online resources. Users create and reinforce links between resources through applying a common tag to those resources. The effectiveness of using such community-driven annotation depends on user participation to provide the critical information. However, the participation of many users is motivated by selfish reasons. An effective way to encourage these users is to create useful or entertaining applications. We demo two such tools -- a browser extension for bookmark management and navigation and a game.

Visualizing social links in exploratory search

The visualization of results is a critical component in search engines, and the standard ranked list interface has been a consistently predominant model. The emergence of social media provides a new opportunity to investigate visualization techniques that expose socially derived links between objects to support their exploration. Here we introduce and evaluate network-based visualizations for facilitating the exploration of a Web knowledge space. We developed a force directed network interface to visualize the result sets provided by GiveALink.org, a social bookmarking site. The classifications and tags by users are aggregated to build a social similarity network between bookmarked resources. We administered a user study to evaluate the potential of leveraging such social links in an exploratory search task. During exploration, the similarity links are used to arrange the resources in a semantic layout. Users in our study prefer a hybrid interface combining a conventional ranked list and a two dimensional network map, allowing them to find the same amount of relevant information using fewer queries. This behavior is a direct result of the additional structural information present in the network visualization, which aids them in the exploration of the information space.

“Socially induced semantic networks and applications” by Benjamin Markines

Social bookmarking systems allow Web users to actively annotate online resources. These annotations incorporate meta-information with Web pages in addition to the actual document contents. From a collection of socially annotated resources, we present various methods for quantifying the relationship between objects, i.e., tags or resources. These relationships can then be represented in a semantic similarity network where the nodes represent objects and the undirected weighted edges represent their relations. These relations are quantied through similarity measures. There are two challenges associated with assembling and maintaining such a similarity network. The first challenge is updating the relations efficiently, i.e., the time and space complexity associated with graph algorithms. The complexity of these algorithms is typically quadratic. We present an incremental process answering both space and time limitations. The second challenge is the quality of the similarity measure. We evaluate various measures through the approximation of reference similarities. We then present a number of applications leveraging socially induced semantic similarity networks. A tag recommendation system, a page recommendation engine, and a Web navigation tool are evaluated through user studies. Finally, we design spam detection algorithms to enhance the functionality of social bookmarking systems.