Krishnan Ramanathan

Clustering short texts using wikipedia

Subscribers to the popular news or blog feeds (RSS/Atom) often face the problem of information overload as these feed sources usually deliver large number of items periodically. One solution to this problem could be clustering similar items in the feed reader to make the information more manageable for a user. Clustering items at the feed reader end is a challenging task as usually only a small part of the actual article is received through the feed. In this paper, we propose a method of improving the accuracy of clustering short texts by enriching their representation with additional features from Wikipedia. Empirical results indicate that this enriched representation of text items can substantially improve the clustering accuracy when compared to the conventional bag of words representation.

Text summarization using Wikipedia

Abstract Automatic text summarization has been an active field of research for many years. Several approaches have been proposed, ranging from simple position and word-frequency methods, to learning and graph based algorithms. The advent of human-generated knowledge bases like Wikipedia offer a further possibility in text summarization – they can be used to understand the input text in terms of salient concepts from the knowledge base. In this paper, we study a novel approach that leverages Wikipedia in conjunction with graph-based ranking. Our approach is to first construct a bipartite sentence–concept graph, and then rank the input sentences using iterative updates on this graph. We consider several models for the bipartite graph, and derive convergence properties under each model. Then, we take up personalized and query-focused summarization, where the sentence ranks additionally depend on user interests and queries, respectively. Finally, we present a Wikipedia-based multi-document summarization algorithm. An important feature of the proposed algorithms is that they enable real-time incremental summarization – users can first view an initial summary, and then request additional content if interested. We evaluate the performance of our proposed summarizer using the ROUGE metric, and the results show that leveraging Wikipedia can significantly improve summary quality. We also present results from a user study, which suggests that using incremental summarization can help in better understanding news articles.

Creating User Profiles Using Wikipedia

Creating user profiles is an important step in personalization. Many methods for user profile creation have been developed to date using different representations such as term vectors and concepts from an ontology like DMOZ. In this paper, we propose and evaluate different methods for creating user profiles using Wikipedia as the representation. The key idea in our approach is to map documents to Wikipedia concepts at different levels of resolution: words, key phrases, sentences, paragraphs, the document summary and the entire document itself. We suggest a method for evaluating profile recall by pooling the relevant results from the different methods and evaluate our results for both precision and recall. We also suggest a novel method for profile evaluation by assessing the recall over a known ontological profile drawn from DMOZ.

Document Summarization using Wikipedia

Although most of the developing world is likely to first access the Internet through mobile phones, mobile devices are constrained by screen space, bandwidth and limited attention span. Single document summarization techniques have the potential to simplify information consumption on mobile phones by presenting only the most relevant information contained in the document. In this paper we present a language independent single-document summarization method. We map document sentences to semantic concepts in Wikipedia and select sentences for the summary based on the frequency of the mapped-to concepts. Our evaluation on English documents using the ROUGE package indicates our summarization method is competitive with the state of the art in single document summarization.

Collaborative filtering on skewed datasets

Many real life datasets have skewed distributions of events when the probability of observing few events far exceeds the others. In this paper, we observed that in skewed datasets the state of the art collaborative filtering methods perform worse than a simple probabilistic model. Our test bench includes a real ad click stream dataset which is naturally skewed. The same conclusion is obtained even from the popular movie rating dataset when we pose a binary prediction problem of whether a user will give maximum rating to a movie or not.

Personalized video: leanback online video consumption

Current user interfaces for online video consumption are mostly browser based, lean forward, require constant interaction and provide a fragmented view of the total content available. For easier consumption, the user interface and interactions need to be redesigned for less interruptive and lean back experience. In this paper, we describe Personalized Video, an application that converts the online video experience into a personalized lean back experience. It has been implemented on the Windows platform and integrated with intuitive user interactions like gesture and face recognition. It also supports group personalization for concurrent users.

STAIR: A System for Topical and Aggregated Information Retrieval

Web content has exploded dramatically in the last decade and search is becoming increasingly com plex. In the current search paradigm, the user has to enter the query and is immediately presented results that are typically accessed sequentially. However, there are scenarios where the above model is not appropriate, either because results being in consumable form is more important than immediacy of results, or because the it is difficult and time consuming to navigate the results in sequential fashion. In this work, we describe the architecture, implementation and utility of STAIR- The System for Topical and Aggregated Information Retrieval, that uses a variant of focused crawling and retrieves just the relevant information from the web. We present a new interface that selects search results from different search engines, ranks the results and presents the most relevant results as an aggregated PDF document. User studies indicate that the relevance of the results produced by our approach is competitive with those of current search engines