Haggai Roitman

Harnessing the crowds for smart city sensing

In this work we discuss the challenge of harnessing the crowd for smart city sensing. Within a citys context, such reports by citizen or city visitor eye witnesses may provide important information to city officials, additionally to more traditional data gathered by other means (e.g., through the citys control center, emergency services, sensors spread across the city, etc). We present an high-level overview of a novel crowd sensing system that we develop in IBM for the smart cities domain. As a proof of concept, we present some preliminary results using public safety as our example usecase.

Extracting user profiles from large scale data

In this work we present the details of a large scale user profiling framework that we developed here in IBM on top of Apache Hadoop. We address the problem of extracting and maintaining a very large number of user profiles from large scale data. We first describe an efficient user profiling framework with high user profiling quality guarantees. We then describe a scalable implementation of the proposed framework in Apache Hadoop and discuss its challenges.

Best-Effort Top-k Query Processing Under Budgetary Constraints

We consider a novel problem of top-k query processing under budget constraints. We provide both a framework and a set of algorithms to address this problem. Existing algorithms for top-k processing are budget-oblivious, i.e., they do not take budget constraints into account when making scheduling decisions, but focus on the performance to compute the final top-k results. Under budget constraints, these algorithms therefore often return results that are a lot worse than the results that can be achieved with a clever, budget-aware scheduling algorithm. This paper introduces novel algorithms for budget-aware top-k processing that produce results that have a significantly higher quality than those of state-of-the-art budget-oblivious solutions.

OntoBuilder: fully automatic extraction and consolidation of ontologies from web sources using sequence semantics

Ontologies, formal specifications of domains, have evolved in recent years as a leading tool in representing and interpreting Web data. The OntoBuilder project supports the extraction of ontologies from Web search interfaces, ranging from simple search engine forms to multiple-pages, complex reservation systems. OntoBuilder enables fully-automatic ontology matching. The use of ontologies, as opposed to relational schema or XML, as an underlying data model allows a flexible representation of metadata, that can be tailored to many different types of applications. OntoBuilder was developed using Java, which makes it portable to various platforms and operating system environments. We demonstrate OntoBuilder using an easy-to-follow example of matching car rental ontologies. The system creates ontologies of car rental Web sites on-the-fly, and combine them into a global ontology. The benefits of OntoBuilder in resolving, in an automatic manner, semantic heterogeneity, including synonyms and designer errors are highlighted.

Social bookmark weighting for search and recommendation

Social bookmarking enables knowledge sharing and efficient discovery on the web, where users can collaborate together by tagging documents of interests. A lot of attention was given lately for utilizing social bookmarking data to enhance traditional IR tasks. Yet, much less attention was given to the problem of estimating the effectiveness of an individual bookmark for the specific tasks. In this work, we propose a novel framework for social bookmark weighting which allows us to estimate the effectiveness of each of the bookmarks individually for several IR tasks. We show that by weighting bookmarks according to their estimated quality, we can significantly improve social search effectiveness. We further demonstrate that using the same framework, we can derive solutions to several recommendation tasks such as tag recommendation, user recommendation, and document recommendation. Empirical evaluation on real data gathered from two large bookmarking systems demonstrates the effectiveness of the new social bookmark weighting framework.

Understanding customer behavior using indoor location analysis and visualization

Understanding customer behavior in brick-and-mortar stores and other physical indoor venues is essential for any business aiming to provide a more personal and compelling shopping experience, optimize store layout, and improve store operations. Achieving these goals ultimately leads to improved user experience, conversion rates, and increased revenue. Todays mobile-based location technologies provide information about the users location that can be used in advanced analytics and visualizations. This means retailers and enterprises can gain insight into customer behavior patterns and understand, for example, how much time customers spend in different areas of the store, what routes they take, how well they are serviced, and more. In this paper, we present a solution approach for better understanding customer behavior based on mobile indoor location data as well as the technologies developed by IBM Research for realizing this solution. We describe significant challenges considering collection, curation, analysis, and visualization of indoor location-based data and illustrate the use of the approach for smarter commerce in a real-world use case.

Towards expressive exploratory search over entity-relationship data

In this paper we describe a novel approach for exploratory search over rich entity-relationship data that utilizes a unique combination of expressive, yet intuitive, query language, faceted search, and graph navigation. We describe an extended faceted search solution which allows to index, search, and browse rich entity-relationship data. We report experimental results of an evaluation study, using a benchmark of several of entity-relationship datasets, demonstrating that our exploratory approach is both effective and efficient compared to other existing approaches.

Folksonomy-Based Term Extraction for Word Cloud Generation

In this work we study the task of term extraction for word cloud generation in sparsely tagged domains, in which manual tags are scarce. We present a folksonomy-based term extraction method, called tag-boost, which boosts terms that are frequently used by the public to tag content. Our experiments with tag-boost based term extraction over different domains demonstrate tremendous improvement in word cloud quality, as reflected by the agreement between manual tags of the testing items and the cloud’s terms extracted from the items’ content. Moreover, our results demonstrate the high robustness of this approach, as compared to alternative cloud generation methods that exhibit a high sensitivity to data sparseness. Additionally, we show that tag-boost can be effectively applied even in nontagged domains, by using an external rich folksonomy borrowed from a well-tagged domain.

Surfacing time-critical insights from social media

We propose to demonstrate an end-to-end framework for leveraging time-sensitive and critical social media information for businesses. More specifically, we focus on identifying, structuring, integrating, and exposing timely insights that are essential to marketing services and monitoring reputation over social media. Our system includes components for information extraction from text, entity resolution and integration, analytics, and a user interface.

Folksonomy-based term extraction for word cloud generation

In this work we study the task of term extraction for word cloud generation. We present a folksonomy-based term extraction method, called tag-boost, which boosts terms that are frequently used by the public to tag content. Our experiments with tag-boost-based term extraction over different domains demonstrate tremendous improvement in word cloud quality, as reflected by the agreement between extracted terms and manually assigned tags of the testing items. Additionally, we show that tag-boost can be effectively applied even in non-tagged domains, by using an external rich folksonomy borrowed from a well-tagged domain.