Doron Cohen

Static index pruning for information retrieval systems

We introduce static index pruning methods that significantly reduce the index size in information retrieval systems.We investigate uniform and term-based methods that each remove selected entries from the index and yet have only a minor effect on retrieval results. In uniform pruning, there is a fixed cutoff threshold, and all index entries whose contribution to relevance scores is bounded above by a given threshold are removed from the index. In term-based pruning, the cutoff threshold is determined for each term, and thus may vary from term to term. We give experimental evidence that for each level of compression, term-based pruning outperforms uniform pruning, under various measures of precision. We present theoretical and experimental evidence that under our term-based pruning scheme, it is possible to prune the index greatly and still get retrieval results that are almost as good as those based on the full index.

Dynamic memory disambiguation for array references

We present a new algorithm for dynamic memory disambiguation for array references that allows us to overcome limitations of static analysis. For array references that cannot be accurately analyzed at compile time, we defer the disambiguation process until run-time. We have implemented our analysis algorithm in a prototype version of the IBM XL compiler and used the generated information for several compiler optimizations: software pipelining with global instruction scheduling, loop-invariant motion and redundant load elimination. We evaluated the algorithm on an IBM POWER2 system using the SPEC92 benchmarks. We show that for numeric C benchmarks, dynamic memory disambiguation can greatly improve run-time performance. Perhaps more importantly, we show that even for the programs that cannot benefit from dynamic analysis, the overhead of our algorithm does not degrade performance.

Understanding customer behavior using indoor location analysis and visualization

Understanding customer behavior in brick-and-mortar stores and other physical indoor venues is essential for any business aiming to provide a more personal and compelling shopping experience, optimize store layout, and improve store operations. Achieving these goals ultimately leads to improved user experience, conversion rates, and increased revenue. Todays mobile-based location technologies provide information about the users location that can be used in advanced analytics and visualizations. This means retailers and enterprises can gain insight into customer behavior patterns and understand, for example, how much time customers spend in different areas of the store, what routes they take, how well they are serviced, and more. In this paper, we present a solution approach for better understanding customer behavior based on mobile indoor location data as well as the technologies developed by IBM Research for realizing this solution. We describe significant challenges considering collection, curation, analysis, and visualization of indoor location-based data and illustrate the use of the approach for smarter commerce in a real-world use case.

Livemaps for collection awareness

With the increasing proliferation of chat applications on the web, the old vision of “adding people” to the web is becoming a reality. Along with collaboration tools, more and more sites offer people awareness mechanisms to let the site visitors know about each other. This reflects the dual nature of the web as a place for virtual meetings as well as an information repository. While standalone chat tools became the killer application of the Internet, site-related awareness applications did not quite catch on. In this work, we suggest possible reasons for this phenomenon and propose a new paradigm for awareness and social navigation. We identify three main obstacles to the existing site-related awareness applications: high sensitivity to the “critical mass” requirement, inflexible meeting place granularity and poor visitor visibility. To address these issues, we extend the well-known “document awareness” concept to a more general one that we call “collection awareness”, which better reflects the graph structure of the web. We introduce a new tool for high-level awareness and collaboration, called Livemaps, which projects live information onto a web site map. We demonstrate how Livemaps addresses the obstacles we pointed out and describe a user study conducted on a “fan” web site for the “Friends” comedy series, so as to verify whether Livemaps actually improves social awareness.

A statistical approach to mining customers' conversational data from social media

In this paper, we present one possible way of analyzing social media conversional data in order to better understand customers. Ultimately, our goal is to analyze customer behavior as it is expressed in free-form conversations and extract from it commercially valuable information about the customer. In this study, we concentrate on using statistical techniques for analyzing this unstructured data at two levels: 1) at the level of the words used in the conversation and 2) by mapping those words to abstract concepts. The goal of such a statistical analysis is twofold. First, the statistically significant terms used by the users and the concepts associated with them provide insight on a users interests that commercial services can use, for example, in order to target advertisements. In addition, knowing the evolution of a customers interests and hobbies can be exploited commercially by retailers, media and entertainment companies, telecommunications companies, and more. In this paper, we describe a general framework for the analysis of social media data and, in turn, the application of the framework to the statistical analysis of the language of tweets.

SoLoMo analytics for telco Big Data monetization

The mobile Internet brought tremendous opportunities for businesses to capitalize on the vast amount of SoLoMo (social-location-mobile) data for delivering high-quality and personalized customer services. In this paper, we describe algorithms and technologies for discovering actionable customer insights using the combined power of social network, location pattern mining, and mobile usage analysis. We illustrate our implementation using Big Data platforms including IBM InfoSphere® BigInsights, IBM InfoSphere Streams, and IBM Netezza® Data Warehouse, while addressing various Big Data-related challenges, such as context generation of unstructured data and high-performance analytics for both data at rest and data in motion. The presented system combines location, social interactions, and user behavior data to find like-minded communities. The system leverages Big Data capabilities to attempt to scale to support the subscriber base of large telecoms in an efficient manner.

Greaaaat bargains starting from just 99p!!!! :-): brand perception in the social media

In this work we discuss the challenges of utilizing social media data, and more specifically microblogs, for helping brand managers. Brand perception is one of the most important tasks of a brand manager, requiring to understand how customers perceive and select brands in specific product categories or market segments. While understanding the brand perception from conventional sources such as reviews and advertisement is well studied and established, gaining insights from social media sources is still an open challenge. In this paper, we present a high-level overview of a novel system that was developed in IBM which aims at extracting brand perception from Twitter. As a proof of concept, we present some preliminary results from the retail domain.

An Extended Relevance Model for Session Search

The session search task aims at best serving the users information need given her previous search behavior during the session. We propose an extended relevance model that captures the users dynamic information need in the session. Our relevance modelling approach is directly driven by the users query reformulation (change) decisions and the estimate of how much the users search behavior affects such decisions. Overall, we demonstrate that, the proposed approach significantly boosts session search performance.

Leveraging Web services for information discovery

We describe a novel application of the Web services model for end-user information discovery needs rather than for the traditional business-to-business applications. We describe a specialization of Web services for information providers and demonstrate, through an exemplary unified information discovery console, how consumers can easily customize their favorite information sources, and obtain information from them in a passive or active but always unobtrusive manner.

The distributed and parallel NIC server

Abstract The NIC-Server is a high-performance, low-overhead experimental system which supports distributed and parallel programming in FORTRAN using asynchronous RPC. The minimal requirements from users are natural to FORTRAN programmers; ANSI C implementation on top of the lower layers of Sun-RPC, and the built-in scheduler, ensure good portability across systems.