Mehdi Elahi

Active learning strategies for rating elicitation in collaborative filtering: A system-wide perspective

The accuracy of collaborative-filtering recommender systems largely depends on three factors: the quality of the rating prediction algorithm, and the quantity and quality of available ratings. While research in the field of recommender systems often concentrates on improving prediction algorithms, even the best algorithms will fail if they are fed poor-quality data during training, that is, garbage in, garbage out. Active learning aims to remedy this problem by focusing on obtaining better-quality data that more aptly reflects a users preferences. However, traditional evaluation of active learning strategies has two major flaws, which have significant negative ramifications on accurately evaluating the systems performance (prediction error, precision, and quantity of elicited ratings). (1) Performance has been evaluated for each user independently (ignoring system-wide improvements). (2) Active learning strategies have been evaluated in isolation from unsolicited user ratings (natural acquisition). In this article we show that an elicited rating has effects across the system, so a typical user-centric evaluation which ignores any changes of rating prediction of other users also ignores these cumulative effects, which may be more influential on the performance of the system as a whole (system centric). We propose a new evaluation methodology and use it to evaluate some novel and state-of-the-art rating elicitation strategies. We found that the system-wide effectiveness of a rating elicitation strategy depends on the stage of the rating elicitation process, and on the evaluation measures (MAE, NDCG, and Precision). In particular, we show that using some common user-centric strategies may actually degrade the overall performance of a system. Finally, we show that the performance of many common active learning strategies changes significantly when evaluated concurrently with the natural acquisition of ratings in recommender systems.

Personality-Based Active Learning for Collaborative Filtering Recommender Systems

Recommender systems (RSs) suffer from the cold-start or new user/item problem, i.e., the impossibility to provide a new user with accurate recommendations or to recommend new items. Active learning (AL) addresses this problem by actively selecting items to be presented to the user in order to acquire her ratings and hence improve the output of the RS. In this paper, we propose a novel AL approach that exploits the users personality - using the Five Factor Model (FFM) - in order to identify the items that the user is requested to rate. We have evaluated our approach in a user study by integrating it into a mobile, context-aware RS that provides users with recommendations for places of interest (POIs). We show that the proposed AL approach significantly increases the number of ratings acquired from the user and the recommendation accuracy.

Context-Aware Points of Interest Suggestion with Dynamic Weather Data Management

Weather plays an important role in tourists’ decision-making and, for instance, some places or activities must not be even suggested under dangerous weather conditions. In this paper we present a context-aware recommender system, named STS, that computes recommendations suited for the weather conditions at the recommended places of interest (POI) by exploiting a novel model-based context-aware recommendation technique. In a live user study we have compared the performance of the system with a variant that does not exploit weather data when generating recommendations. The results of our experiment have shown that the proposed approach obtains a higher perceived recommendation quality and choice satisfaction.

Content-Based Video Recommendation System Based on Stylistic Visual Features

This paper investigates the use of automatically extracted visual features of videos in the context of recommender systems and brings some novel contributions in the domain of video recommendations. We propose a new content-based recommender system that encompasses a technique to automatically analyze video contents and to extract a set of representative stylistic features (lighting, color, and motion) grounded on existing approaches of Applied Media Theory. The evaluation of the proposed recommendations, assessed w.r.t. relevance metrics (e.g., recall) and compared with existing content-based recommender systems that exploit explicit features such as movie genre, shows that our technique leads to more accurate recommendations. Our proposed technique achieves better results not only when visual features are extracted from full-length videos, but also when the feature extraction technique operates on movie trailers, pinpointing that our approach is effective also when full-length videos are not available or when there are performance requirements. Our recommender can be used in combination with more traditional content-based recommendation techniques that exploit explicit content features associated to video files, to improve the accuracy of recommendations. Our recommender can also be used alone, to address the problem originated from video files that have no meta-data, a typical situation of popular movie-sharing websites (e.g., YouTube) where every day hundred millions of hours of videos are uploaded by users and may contain no associated information. As they lack explicit content, these items cannot be considered for recommendation purposes by conventional content-based techniques even when they could be relevant for the user.

Rating Elicitation Strategies for Collaborative Filtering

The accuracy of collaborative filtering recommender systems largely depends on two factors: the quality of the recommendation algorithm and the nature of the available item ratings. In general, the more ratings are elicited from the users, the more effective the recommendations are. However, not all the ratings are equally useful and therefore, in order to minimize the users’ rating effort, only some of them should be requested or acquired. In this paper we consider several rating elicitation strategies and we evaluate their system utility, i.e., how the overall behavior of the system changes when these new ratings are added. We simulate the limited knowledge of users, i.e., not all the rating requests of the system are satisfied by the users, and we compare the capability of the considered strategies in requesting ratings for items that the user experienced. We show that different strategies can improve different aspects of the recommendation quality with respect to several metrics (MAE, precision, ranking quality and coverage) and we introduce a voting-based strategy that can achieve an excellent overall performance.

Usability Assessment of a Context-Aware and Personality-Based Mobile Recommender System

In this paper we present STS (South Tyrol Suggests), a context-aware mobile recommender system for places of interest (POIs) that integrates some innovative components, including: a personality questionnaire, i.e., a brief and entertaining questionnaire used by the system to learn users’ personality; an active learning module that acquires ratings-in-context for POIs that users are likely to have experienced; and a matrix factorization based recommendation module that leverages the personality information and several contextual factors in order to generate more relevant recommendations.

Active Learning in Collaborative Filtering Recommender Systems

In Collaborative Filtering Recommender Systems user’s preferences are expressed in terms of rated items and each rating allows to improve system prediction accuracy. However, not all of the ratings bring the same amount of information about the user’s tastes. Active Learning aims at identifying rating data that better reflects users’ preferences. Active learning Strategies are used to selectively choose the items to present to the user in order to acquire her ratings and ultimately improve the recommendation accuracy. In this survey article, we review recent active learning techniques for collaborative filtering along two dimensions: (a) whether the system requested ratings are personalised or not, and, (b) whether active learning is guided by one criterion (heuristic) or multiple criteria.

Context Dependent Preference Acquisition with Personality-Based Active Learning in Mobile Recommender Systems

Nowadays, Recommender Systems (RSs) play a key role in many businesses. They provide consumers with relevant recommendations, e.g., Places of Interest (POIs) to a tourist, based on user preference data, mainly in the form of ratings for items. The accuracy of recommendations largely depends on the quality and quantity of the ratings (preferences) provided by the users. However, users often tend to rate no or only few items, causing low accuracy of the recommendation. Active Learning (AL) addresses this problem by actively selecting items to be presented to the user in order to acquire a larger number of high-quality ratings (preferences), and hence, improve the recommendation accuracy. In this paper, we propose a personalized active learning approach that leverages user’s personality data to get more and better in-context ratings. We have designed a novel human computer interaction and assessed our proposed approach in a live user study - which is not common in active learning research. The main result is that the system is able to collect better ratings and provide more relevant recommendations compared to a variant that is using a state of the art approach to preference acquisition.

Adapting to natural rating acquisition with combined active learning strategies

The accuracy of collaborative-filtering recommender systems largely depends on the quantity and quality of the ratings added to the system over time. Active learning (AL) aims to improve the quality of ratings by selectively finding and soliciting the most informative ratings. However previous AL techniques have been evaluated assuming a rather artificial scenario: where AL is the only source of rating acquisition. However, users do frequently rate items on their own, without being prompted by the AL algorithms (natural acquisition). In this paper we show that different AL strategies work better under different conditions, and adding naturally acquired ratings changes these conditions and may result in a decreased effectiveness for some of them. While we are unable to control the naturally occurring changes in conditions, we should adaptively select the AL strategies which are well suited for the conditions at hand. We show that choosing AL strategies adaptively outperforms any of the individual AL strategies.

Recommending Movies Based on Mise-en-Scene Design

In this paper, we present an ongoing work that will ultimately result in a movie recommender system based on the Mise-en-Scène characteristics of the movies. We believe that the preferences of users on movies can be well described in terms of the mise-en-scène, i.e., the design aspects of movie making influencing aesthetic and style. Examples of mise-en-scène characteristics are lighting, colors, background, and movements. Our recommender system opens new opportunities in the design of new user interfaces able to offer a personalized way to search for interesting movies through the analysis of film styles rather than using the traditional classifications of movies based on explicit attributes such as genre and cast.