Tien T. Nguyen

Rating support interfaces to improve user experience and recommender accuracy

One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.

User Session Identification Based on Strong Regularities in Inter-activity Time

Session identification is a common strategy used to develop metrics for web analytics and perform behavioral analyses of user-facing systems. Past work has argued that session identification strategies based on an inactivity threshold is inherently arbitrary or has advocated that thresholds be set at about 30 minutes. In this work, we demonstrate a strong regularity in the temporal rhythms of user initiated events across several different domains of online activity (incl. video gaming, search, page views and volunteer contributions). We describe a methodology for identifying clusters of user activity and argue that the regularity with which these activity clusters appear implies a good rule-of-thumb inactivity threshold of about 1 hour. We conclude with implications that these temporal rhythms may have for system design based on our observations and theories of goal-directed human activity.

Early Activity Diversity: Assessing Newcomer Retention from First-Session Activity

Online communities suffer serious newcomer attrition. This paper explores whether and how early activity diversity - the degree to which a newcomer engages in a wide range of a sites activities in the first session - is associated with their longevity. We introduce a metric (DSCORE) to characterize early activity diversity in online sites and run our analyses on an online community ‘MovieLens’. We find that DSCORE is significant both by itself and in conjunction with a measure of quantity of activity in predicting longevity. This finding is robust to different measures of longevity (aggregate number of sessions and attritions after sessions 1, 5, and 10). The immediate implication is an effective classifier for identifying users with higher (or lower) expected longevity from the first-session activity. We also find DSCORE is more useful than a traditional measure of measuring diversity such as the Gini-Simpson index. We conclude by discussing how early activity diversity may be more broadly effective in supporting design and management of online communities.

Predicting Users' Preference from Tag Relevance

Tagging has become a powerful means for users to find, organize, understand and express their ideas about online entities. However, tags present great challenges when researchers try to incorporate them into the prediction task of recommender systems. In this paper, we propose a novel approach to infer user preference from tag relevance, an indication of how strong each tag applies to each item in recommender systems. We also present a methodology to choose tags that tell most about each user’s preference. Our preliminary results show that at certain levels, some of our algorithms perform better than previous work.

Exploring the Value of Personality in Predicting Rating Behaviors: A Study of Category Preferences on MovieLens

Prior work relevant to incorporating personality into recommender systems falls into two categories: social science studies and algorithmic ones. Social science studies of preference have found only small relationships between personality and category preferences, whereas, algorithmic approaches found a little improvement when incorporating personality into recommendations. As a result, despite good reasons to believe personality assessments should be useful in recommenders, we are left with no substantial demonstrated impact. In this work, we start with user data from a live recommender system, but study category-by-category variations in preference (both rating levels and distribution) across different personality types. By doing this, we hope to isolate specific areas where personality is most likely to provide value in recommender systems, while also modeling an analytic process that can be used in other domains. After controlling for the family-wise error rate, we find that High Agreeableness users rate at least 0.5 stars higher on a 5-star scale compared to low Agreeableness users. We also find differences in consumption in four different personality types between people who manifested high and low levels of that personality.

User Personality and User Satisfaction with Recommender Systems

In this study, we show that individual users’ preferences for the level of diversity, popularity, and serendipity in recommendation lists cannot be inferred from their ratings alone. We demonstrate that we can extract strong signals about individual preferences for recommendation diversity, popularity and serendipity by measuring their personality traits. We conducted an online experiment with over 1,800 users for six months on a live recommendation system. In this experiment, we asked users to evaluate a list of movie recommendations with different levels of diversity, popularity, and serendipity. Then, we assessed users’ personality traits using the Ten-item Personality Inventory (TIPI). We found that ratings-based recommender systems may often fail to deliver preferred levels of diversity, popularity, and serendipity for their users (e.g. users with high-serendipity preferences). We also found that users with different personalities have different preferences for these three recommendation properties. Our work suggests that we can improve user satisfaction when we integrate users’ personality traits into the process of generating recommendations.

Personality, User Preferences and Behavior in Recommender systems

This paper reports on a study of 1840 users of the MovieLens recommender system with identified Big-5 personality types. Based on prior literature that suggests that personality type is a stable predictor of user preferences and behavior, we examine factors of user retention and engagement, content preferences, and rating patterns to identify recommender-system related behaviors and preferences that correlate with user personality. We find that personality traits correlate significantly with behaviors and preferences such as newcomer retention, intensity of engagement, activity types, item categories, consumption versus contribution, and rating patterns.

Improving recommender systems: user roles and lifecycles

In the era of big data, it is usually agreed that the more data we have, the better results we can get. However, for some domains that heavily depend on user inputs (such as recommender systems), the performance evaluation metrics are sensitive to the amount of noise introduced by users. Such noise can be from users who only wanted to explore the systems, and thus did not spend efforts to provide accurate inputs. Noise can also be introduced by the methods of collecting user ratings. In my dissertation, I study how user data can affect prediction accuracies and performances of recommendation algorithms. To that end, I investigate how the data collection methods and the life cycles of users affect the prediction accuracies and the performance of recommendation algorithms.