Neal Lathia

Temporal diversity in recommender systems

Collaborative Filtering (CF) algorithms, used to build web-based recommender systems, are often evaluated in terms of how accurately they predict user ratings. However, current evaluation techniques disregard the fact that users continue to rate items over time: the temporal characteristics of the systems top-N recommendations are not investigated. In particular, there is no means of measuring the extent that the same items are being recommended to users over and over again. In this work, we show that temporal diversity is an important facet of recommender systems, by showing how CF data changes over time and performing a user survey. We then evaluate three CF algorithms from the point of view of the diversity in the sequence of recommendation lists they produce over time. We examine how a number of characteristics of user rating patterns (including profile size and time between rating) affect diversity. We then propose and evaluate set methods that maximise temporal recommendation diversity without extensively penalising accuracy.

Recommending Social Events from Mobile Phone Location Data

A city offers thousands of social events a day, and it is difficult for dwellers to make choices. The combination of mobile phones and recommender systems can change the way one deals with such abundance. Mobile phones with positioning technology are now widely available, making it easy for people to broadcast their whereabouts, recommender systems can now identify patterns in people’s movements in order to, for example, recommend events. To do so, the system relies on having mobile users who share their attendance at a large number of social events: cold-start users, who have no location history, cannot receive recommendations. We set out to address the mobile cold-start problem by answering the following research question: how can social events be recommended to a cold-start user based only on his home location? To answer this question, we carry out a study of the relationship between preferences for social events and geography, the first of its kind in a large metropolitan area. We sample location estimations of one million mobile phone users in Greater Boston, combine the sample with social events in the same area, and infer the social events attended by 2,519 residents. Upon this data, we test a variety of algorithms for recommending social events. We find that the most effective algorithm recommends events that are popular among residents of an area. The least effective, instead, recommends events that are geographically close to the area. This last result has interesting implications for location-based services that emphasize recommending nearby events.

The wisdom of the few: a collaborative filtering approach based on expert opinions from the web

Nearest-neighbor collaborative filtering provides a successful means of generating recommendations for web users. However, this approach suffers from several shortcomings, including data sparsity and noise, the cold-start problem, and scalability. In this work, we present a novel method for recommending items to users based on expert opinions. Our method is a variation of traditional collaborative filtering: rather than applying a nearest neighbor algorithm to the user-rating data, predictions are computed using a set of expert neighbors from an independent dataset, whose opinions are weighted according to their similarity to the user. This method promises to address some of the weaknesses in traditional collaborative filtering, while maintaining comparable accuracy. We validate our approach by predicting a subset of the Netflix data set. We use ratings crawled from a web portal of expert reviews, measuring results both in terms of prediction accuracy and recommendation list precision. Finally, we explore the ability of our method to generate useful recommendations, by reporting the results of a user-study where users prefer the recommendations generated by our approach.

Trust based collaborative filtering

k-nearest neighbour (kNN) collaborative filtering (CF), the widely successful algorithm supporting recommender systems, attempts to relieve the problem of information overload by generating predicted ratings for items users have not expressed their opinions about; to do so, each predicted rating is computed based on ratings given by like-minded individuals. Like-mindedness, or similarity-based recommendation, is the cause of a variety of problems that plague recommender systems. An alternative view of the problem, based on trust, offers the potential to address many of the previous limiations in CF. In this work we present a varation of kNN, the trusted k-nearest recommenders (or kNR) algorithm, which allows users to learn who and how much to trust one another by evaluating the utility of the rating information they receive. This method redefines the way CF is performed, and while avoiding some of the pitfalls that similarity-based CF is prone to, outperforms the basic similarity-based methods in terms of prediction accuracy.

Mining mobility data to minimise travellers' spending on public transport

As the public transport infrastructure of large cities expands, transport operators are diversifying the range and prices of tickets that can be purchased for travel. However, selecting the best fare for each individual travellers needs is a complex process that is left almost completely unaided. By examining the relation between urban mobility and fare purchasing habits in large datasets from London, Englands public transport network, we estimate that travellers in the city cumulatively spend, per year, up to approximately GBP 200 million more than they need to, as a result of purchasing the incorrect fares. We propose to address these incorrect purchases by leveraging the huge volumes of data that travellers create as they move about the city, by providing, to each of them, personalised ticket recommendations based on their estimated future travel patterns. In this work, we explore the viability of building a fare-recommendation system for public transport networks by (a) formalising the problem as two separate prediction problems and (b) evaluating a number of algorithms that aim to match travellers to the best fare. We find that applying data mining techniques to public transport data has the potential to provide travellers with substantial savings.

kNN CF: a temporal social network

Recommender systems, based on collaborative filtering, draw their strength from techniques that manipulate a set of user-rating profiles in order to compute predicted ratings of unrated items. There are a wide range of techniques that can be applied to this problem; however, the k-nearest neighbour (kNN) algorithm has become the dominant method used in this context. Much research to date has focused on improving the performance of this algorithm, without considering the properties that emerge from manipulating the user data in this way. In order to understand the effect of kNN on a user-rating dataset, the algorithm can be viewed as a process that generates a graph, where nodes are users and edges connect similar users: the algorithm generates an implicit social network amongst the system subscribers. Temporal updates of the recommender system will impose changes on the graph. In this work we analyse user-user kNN graphs from a temporal perspective, retrieving characteristics such as dataset growth, the evolution of similarity between pairs of users, the volatility of user neighbourhoods over time, and emergent properties of the entire graph as the algorithm parameters change. These insights explain why certain kNN parameters and similarity measures outperform others, and show that there is a surprising degree of structural similarity between these graphs and explicit user social networks.

Mining Public Transport Usage for Personalised Intelligent Transport Systems

Traveller information, route planning, and service updates have become essential components of public transport systems: they help people navigate built environments by providing access to information regarding delays and service disruptions. However, one aspect that these systems lack is a way of tailoring the information they offer in order to provide personalised trip time estimates and relevant notifications to each traveller. Mining each user’s travel history, collected by automated ticketing systems, has the potential to address this gap. In this work, we analyse one such dataset of travel history on the London underground. We then propose and evaluate methods to (a) predict personalised trip times for the system users and (b) rank stations based on future mobility patterns, in order to identify the subset of stations that are of greatest interest to the user and thus provide useful travel updates.

The effect of correlation coefficients on communities of recommenders

Recommendation systems, based on collaborative filtering, offer a means of sifting through the enourmous amounts of content on the web by composing user ratings in order to generate predicted ratings for other users. These kinds of systems can be viewed as a network of interacting peers, where each user is a node and the links to all other nodes are weighted according to how similar the corresponding users are. Predicted ratings are generated for a user for unknown items by requesting and aggregating rating information from the surrounding neighbors. However, the different methods of computing user similarity, or weighting the network links, very often do not agree with each other, and, as a result, the structure of the network of recommenders changes completely. In this work we perform an analysis of a range of similarity measures, comparing their performance in terms of prediction accuracy and coverage. This allows us to understand the effect that similarity measures have on predicted ratings. Based on the obtained results, we argue that user-similarity may not sufficiently capture the relationships that recommenders could otherwise share in order to maximise the utility of these communities.

Temporal defenses for robust recommendations

Recommender systems are vulnerable to attack: malicious users may deploy a set of sybils (pseudonymous, automated entities) to inject ratings in order to damage or modify the output of Collaborative Filtering (CF) algorithms. To protect against these attacks, previous work focuses on designing sybil profile classification algorithms, whose aim is to find and isolate sybils. These methods, however, assume that the full sybil profiles have already been input to the system. Deployed recommender systems, on the other hand, operate over time, and recommendations may be damaged while sybils are still injecting their profiles, rather than only after all malicious ratings have been input. Furthermore, system administrators do not know when their system is under attack, and thus when to run these classification techniques, thus risking to leave their recommender system vulnerable to attacks. In this work, we address the problem of temporal sybil attacks, and propose and evaluate methods for monitoring global, user and item behaviour over time, in order to detect rating anomalies that reflect an ongoing attack.

A mindfulness-based intervention to increase resilience to stress in university students (the Mindful Student Study): a pragmatic randomised controlled trial

Summary Background The rising number of young people going to university has led to concerns about an increasing demand for student mental health services. We aimed to assess whether provision of mindfulness courses to university students would improve their resilience to stress. Methods We did this pragmatic randomised controlled trial at the University of Cambridge, UK. Students aged 18 years or older with no severe mental illness or crisis (self-assessed) were randomly assigned (1:1), via remote survey software using computer-generated random numbers, to receive either an 8 week mindfulness course adapted for university students (Mindfulness Skills for Students [MSS]) plus mental health support as usual, or mental health support as usual alone. Participants and the study management team were aware of group allocation, but allocation was concealed from the researchers, outcome assessors, and study statistician. The primary outcome was self-reported psychological distress during the examination period, as measured with the Clinical Outcomes in Routine Evaluation Outcome Measure (CORE–OM), with higher scores indicating more distress. The primary analysis was by intention to treat. This trial is registered with the Australia and New Zealand Clinical Trials Registry, number ACTRN12615001160527. Findings Between Sept 28, 2015, and Jan 15, 2016, we randomly assigned 616 students to the MSS group (n=309) or the support as usual group (n=307). 453 (74%) participants completed the CORE–OM during the examination period and 182 (59%) MSS participants completed at least half of the course. MSS reduced distress scores during the examination period compared with support as usual, with mean CORE–OM scores of 0·87 (SD 0·50) in 237 MSS participants versus 1·11 (0·57) in 216 support as usual participants (adjusted mean difference −0·14, 95% CI −0·22 to −0·06; p=0·001), showing a moderate effect size (β −0·44, 95% CI −0·60 to −0·29; p<0·0001). 123 (57%) of 214 participants in the support as usual group had distress scores above an accepted clinical threshold compared with 88 (37%) of 235 participants in the MSS group. On average, six students (95% CI four to ten) needed to be offered the MSS course to prevent one from experiencing clinical levels of distress. No participants had adverse reactions related to self-harm, suicidality, or harm to others. Interpretation Our findings show that provision of mindfulness training could be an effective component of a wider student mental health strategy. Further comparative effectiveness research with inclusion of controls for non-specific effects is needed to define a range of additional, effective interventions to increase resilience to stress in university students. Funding University of Cambridge and National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care East of England.