Gábor Takács

Investigation of various matrix factorization methods for large recommender systems

Matrix factorization (MF) based approaches have proven to be efficient for rating-based recommendation systems. In this work, we propose several matrix factorization approaches with improved prediction accuracy. We introduce a novel and fast (semi)-positive MF approach that approximates the features by using positive values for either users or items. We describe a momentum-based MF approach. A transductive version of MF is also introduced, which uses information from test instances (namely the ratings users have given for certain items) to improve prediction accuracy. We describe an incremental variant of MF that efficiently handles new users/ratings, which is crucial in a real-life recommender system. A hybrid MF--neighbor-based method is also discussed that further improves the performance of MF.The proposed methods are evaluated on the Netflix Prize dataset, and we show that they can achieve very favorable Quiz RMSE (best single method: 0.8904, combination: 0.8841) and running time.

Matrix factorization and neighbor based algorithms for the netflix prize problem

Collaborative filtering (CF) approaches proved to be effective for recommender systems in predicting user preferences in item selection using known user ratings of items. This subfield of machine learning has gained a lot of popularity with the Netflix Prize competition started in October 2006. Two major approaches for this problem are matrix factorization (MF) and the neighbor based approach (NB). In this work, we propose various variants of MF and NB that can boost the performance of the usual ensemble based scheme. First, we investigate various regularization scenarios for MF. Second, we introduce two NB methods: one is based on correlation coefficients and the other on linear least squares. At the experimentation part, we show that the proposed approaches compare favorably with existing ones in terms of prediction accuracy and/or required training time. We present results of blending the proposed methods.

Alternating least squares for personalized ranking

Two flavors of the recommendation problem are the explicit and the implicit feedback settings. In the explicit feedback case, users rate items and the user-item preference relationship can be modelled on the basis of the ratings. In the harder but more common implicit feedback case, the system has to infer user preferences from indirect information: presence or absence of events, such as a user viewed an item. One approach for handling implicit feedback is to minimize a ranking objective function instead of the conventional prediction mean squared error. The naive minimization of a ranking objective function is typically expensive. This difficulty is usually overcome by a trade-off: sacrificing the accuracy to some extent for computational efficiency by sampling the objective function. In this paper, we present a computationally effective approach for the direct minimization of a ranking objective function, without sampling. We demonstrate by experiments on the Y!Music and Netflix data sets that the proposed method outperforms other implicit feedback recommenders in many cases in terms of the ErrorRate, ARP and Recall evaluation metrics.

A unified approach of factor models and neighbor based methods for large recommender systems

Matrix factorization (MF) based approaches have proven to be efficient for rating-based recommendation systems. In this paper, we propose a hybrid approach that alloys an improved MF and the so-called NSVD1 approach, resulting in a very accurate factor model. After that, we propose a unification of factor models and neighbor based approaches, which further improves the performance. The approaches are evaluated on the Netflix Prize dataset, and they provide very low RMSE, and favorable running time. Our best solution presented here with Quiz RMSE 0.8851 outperforms all published single methods in the literature.

Applications of the conjugate gradient method for implicit feedback collaborative filtering

The need for solving weighted ridge regression (WRR) problems arises in a number of collaborative filtering (CF) algorithms. Often, there is not enough time to calculate the exact solution of the WRR problem, or it is not required. The conjugate gradient (CG) method is a state-of-the-art approach for the approximate solution of WRR problems. In this paper, we investigate some applications of the CG method for new and existing implicit feedback CF models. We demonstrate through experiments on the Netflix dataset that CG can be an efficient tool for training implicit feedback CF models.

Visualization of movie features in collaborative filtering

In this paper we will describe a modification of the matrix factorization (MF) algorithm which allows visualizing the user and item characteristics. When applying MF for collaborative filtering, we get a model that represents the attributes of users and items by feature vectors. Some elements of these vectors may have understandable meaning for humans but due to the lack of internal connections between the feature vectors, these are difficult to visualize. In this paper we give a detailed description of a MF method enabling better visualization of features by arranging them into a 2D map, where via the calculation of the feature values we try to position features with similar “meaning” close to each other. To achieve this first we define a neighborhood relation on features, then we modify the MF so that we introduce a new term in the error function which penalize the difference between the neighbor features. We show that this modification slightly decrease the accuracy of the model but we get well visualized feature maps. On the feature maps meanings can be associated with regions, and so we can provide an interesting explanation for the user why he/she was recommended the movie. Such plausible explanations may result in that users will better understand how the system works, which can also increase customer loyalty towards the service provider.

Predicting flight arrival times with a multistage model

Airlines are constantly looking for ways to cut flight delays, in order to enhance service quality and reduce operational costs. The goal of the data science contest, GE Flight Quest (https://www.gequest.com/c/flight), was to make flights more efficient by improving the accuracy of arrival time estimates. The data set of the contest was 128 GB in size and contained 252 data columns arranged in 34 tables. This paper presents my solution that won third prize under team name Taki. The solution employs a 6-stage model consisting of successive ridge regressions and gradient boosting machines, built on 56 features constructed from the raw data. The hardware environment used for training and running the model was a 64 core machine with 1 terabyte of memory.

A dynamic programming approach for 4D flight route optimization

This paper describes our solution for the GE Flight Quest 2 (FQ2) challenge, organized by Kaggle. FQ2 aimed at optimizing flight routes so that the overall cost depending on fuel consumption and delay is as low as possible. The contestants could use several data tables as inputs, including aircraft positions and destinations, weather information and other aviation related data. Their task was to produce a flight plan for each flight, given as a list of (latitude, longitude, altitude, airspeed) quadruplets. The cost of the flight plans was evaluated with an open source simulator. Our proposed method produces an initial solution with the Dijkstras algorithm to avoid restricted zones, and then refines it using dynamic programming and local search techniques. We can extensively utilize wind forecasts and significantly divert the planes from the the great circle route if necessary. Moreover, our method tries to set the ascending and descending profiles of the flights to further decrease the cost. Our algorithm achieved second place on the public, and fifth place on the private leaderboard of the contest.

Predictor set optimization for collaborative filtering

One of the most efficient approaches to create a recommender system is collaborative filtering (CF). CF does not require metadata about users and items, but only interactions between users and items (e.g. ratings), therefore it can be applied in many problem domains. Experience shows that for achieving high accuracy, it is worthwhile to use a blended solution, consisting of many predictors. This paper presents an algorithm for constructing a set of CF predictors so that the overall accuracy of the set is high. The algorithm was tested on the Netflix Prize dataset that contains 100 million ratings.