Lucas Drumond

Learning Attribute-to-Feature Mappings for Cold-Start Recommendations

Cold-start scenarios in recommender systems are situations in which no prior events, like ratings or clicks, are known for certain users or items. To compute predictions in such cases, additional information about users (user attributes, e.g. gender, age, geographical location, occupation) and items (item attributes, e.g. genres, product categories, keywords) must be used. We describe a method that maps such entity (e.g. user or item) attributes to the latent features of a matrix (or higher-dimensional) factorization model. With such mappings, the factors of a MF model trained by standard techniques can be applied to the new-user and the new-item problem, while retaining its advantages, in particular speed and predictive accuracy. We use the mapping concept to construct an attribute-aware matrix factorization model for item recommendation from implicit, positive-only feedback. Experiments on the new-item problem show that this approach provides good predictive accuracy, while the prediction time only grows by a constant factor.

Predicting RDF triples in incomplete knowledge bases with tensor factorization

On RDF datasets, the truth values of triples are known when they are either explicitly stated or can be inferred using logical entailment. Due to the open world semantics of RDF, nothing can be said about the truth values of triples that are neither in the dataset nor can be logically inferred. By estimating the truth values of such triples, one could discover new information from the database thus enabling to broaden the scope of queries to an RDF base that can be answered, support knowledge engineers in maintaining such knowledge bases or recommend users resources worth looking into for instance. In this paper, we present a new approach to predict the truth values of any RDF triple. Our approach uses a 3-dimensional tensor representation of the RDF knowledge base and applies tensor factorization techniques that take open world semantics into account to predict new true triples given already observed ones. We report results of experiments on real world datasets comparing different tensor factorization models. Our empirical results indicate that our approach is highly successful in estimating triple truth values on incomplete RDF datasets.

Extracting ontology concept hierarchies from text using Markov logic

Ontologies have proven to be a powerful tool for many tasks such as natural language processing and information filtering and retrieval. However their development is an error prone and expensive task. One approach for this problem is to provide automatic or semi-automatic support for ontology construction. This work presents the Probabilistic Relational Hierarchy Extraction (PREHE) technique, an approach for extracting concept hierarchies from text that uses statistical relational learning and natural language processing for combining cues from many state-of-the-art techniques. A Markov Logic Network has been developed for this task and is described here. A preliminary evaluation of the proposed approach is also outlined.

Optimizing Multi-Relational Factorization Models for Multiple Target Relations

Multi-matrix factorization models provide a scalable and effective approach for multi-relational learning tasks such as link prediction, Linked Open Data (LOD) mining, recommender systems and social network analysis. Such models are learned by optimizing the sum of the losses on all relations in the data. Early models address the problem where there is only one target relation for which predictions should be made. More recent models address the multi-target variant of the problem and use the same set of parameters to make predictions for all target relations. In this paper, we argue that a model optimized for each target relation individually has better predictive performance than models optimized for a compromise on the performance on all target relations. We introduce specific parameters for each target but, instead of learning them independently from each other, we couple them through a set of shared auxiliary parameters, which has a regularizing effect on the target specific ones. Experiments on large Web datasets derived from DBpedia, Wikipedia and BlogCatalog show the performance improvement obtained by using target specific parameters and that our approach outperforms competitive state-of-the-art methods while being able to scale gracefully to big data.

Hyperparameter Optimization with Factorized Multilayer Perceptrons

In machine learning, hyperparameter optimization is a challenging task that is usually approached by experienced practitioners or in a computationally expensive brute-force manner such as grid-search. Therefore, recent research proposes to use observed hyperparameter performance on already solved problems (i.e. data sets) in order to speed up the search for promising hyperparameter configurations in the sequential model based optimization framework. In this paper, we propose multilayer perceptrons as surrogate models as they are able to model highly nonlinear hyperparameter response surfaces. However, since interactions of hyperparameters, data sets and metafeatures are only implicitly learned in the subsequent layers, we improve the performance of multilayer perceptrons by means of an explicit factorization of the interaction weights and call the resulting model a factorized multilayer perceptron. Additionally, we evaluate different ways of obtaining predictive uncertainty, which is a key ingredient for a decent tradeoff between exploration and exploitation. Our experimental results on two public meta data sets demonstrate the efficiency of our approach compared to a variety of published baselines. For reproduction purposes, we make our data sets and all the program code publicly available on our supplementary webpage.

Architectural design of a multi-agent recommender system for the legal domain

Legal information sources are characterized by their growth and dynamism since new laws are written every day. Recommender systems are used as an approach to the information overload problem. Thus they can help professionals of the legal area to deal with legal information sources. This paper describes the architectural design of Infonorma, a multi-agent recommender system for the legal domain. Infonorma monitors a repository of legal normative instruments and classifies them into legal branches. Each user specifies his/her interests for certain legal branches and receives recommendations of instruments they might be interested in. The information source is entirely written according to Semantic Web standards. Infonorma was developed under the guidelines of MAAEM, a software development methodology for multi-agent application engineering.

Personalized Deep Learning for Tag Recommendation

Social media services deploy tag recommendation systems to facilitate the process of tagging objects which depends on the information of both the user’s preferences and the tagged object. However, most image tag recommender systems do not consider the additional information provided by the uploaded image but rely only on textual information, or make use of simple low-level image features. In this paper, we propose a personalized deep learning approach for the image tag recommendation that considers the user’s preferences, as well as visual information. We employ Convolutional Neural Networks (CNNs), which already provide excellent performance for image classification and recognition, to obtain visual features from images in a supervised way. We provide empirical evidence that features selected in this fashion improve the capability of tag recommender systems, compared to the current state of the art that is using hand-crafted visual features, or is solely based on the tagging history information. The proposed method yields up to at least two percent accuracy improvement in two real world datasets, namely NUS-WIDE and Flickr-PTR.

Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons

Recent work has demonstrated that hyperparameter optimization within the sequential model-based optimization (SMBO) framework is generally possible. This approach replaces the expensive-to-evaluate function that maps hyperparameters to the performance of a learned model on validation data by a surrogate model which is much cheaper to evaluate. The current state of the art in hyperparameter optimization learns these surrogate models across a variety of solved data sets where a grid search has already been employed. In this way, surrogate models are learned across data sets, and thus able to generalize better. However, meta features that describe characteristics of a data set are usually needed in order for the surrogate model to differentiate between same hyperparameter configurations on different data sets. Another research area that is closely related focuses on model choice, i.e. picking the right model for a given task, which is also a problem that many practitioners face in machine learning. In this paper, we aim to solve both of these problems with a unified surrogate model that learns across different data sets, different classifiers and their respective hyperparameters. We employ factorized multilayer perceptrons, a surrogate model that consists of a multilayer perceptron architecture, but offers the prediction of a factorization machine in the first layer. In this way, data sets, models and hyperparameters are being represented in a joint lower dimensional latent feature space. Experiments on a publicly available meta data set containing 59 individual data sets and 19 prediction models demonstrate the efficiency of our approach.

Collective Matrix Factorization of Predictors, Neighborhood and Targets for Semi-supervised Classification

Due to the small size of available labeled data for semi-supervised learning, approaches to this problem make strong assumptions about the data, performing well only when such assumptions hold true. However, a lot of effort may have to be spent in understanding the data so that the most suitable model can be applied. This process can be as critical as gathering labeled data. One way to overcome this hindrance is to control the contribution of different assumptions to the model, rendering it capable of performing reasonably in a wide range of applications. In this paper we propose a collective matrix factorization model that simultaneously decomposes the predictor, neighborhood and target matrices (PNT-CMF) to achieve semi-supervised classification. By controlling how strongly the model relies on different assumptions, PNT-CMF is able to perform well on a wider variety of datasets. Experiments on synthetic and real world datasets show that, while state-of-the-art models (TSVM and LapSVM) excel on datasets that match their characteristics and have a performance drop on the others, our approach outperforms them being consistently competitive in different situations.

An Effective Approach for Geolocation Prediction in Twitter Streams Using Clustering Based Discretization

Micro-blogging services, such as Twitter, have provided an indispensable channel to communicate, access, and exchange current affairs. Understanding the dynamics of users behavior and their geographical location is key to providing services such as event detection, geo-aware recommendation and local search. The geographical location prediction problem we address is to predict the geolocation of a user based on textual tweets. In this paper, we develop a clustering based discretization approach which is an effective combination of three well-known machine learning algorithms, e.g. K-means clustering, support vector machines, and K-nearest neighbor, to tackle the task of geolocation prediction in Twitter streams. Our empirical results indicate that our approach outperforms previous attempts on a publicly available dataset and that it achieves state-of-the-art performance. Nghia Duong-Trung University of Hildesheim, Universitätsplatz 1, duongn@ismll.uni-hildesheim.de Nicolas Schilling University of Hildesheim, Universitätsplatz 1, schilling@ismll.uni-hildesheim.de Lucas Rego Drumond University of Hildesheim, Universitätsplatz 1, ldrumond@ismll.uni-hildesheim.de Lars Schmidt-Thieme University of Hildesheim, Universitätsplatz 1, schmidt-thieme@ismll.uni-hildesheim.de A R C H I V E S O F D ATA S C I E N C E ( O N L I N E F I R S T ) DOI 10.5445/KSP/1000058749/13 K I T S C I E N T I F I C P U B L I S H I N G ISSN 2363-9881 Vol. 2, No. 1, 2017 2 Duong-Trung et al.