Anupriya Gogna

Matrix completion incorporating auxiliary information for recommender system design

Abstract Rating prediction accuracy of latent factor analysis based techniques in collaborative filtering is limited by the sparsity of available ratings. Usually more than 90% of the missing ratings need to be predicted from less than 10% of available ratings. The problem is highly under-determined. In this work, we propose to improve the prediction accuracy by exploiting the user’s demographic information. We propose a new formulation to incorporate this information into the matrix completion framework of latent factor based collaborative filtering. The ensuing problem is efficiently solved using the split Bregman technique. Experimental evaluation indicates that the use of additional information indeed improves the accuracy of rating prediction. We also compared our proposed approach with an existing technique that incorporates auxiliary information using a graph-Laplacian framework and one utilizing neighborhood based approach; we find that our proposed method yields considerably superior results.

Split Bregman algorithms for sparse / joint-sparse and low-rank signal recovery: Application in compressive hyperspectral imaging

In this work we derive algorithms for solving two problems - the first one is the combined l1-norm (sparsity) and nuclear norm (low rank) regularized least squares problem and the second one is the l2, 1-norm (joint sparsity) and nuclear norm regularized least squares problem. There are no efficient general purpose solvers for these problems; our work plugs this gap by deriving Split Bregman based algorithms for solving the said problems. Both algorithms are applicable for recovering hyperspectral images from their compressive measurements obtained via the single pixel camera. We show that our proposed techniques significantly outperform previous methods in terms of recovery accuracy.

A Low-Rank Matrix Recovery Approach for Energy Efficient EEG Acquisition for a Wireless Body Area Network

We address the problem of acquiring and transmitting EEG signals in Wireless Body Area Networks (WBAN) in an energy efficient fashion. In WBANs, the energy is consumed by three operations: sensing (sampling), processing and transmission. Previous studies only addressed the problem of reducing the transmission energy. For the first time, in this work, we propose a technique to reduce sensing and processing energy as well: this is achieved by randomly under-sampling the EEG signal. We depart from previous Compressed Sensing based approaches and formulate signal recovery (from under-sampled measurements) as a matrix completion problem. A new algorithm to solve the matrix completion problem is derived here. We test our proposed method and find that the reconstruction accuracy of our method is significantly better than state-of-the-art techniques; and we achieve this while saving sensing, processing and transmission energy. Simple power analysis shows that our proposed methodology consumes considerably less power compared to previous CS based techniques.

A Comprehensive Recommender System Model: Improving Accuracy for Both Warm and Cold Start Users

Sparsity of the ratings available in the recommender system database makes the task of rating prediction a highly underdetermined problem. This poses a limit on the accuracy and the quality of prediction. In this paper, we utilize secondary information pertaining to users demography and item categories to enhance prediction accuracy. Within the matrix factorization framework, we introduce additional supervised label consistency terms that match the user and item factor matrices to the available secondary information (metadata). Matrix factorization model-conventionally employed in collaborative filtering techniques-yields dense user and dense item factor matrices-the assumption is that users have an affinity toward all latent factors and items possess all latent factors. Our formulation, based on a recent work, aims to recover a dense user and a sparse item factor matrix-this is a more reasonable model. Human beings show a natural interest toward all the factors, but every item cannot possess all the factors; this leads to a sparse item factor matrix. A natural outcome of our proposal is a solution to the pure cold start problem. We utilize the label consistency map generated from the proposed model to make reasonable recommendations for new users and new items which have not (been) rated yet. We demonstrate the performance of our model for a movie recommendation system. We also design an efficient algorithm for our formulation.

Semi-supervised Stacked Label Consistent Autoencoder for Reconstruction and Analysis of Biomedical Signals

Objective: An autoencoder-based framework that simultaneously reconstruct and classify biomedical signals is proposed. Previous work has treated reconstruction and classification as separate problems. This is the first study that proposes a combined framework to address the issue in a holistic fashion. Methods: For telemonitoring purposes, reconstruction techniques of biomedical signals are largely based on compressed sensing (CS); these are “designed” techniques where the reconstruction formulation is based on some “assumption” regarding the signal. In this study, we propose a new paradigm for reconstruction—the reconstruction is “learned,” using an autoencoder; it does not require any assumption regarding the signal as long as there is sufficiently large training data. But since the final goal is to analyze/classify the signal, the system can also learn a linear classification map that is added inside the autoencoder. The ensuing optimization problem is solved using the Split Bregman technique. Results: Experiments were carried out on reconstructing and classifying electrocardiogram (ECG) (arrhythmia classification) and EEG (seizure classification) signals. Conclusion: Our proposed tool is capable of operating in a semi-supervised fashion. We show that our proposed method is better in reconstruction and more than an order magnitude faster than CS based methods; it is capable of real-time operation. Our method also yields better results than recently proposed classification methods. Significance: This is the first study offering an alternative to CS-based reconstruction. It also shows that the representation learning approach can yield better results than traditional methods that use hand-crafted features for signal analysis.

Matrix Recovery Using Split Bregman

In this paper we address the problem of recovering a matrix, with inherent low rank structure, from its lower dimensional projections. This problem is frequently encountered in wide range of areas including pattern recognition, wireless sensor networks, control systems, recommender systems, image/video reconstruction etc. Both in theory and practice, the most optimal way to solve the low rank matrix recovery problem is via nuclear norm minimization. In this paper, we propose a Split Bregman algorithm for nuclear norm minimization. The use of Bregman technique improves the convergence speed of our algorithm and gives a higher success rate. Also, the accuracy of reconstruction is much better even for cases where small number of linear measurements are available. Our claim is supported by empirical results obtained using our algorithm and its comparison to other existing methods for matrix recovery. The algorithms are compared on the basis of NMSE, execution time and success rate for varying ranks and sampling ratios.

DiABlO: Optimization based design for improving diversity in recommender system

Abstract Primary task of a recommender system is to improve users experience by recommending relevant and interesting items to the users. To this effect, diversity in item suggestion is as important as the accuracy of recommendations. Existing literature aimed at improving diversity primarily suggests a 2-stage mechanism – an existing CF scheme for rating prediction, followed by a modified ranking strategy. This approach requires heuristic selection of parameters and ranking strategies. Also most works focus on diversity from either the user or systems perspective. In this work, we propose a single stage optimization based solution to achieve high diversity while maintaining requisite levels of accuracy. We propose to incorporate additional diversity enhancing constraints, in the matrix factorization model for collaborative filtering. However, unlike traditional MF scheme generating dense user and item latent factor matrices, our base MF model recovers a dense user and a sparse item latent factor matrix; based on a recent work. The idea is motivated by the fact that although a user will demonstrate some affinity towards all latent factors, an item will never possess all features; thereby yielding a sparse structure. We also propose an algorithm for our formulation. The superiority of our model over existing state of the art techniques is demonstrated by the results of experiments conducted on real world movie database.

Deep Dictionary Learning vs Deep Belief Network vs Stacked Autoencoder: An Empirical Analysis

A recent work introduced the concept of deep dictionary learning. The first level is a dictionary learning stage where the inputs are the training data and the outputs are the dictionary and learned coefficients. In subsequent levels of deep dictionary learning, the learned coefficients from the previous level acts as inputs. This is an unsupervised representation learning technique. In this work we empirically compare and contrast with similar deep representation learning techniques – deep belief network and stacked autoencoder. We delve into two aspects; the first one is the robustness of the learning tool in the presence of noise and the second one is the robustness with respect to variations in the number of training samples. The experiments have been carried out on several benchmark datasets. We find that the deep dictionary learning method is the most robust.

SVD free matrix completion with online bias correction for Recommender systems

In this work we address the problem of design of an efficient Recommender system based on collaborative filtering framework which achieves improved accuracy with reduced computational complexity and shorter run times. This work is based on representing the low rank constraint as the Ky-Fan norm instead of the commonly employed nuclear norm term. Our formulation uses majorization minimization approach to cast the problem as simple least squares. The enhanced efficiency of our algorithm in terms of higher accuracy of recovery and shorter execution times is demonstrated by comparison to existing techniques for matrix completion.

Semi Supervised Autoencoder

Autoencoders are self-supervised learning tools, but are unsupervised in the sense that class information is not required for training; but almost invariably they are used for supervised classification tasks. We propose to learn the autoencoder for a semi-supervised paradigm, i.e. with both labeled and unlabeled samples available. Given labeled and unlabeled data, our proposed autoencoder automatically adjusts --- for unlabeled data it acts as a standard autoencoder unsupervised and for labeled data it additionally learns a linear classifier. We use our proposed semi-supervised autoencoder to greedily construct a stacked architecture. We demonstrate the efficacy our design in terms of both accuracy and run time requirements for the case of image classification. Our model is able to provide high classification accuracy with even simple classification schemes as compared to existing models for deep architectures.