Deep Neural Network Regularization for Feature Selection in Learning-to-Rank

Abstract

Learning-to-rank is an emerging area of research for a wide range of applications. Many algorithms are devised to tackle the problem of learning-to-rank. However, very few existing algorithms deal with deep learning. Previous research depicts that deep learning makes significant improvements in a variety of applications. The proposed model makes use of the deep neural network for learning-to-rank for document retrieval. It employs a regularization technique particularly suited for the deep neural network to improve the results significantly. The main aim of regularization is optimizing the weight of neural network, selecting the relevant features with active neurons at the input layer, and pruning of the network by selecting only active neurons at hidden layer while learning. Specifically, we use group <inline-formula> <tex-math notation= LaTeX >$\\ell _{1}$ </tex-math></inline-formula> regularization in order to induce the group level sparsity on the network’s connections. Set of outgoing weights from each hidden layer represents the group here. The sparsity of network is measured by the sparsity ratio and it is compared with learning-to-rank models, which adopt the embedded method for feature selection. An extensive experimental evaluation considers the performance of the extended <inline-formula> <tex-math notation= LaTeX >$\\ell _{1}$ </tex-math></inline-formula> regularization technique against classical regularization techniques. The empirical results confirm that sparse group <inline-formula> <tex-math notation= LaTeX >$\\ell _{1}$ </tex-math></inline-formula> regularization is able to achieve competitive performance while simultaneously making the network compact with less number of input features. The model is analyzed with respect to evaluating measures, such as prediction accuracy, <italic>NDCG@n</italic>, <italic>MAP</italic>, and <italic>Precision</italic> on benchmark datasets, which demonstrate improved results over other state-of-the-art methods.