Yannis Papanikolaou

Subset Labeled LDA: A Topic Model for Extreme Multi-label Classification

Labeled Latent Dirichlet Allocation (LLDA) is an extension of the standard unsupervised Latent Dirichlet Allocation (LDA) algorithm, to address multi-label learning tasks. Previous work has shown it to perform en par with other state-of-the-art multi-label methods. Nonetheless, with increasing number of labels LLDA encounters scalability issues. In this work, we introduce Subset LLDA, a topic model that extends the standard LLDA algorithm, that not only can efficiently scale up to problems with hundreds of thousands of labels but also improves over the LLDA state-of-the-art in terms of prediction accuracy. We conduct experiments on eight data sets, with labels ranging from hundreds to hundreds of thousands, comparing our proposed algorithm with the other LLDA algorithms (Prior–LDA, Dep–LDA), as well as the state-of-the-art in extreme multi-label classification. The results show a steady advantage of our method over the other LLDA algorithms and competitive results compared to the extreme multi-label classification algorithms.

Hierarchical partitioning of the output space in multi-label data

Abstract Hierarchy Of Multi-label classifiERs (HOMER) is a multi-label learning algorithm that breaks the initial learning task to several, easier sub-tasks by first constructing a hierarchy of labels from a given label set and secondly employing a given base multi-label classifier (MLC) to the resulting sub-problems. The primary goal is to effectively address class imbalance and scalability issues that often arise in real-world multi-label classification problems. In this work, we present the general setup for a HOMER model and a simple extension of the algorithm that is suited for MLCs that output rankings. Furthermore, we provide a detailed analysis of the properties of the algorithm, both from an aspect of effectiveness and computational complexity. A secondary contribution involves the presentation of a balanced variant of the k means algorithm, which serves in the first step of the label hierarchy construction. We conduct extensive experiments on six real-world data sets, studying empirically HOMERs parameters and providing examples of instantiations of the algorithm with different clustering approaches and MLCs, The empirical results demonstrate a significant improvement over the given base MLC.

Denoising Autoencoder Self-Organizing Map (DASOM)

In this report, we address the question of combining nonlinearities of neurons into networks for modeling increasingly varying and progressively more complex functions. A fundamental approach is the use of higher-level representations devised by restricted Boltzmann machines and (denoising) autoencoders. We present the Denoising Autoencoder Self-Organizing Map (DASOM) that integrates the latter into a hierarchically organized hybrid model where the front-end component is a grid of topologically ordered neurons. The approach is to interpose a layer of hidden representations between the input space and the neural lattice of the self-organizing map. In so doing the parameters are adjusted by the proposed unsupervised learning algorithm. The model therefore maintains the clustering properties of its predecessor, whereas by extending and enhancing its visualization capacity enables an inclusion and an analysis of the intermediate representation space. A comprehensive series of experiments comprising optical recognition of text and images, and cancer type clustering and categorization is used to demonstrate DASOMs efficiency, performance and projection capabilities.

Large-scale online semantic indexing of biomedical articles via an ensemble of multi-label classification models

BackgroundIn this paper we present the approach that we employed to deal with large scale multi-label semantic indexing of biomedical papers. This work was mainly implemented within the context of the BioASQ challenge (2013–2017), a challenge concerned with biomedical semantic indexing and question answering.MethodsOur main contribution is a MUlti-Label Ensemble method (MULE) that incorporates a McNemar statistical significance test in order to validate the combination of the constituent machine learning algorithms. Some secondary contributions include a study on the temporal aspects of the BioASQ corpus (observations apply also to the BioASQ’s super-set, the PubMed articles collection) and the proper parametrization of the algorithms used to deal with this challenging classification task.ResultsThe ensemble method that we developed is compared to other approaches in experimental scenarios with subsets of the BioASQ corpus giving positive results. In our participation in the BioASQ challenge we obtained the first place in 2013 and the second place in the four following years, steadily outperforming MTI, the indexing system of the National Library of Medicine (NLM).ConclusionsThe results of our experimental comparisons, suggest that employing a statistical significance test to validate the ensemble method’s choices, is the optimal approach for ensembling multi-label classifiers, especially in contexts with many rare labels.