Robin Senge

Dependent binary relevance models for multi-label classification

Several meta-learning techniques for multi-label classification (MLC), such as chaining and stacking, have already been proposed in the literature, mostly aimed at improving predictive accuracy through the exploitation of label dependencies. In this paper, we propose another technique of that kind, called dependent binary relevance (DBR) learning. DBR combines properties of both, chaining and stacking. We provide a careful analysis of the relationship between these and other techniques, specifically focusing on the underlying dependency structure and the type of training data used for model construction. Moreover, we offer an extensive empirical evaluation, in which we compare different techniques on MLC benchmark data. Our experiments provide evidence for the good performance of DBR in terms of several evaluation measures that are commonly used in MLC. HighlightsWe propose DBR as a multi-label classifier that exploits conditional label dependence.DBR combines properties of both, chaining and stacking learning strategies.We provide a careful analysis of the relationship between these techniques.We study the underlying dependency structure and the type of training data used.Our experiments show the good performance of DBR in terms of several measures.

Comparing Fuzzy Partitions: A Generalization of the Rand Index and Related Measures

In this paper, we introduce a fuzzy extension of a class of measures to compare clustering structures, namely, measures that are based on the number of concordant and the number of discordant pairs of data points. This class includes the well-known Rand index but also commonly used alternatives, such as the Jaccard measure. In contrast with previous proposals, our extension exhibits desirable metrical properties. Apart from elaborating on formal properties of this kind, we present an experimental study in which we compare different fuzzy extensions of the Rand index and the Jaccard measure.

Top-Down Induction of Fuzzy Pattern Trees

Fuzzy pattern tree induction was recently introduced as a novel machine learning method for classification. Roughly speaking, a pattern tree is a hierarchical, tree-like structure, whose inner nodes are marked with generalized (fuzzy) logical operators and whose leaf nodes are associated with fuzzy predicates on input attributes. A pattern-tree classifier is composed of an ensemble of such pattern trees: one for each class label. This type of classifier is interesting for several reasons. For example, since a single pattern tree can be considered as a kind of logical description of a class, it is quite appealing from an interpretation point of view. Moreover, in terms of classification accuracy, the method has shown promising performance in first experimental studies. Yet, as will be argued in this paper, the algorithm that has originally been proposed for learning fuzzy pattern trees from data offers scope for improvement. Here, we propose a new method that modifies the original proposal in several ways. Notably, our learning algorithm reverses the direction of pattern tree construction from bottom-up to top-down. Additionally, a different termination criterion is proposed that is more adapted to the learning problem at hand. Experimentally, it will be shown that our approach is indeed able to outperform the original learning method in terms of predictive accuracy.

Evolving fuzzy pattern trees for binary classification on data streams

Fuzzy pattern trees (FPTs) have recently been introduced as a novel model class for machine learning. In this paper, we consider the problem of learning fuzzy pattern trees for binary classification from data streams. Apart from its practical relevance, this problem is also interesting from a methodological point of view. First, the aspect of efficiency plays an important role in the context of data streams, since learning has to be accomplished under hard time (and memory) constraints. Moreover, a learning algorithm should be adaptive in the sense that an up-to-date model is offered at any time, taking new data items into consideration as soon as they arrive and perhaps forgetting old ones that have become obsolete due to a change of the underlying data generating process. To meet these requirements, we develop an evolving version of fuzzy pattern tree learning, in which model adaptation is realized by anticipating possible local changes of the current model, and confirming these changes through statistical hypothesis testing. In experimental studies, we compare our method to a state-of-the-art tree-based classifier for learning from data streams, showing that evolving pattern trees are competitive in terms of performance while typically producing smaller and more compact models.

On the Problem of Error Propagation in Classifier Chains for Multi-label Classification

So-called classifier chains have recently been proposed as an appealing method for tackling the multi-label classification task. In this paper, we analyze the influence of a potential pitfall of the learning process, namely the discrepancy between the feature spaces used in training and testing: while true class labels are used as supplementary attributes for training the binary models along the chain, the same models need to rely on estimations of these labels when making a prediction. We provide first experimental results suggesting that the attribute noise thus created can affect the overall prediction performance of a classifier chain.

Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty

A proper representation of the uncertainty involved in a prediction is an important prerequisite for the acceptance of machine learning and decision support technology in safety-critical application domains such as medical diagnosis. Despite the existence of various probabilistic approaches in these fields, there is arguably no method that is able to distinguish between two very different sources of uncertainty: aleatoric uncertainty, which is due to statistical variability and effects that are inherently random, and epistemic uncertainty which is caused by a lack of knowledge. In this paper, we propose a method for binary classification that does not only produce a prediction of the class of a query instance but also a quantification of the two aforementioned sources of uncertainty. Despite being grounded in probability and statistics, the method is formalized within the framework of fuzzy preference relations. The usefulness and reasonableness of our approach is confirmed on a suitable data set with information about patients suffering from chest pain.

Pattern trees for regression and fuzzy systems modeling

Fuzzy pattern tree induction has recently been introduced as a novel classification method in the context of machine learning. Roughly speaking, a pattern tree is a hierarchical, tree-like structure, whose inner nodes are marked with generalized (fuzzy) logical operators and whose leaf nodes are associated with fuzzy predicates on input attributes. In this paper, we adapt the method of pattern tree induction so as to make it applicable to another learning task, namely regression. Thus, instead of predicting one among a finite number of discrete class labels, we address the problem of predicting a real-valued target output. Apart from showing that fuzzy pattern trees are able to approximate real-valued functions in an accurate manner, we argue that such trees are also interesting from a modeling point of view. In fact, by describing a functional relationship between several input attributes and an output variable in an interpretable way, pattern trees constitute a viable alternative to classical fuzzy rule models. Compared to flat rule models, the hierarchical structure of patterns trees further allows for a more compact representation and for trading off accuracy against model simplicity in a seamless manner.

Exploiting HIV-1 protease and reverse transcriptase cross-resistance information for improved drug resistance prediction by means of multi-label classification

BackgroundAntiretroviral therapy is essential for human immunodeficiency virus (HIV) infected patients to inhibit viral replication and therewith to slow progression of disease and prolong a patient’s life. However, the high mutation rate of HIV can lead to a fast adaptation of the virus under drug pressure and thereby to the evolution of resistant variants. In turn, these variants will lead to the failure of antiretroviral treatment. Moreover, these mutations cannot only lead to resistance against single drugs, but also to cross-resistance, i.e., resistance against drugs that have not yet been applied.Methods662 protease sequences and 715 reverse transcriptase sequences with complete resistance profiles were analyzed using machine learning techniques, namely binary relevance classifiers, classifier chains, and ensembles of classifier chains.ResultsIn our study, we applied multi-label classification models incorporating cross-resistance information to predict drug resistance for two of the major drug classes used in antiretroviral therapy for HIV-1, namely protease inhibitors (PIs) and non-nucleoside reverse transcriptase inhibitors (NNRTIs). By means of multi-label learning, namely classifier chains (CCs) and ensembles of classifier chains (ECCs), we were able to improve overall prediction accuracy for all drugs compared to hitherto applied binary classification models.ConclusionsThe development of fast and precise models to predict drug resistance in HIV-1 is highly important to enable a highly effective personalized therapy. Cross-resistance information can be exploited to improve prediction accuracy of computational drug resistance models.

Fast Fuzzy Pattern Tree Learning for Classification

Fuzzy pattern trees have recently been introduced as a novel type of fuzzy system, specifically with regard to the modeling of classification functions in machine learning. Moreover, different algorithms for learning pattern trees from data have been proposed in the literature. While showing strong performance in terms of predictive accuracy, these algorithms exhibit a rather high computational complexity, and their runtime may become prohibitive for large datasets. In this paper, we therefore propose extensions of an existing state-of-the-art algorithm for fuzzy pattern tree induction, which are aimed at making this algorithm faster without compromising its predictive accuracy. These extensions include the use of adaptive sampling schemes, as well as heuristics for guiding the growth of pattern trees. The effectiveness of our modified algorithm is confirmed by means of several experimental studies.

Fuzzy pattern trees as an alternative to rule-based fuzzy systems: Knowledge-driven, data-driven and hybrid modeling of color yield in polyester dyeing

This paper advocates a novel approach to fuzzy systems modeling called fuzzy pattern trees. This approach is largely motivated by alleged disadvantages of rule-based system architectures that still dominate the field. Due to its hierarchical, modular structure and the use of different types of (nonlinear) aggregation operators, a fuzzy pattern tree has the ability to represent functional dependencies in a more flexible and more compact way, thereby offering a reasonable balance between accuracy and model transparency. We evaluate this new model class in the context of a concrete case study, namely the modeling of color yield in polyester high temperature dyeing as a function of disperse dyes concentration, temperature and time. To this end, we compare three possibilities for model construction: purely knowledge-driven, purely data-driven and a hybrid approach combining these two. Our results show that, in comparison to conventional fuzzy modeling using Mamdani rules, fuzzy pattern trees are not only more accurate but also more compact and therefore more easily interpretable, regardless of whether the models are constructed in a knowledge-driven, data-driven or hybrid manner. Moreover, we show that a hybrid modeling approach can outperform a purely data-driven and a purely knowledge-driven approach if expert knowledge and model calibration are combined in a suitable way.