Mikel Elkano

Enhancing Multiclass Classification in FARC-HD Fuzzy Classifier: On the Synergy Between

There are many real-world classification problems involving multiple classes, e.g., in bioinformatics, computer vision, or medicine. These problems are generally more difficult than their binary counterparts. In this scenario, decomposition strategies usually improve the performance of classifiers. Hence, in this paper, we aim to improve the behavior of fuzzy association rule-based classification model for high-dimensional problems (FARC-HD) fuzzy classifier in multiclass classification problems using decomposition strategies, and more specifically One-versus-One (OVO) and One-versus-All (OVA) strategies. However, when these strategies are applied on FARC-HD, a problem emerges due to the low-confidence values provided by the fuzzy reasoning method. This undesirable condition comes from the application of the product t-norm when computing the matching and association degrees, obtaining low values, which are also dependent on the number of antecedents of the fuzzy rules. As a result, robust aggregation strategies in OVO, such as the weighted voting obtain poor results with this fuzzy classifier. In order to solve these problems, we propose to adapt the inference system of FARC-HD replacing the product t-norm with overlap functions. To do so, we define n-dimensional overlap functions. The usage of these new functions allows one to obtain more adequate outputs from the base classifiers for the subsequent aggregation in OVO and OVA schemes. Furthermore, we propose a new aggregation strategy for OVO to deal with the problem of the weighted voting derived from the inappropriate confidences provided by FARC-HD for this aggregation method. The quality of our new approach is analyzed using 20 datasets and the conclusions are supported by a proper statistical analysis. In order to check the usefulness of our proposal, we carry out a comparison against some of the state-of-the-art fuzzy classifiers. Experimental results show the competitiveness of our method.

n

We study the influence of the usage of n-dimensional overlap functions to model the conjunction in Fuzzy Rule Based Classification Systems (FRBCSs).We analyze the behavior of these functions when using both decomposition strategies and baseline classifiers.We consider four well-known FRBCSs (CHI, SLAVE, FURIA, and FARC-HD) and One-vs-All (OVA) and One-vs-One (OVO) strategies.The benefits obtained from overlap functions strongly depend on the learning process and the rule structure of each algorithm. Multi-class classification problems appear in a broad variety of real-world problems, e.g., medicine, genomics, bioinformatics, or computer vision. In this context, decomposition strategies are useful to increase the classification performance of classifiers. For this reason, in a previous work we proposed to improve the performance of FARC-HD (Fuzzy Association Rule-based Classification model for High-Dimensional problems) fuzzy classifier using One-vs-One (OVO) and One-vs-All (OVA) decomposition strategies. As a result of an exhaustive experimental analysis, we concluded that even though the usage of decomposition strategies was worth to be considered, further improvements could be achieved by introducing n-dimensional overlap functions instead of the product t-norm in the Fuzzy Reasoning Method (FRM). In this way, we can improve confidences for the subsequent processing performed in both OVO and OVA.In this paper, we want to conduct a broader study of the influence of the usage of n-dimensional overlap functions to model the conjunction in several Fuzzy Rule-Based Classification Systems (FRBCSs) in order to enhance their performance in multi-class classification problems applying decomposition techniques. To do so, we adapt the FRM of four well-known FRBCSs (CHI, SLAVE, FURIA, and FARC-HD itself). We will show that the benefits of the usage of n-dimensional overlap functions strongly depend on both the learning algorithm and the rule structure of each classifier, which explains why FARC-HD is the most suitable one for the usage of these functions.

-Dimensional Overlap Functions and Decomposition Strategies

This paper introduces the concept of Choquet-like Copula-based aggregation function (CC-integral) and its application in fuzzy rule-based classification systems. The standard Choquet integral is expanded by distributing the product operation. Then, the product operation is generalized by a copula. Unlike the generalization of the Choquet integral by t-norms using its standard form (i.e., without distributing the product operator), which results in a pre-aggregation function, the CC-integral satisfies all the conditions required for an aggregation function. We build some examples of CC-integrals considering different examples of copulas, including t-norms, overlap functions and copulas that are neither t-norms nor overlap functions. We show that the CC-integral based on the minimum t-norm, when applied in fuzzy rule-based classification systems, obtains a performance that is, with a high level of confidence, better than that which adopts the winning rule (maximum). We concluded that the behavior of CC-integral is similar to the best Choquet-like pre-aggregation function. Consequently, the CC-integrals introduced in this paper can enlarge the scope of the applications by offering new possibilities for defining fuzzy reasoning methods with a similar gain in performance.

Fuzzy Rule-Based Classification Systems for multi-class problems using binary decomposition strategies

We present the concept of admissible order for n-dimensional fuzzy sets.We give a construction method for those admissible orders using aggregation functions.We extent to the field of n-dimensional fuzzy sets the concept of OWA operators (which are always associated to a linear order).We design a decision making algorithm using n-dimensional fuzzy sets and n-tuple aggregation OWA operators.We justify our theoretical developments with an illustrative example applying the proposed algorithm. In this paper we propose an algorithm to solve group decision making problems using n-dimensional fuzzy sets, namely, sets in which the membership degree of each element to the set is given by an increasing tuple of n elements. The use of these sets has naturally led us to define admissible orders for n-dimensional fuzzy sets, to present a construction method for those orders and to study OWA operators for aggregating the tuples used to represent the membership degrees of the elements. In these conditions, we present an algorithm and apply it to a case study, in which we show that the exploitation phase which appears in many decision making methods can be omitted by just considering linear orders between tuples.

CC-integrals

Display Omitted A consensus method via penalty functions for decision making in ensembles of fuzzy rule-based classification systems is introduced.Overlap indices are built using overlap functions.A method for constructing confidence and support measures from overlap indices is presented.A new fuzzy rule mechanism is proposed, considering different overlap indices, which generalizes the classical methods.An example of a generation of fuzzy rule-based ensembles and the decision making by consensus via penalty functions is presented. The aim of this paper is to propose a consensus method via penalty functions for decision making in ensembles of fuzzy rule-based classification systems (FRBCSs). For that, we first introduce a method based on overlap indices for building confidence and support measures, which are usually used to evaluate the degree of certainty or interest of a certain association rule. Those overlap indices (a generalizations of the Zadehs consistency index between two fuzzy sets) are built using overlap functions, which are a special kind of non necessarily associative aggregation functions proposed for applications related to the overlap problem and/or when the associativity property is not demanded. Then, we introduce a new FRM for the FRBCS, considering different overlap indices, which generalizes the classical methods. By considering several overlap indices and aggregation functions, we generate fuzzy rule-based ensembles, providing different results. For the decision making related to the selection of the best class, we introduce a consensus method for classification, based on penalty functions. We also present theoretical results related to the developed methods. A detailed example of a generation of fuzzy rule-based ensembles based on the proposed approach, and the decision making by consensus via penalty functions, is presented.

An algorithm for group decision making using n-dimensional fuzzy sets, admissible orders and OWA operators

We study interval-valued fuzzy relations for the Generalized Modus Ponens.We study the composition of these relations using interval aggregation functions.We study the properties of these compositions.We develop an inference method using interval-valued fuzzy relations.We present an illustrative example. In this paper we present the composition of interval-valued fuzzy relations using interval-valued aggregation functions. In particular, we propose a generalization of Zadehs composition rule, replacing the minimum by an interval-valued aggregation function. We analyze the preservation of different properties of interval-valued fuzzy relations by this new composition, and we include an illustrative example in approximate reasoning in order to justify our proposal.

Consensus via penalty functions for decision making in ensembles in fuzzy rule-based classification systems

In this work we present an optimization of the only Fuzzy Rule-Based Classification System that is able to face Big Data classification problems to date, i.e., Chi-FRBCS-BigDataCS. The aim of this optimization is to speed up the learning process of the algorithm without affecting the model obtained. Our proposal is based on the usage of Look-Up-Tables to pre-compute the membership degrees of the different linguistic labels, avoiding millions of computations. Furthermore, we propose a new data flow that: 1) takes advantage of the sorting and merging phases of MapReduce to group duplicated/conflicting rules and avoids an exhaustive search over the rule base; 2) removes the bottleneck caused by the execution of a single reducer by supporting as many reducers as needed. These adaptations have shown a significant improvement in terms of runtime without altering the model obtained by Chi-FRBCS-BigDataCS.

Composition of interval-valued fuzzy relations using aggregation functions

Classification problems with multiple classes suppose a challenge in Data Mining tasks. There is a difficulty inherent to the learning process when trying to find the most adequate discrimination functions among the different concepts within the dataset. Using Fuzzy Rule Based Classification Systems in general, and Evolutionary Fuzzy Systems in particular, provide the advantage of describing smoother borderline areas, thanks to the linguistic label-based representation.In multi-classification, the pairwise learning approach (One-vs-One) has gained a notorious attention. However, there is certain dependence between the goodness of the confidence degrees or scores of binary classifiers, and the final performance shown by the global model. Regarding this fact, the problem of non-competent classifiers is of special relevance. It occurs when a binary classifier outputs a positive score for a couple of classes unrelated with the input example, which may degrade the final accuracy. Precisely, the previously exposed properties of fuzzy classifiers make them more prone to the former condition.In this paper, we propose an extension of the distance-based combination strategy to overcome this non-competence problem. It is based on the truncation of the confidence degrees of the classes prior to the distance-based tuning. This allows taking advantage of the good classification abilities of Evolutionary Fuzzy Systems, while diminishing the adverse effect of the aforementioned non-competence. Experimental results, using FARC-HD with overlap functions as the fuzzy learning algorithm, show that this new adaptation of the Distance-based Relative Competence Weighting model outperforms both the OVO and standard distance-based approaches, and it is competitive with robust classifiers such as Support Vector Machines. The problem of non-competence for pairwise learning is addressed.A new methodology, based on truncation of the confidence degrees, is proposed.The properties of Fuzzy Rule Based Classification Systems are taken into account in the design of this novel model.A distance-based tuning is carried out to adapt the score-matrix of the One-vs-One procedure.Experimental results versus the state-of-the-art show the goodness of this approach.

Optimization of Chi-FRBCS-BigDataCS Fuzzy Rule-Based Classification System

Decomposition strategies have been shown to be a successful methodology to tackle multi-class classification problems. Among them, One-vs-One approach is a commonly used technique that consists in dividing the original multi-class problem into easier-to-solve binary sub-problems considering each possible pair of classes. However, this methodology is limited to those classifiers returning a single real value for each prediction. In this work, we present a new One-vs-One approach that is able to deal with interval-valued outputs. In order to achieve this goal, we propose applying a normalization method for intervals along with the corresponding extension of three different aggregation strategies: voting, weighted voting, and WinWV. The experimental results show the suitability of the normalization method and the improvement obtained by One-vs-One with respect to a state-of-the-art interval-valued Fuzzy Rule-Based Classification System (IVTURS).

Enhancing evolutionary fuzzy systems for multi-class problems

The scarcity of labeled data has limited the capacity of convolutional neural networks (CNNs) until not long ago and still represents a serious problem in a number of image processing applications. Unsupervised methods have been shown to perform well in feature extraction and clustering tasks, but further investigation on unsupervised solutions for CNNs is needed. In this work, we propose a bio-inspired methodology that applies a deep generative model to help the CNN take advantage of unlabeled data and improve its classification performance. Inspired by the human “sleep-wake cycles”, the proposed method divides the learning process into sleep and waking periods. During the waking period, both the generative model and the CNN learn from real training data simultaneously. When sleep begins, none of the networks receive real data and the generative model creates a synthetic dataset from which the CNN learns. The experimental results showed that the generative model was able to teach the CNN and improve its classification performance.