Eric C. C. Tsang

Building a Rule-Based Classifier—A Fuzzy-Rough Set Approach

The fuzzy-rough set (FRS) methodology, as a useful tool to handle discernibility and fuzziness, has been widely studied. Some researchers studied on the rough approximation of fuzzy sets, while some others focused on studying one application of FRS: attribute reduction (i.e., feature selection). However, constructing classifier by using FRS, as another application of FRS, has been less studied. In this paper, we build a rule-based classifier by using one generalized FRS model after proposing a new concept named as ¿consistence degree¿ which is used as the critical value to keep the discernibility information invariant in the processing of rule induction. First, we generalized the existing FRS to a robust model with respect to misclassification and perturbation by incorporating one controlled threshold into knowledge representation of FRS. Second, we propose a concept named as ¿consistence degree¿ and by the strict mathematical reasoning, we show that this concept is reasonable as a critical value to reduce redundant attribute values in database. By employing this concept, we then design a discernibility vector to develop the algorithms of rule induction. The induced rule set can function as a classifier. Finally, the experimental results show that the proposed rule-based classifier is feasible and effective on noisy data.

Fuzzy support vector machines for solving two-class problems

A support vector machine (SVM) was originally developed to solve two-class non-fuzzy problems. An SVM can act as a linear learning machine when handling data in a high dimensional feature space for non-linear separable and non-separable problems. A few methods have been proposed to solve two-class and multi-class classification problems by including fuzzy concepts. In this paper, we propose a new fuzzy support vector machine which improves the traditional SVM by adding fuzzy memberships to each training sample to indicate degree of membership of this sample to different classes. This fuzzy SVM is more complete and meaningful, and could generalize the traditional non-fuzzy SVM to a fuzzy one, i.e., the traditional non-fuzzy SVM is an extreme case of our fuzzy SVM when the degrees of membership of a sample to two different classes are the same.

Nested structure in parameterized rough reduction

Abstract In this paper, by strict mathematical reasoning, we discover the relationship between the parameters and the reducts in parameterized rough reduction. This relationship, named the nested reduction, shows that the reducts act as a nested structure with the monotonically increasing parameter. We present a systematic theoretical framework that provides some basic principles for constructing the nested structure in parameterized rough reduction. Some specific parameterized rough set models in which the nested reduction can be constructed are pointed out by strict mathematical reasoning. Based on the nested reduction, we design several quick algorithms to find a different reduct when one reduct is already given. Here ‘different’ refers to the reducts obtained on the different parameters. All these algorithms are helpful for quickly finding a proper reduct in the parameterized rough set models. The numerical experiments demonstrate the feasibility and the effectiveness of the nested reduction approach.

Multi-label learning with label-specific feature reduction

We propose two multi-label learning approaches with LIFT reduction.The idea of fuzzy rough set attribute reduction is adopted in our approaches.Sample selection improves the efficiency in feature dimension reduction. In multi-label learning, since different labels may have some distinct characteristics of their own, multi-label learning approach with label-specific features named LIFT has been proposed. However, the construction of label-specific features may encounter the increasing of feature dimensionalities and a large amount of redundant information exists in feature space. To alleviate this problem, a multi-label learning approach FRS-LIFT is proposed, which can implement label-specific feature reduction with fuzzy rough set. Furthermore, with the idea of sample selection, another multi-label learning approach FRS-SS-LIFT is also presented, which effectively reduces the computational complexity in label-specific feature reduction. Experimental results on 10 real-world multi-label data sets show that, our methods can not only reduce the dimensionality of label-specific features when compared with LIFT, but also achieve satisfactory performance among some popular multi-label learning approaches.

Double-quantitative rough fuzzy set based decisions

As two important expanded quantification rough set models, the probabilistic rough set (PRS) model and the graded rough set (GRS) model are used to measure relative quantitative information and absolute quantitative information between the equivalence classes and a basic concept, respectively. The decision-theoretic rough set (DTRS) model is a special case of PRS model which mainly utilizes the conditional probability to express relative quantification. Since the fuzzy concept is more general than classical concept in real life, how to make decision for a fuzzy concept using relative and absolute quantitative information is becoming a hot topic. In this paper, a couple of double-quantitative decision-theoretic rough fuzzy set (Dq-DTRFS) models based on logical conjunction and logical disjunction operation are proposed. Furthermore, we discuss decision rules and the inner relationship between these two models. Then, an experiment in the medical diagnosis is studied to support the theories. Finally, to apply our methods to solve a pattern recognition problem in big data, experiments on data sets downloaded from UCI are conducted to test the proposed models. In addition, we also offer a comparative analysis using two non-rough set based methods. From the results obtained, one finds that the proposed method is efficient for dealing with practical issues.

Feature and instance reduction for PNN classifiers based on fuzzy rough sets

Abstract Instance reduction for K-nearest-neighbor classification rules (KNN) has attracted much attention these years, and most of the existing approaches lose the semantics of probability of original data. In this work, we propose a new reduced KNN rule, called FAIR-KNN, to perform feature and instance reduction based on fuzzy rough set theory. First, we use fuzzy rough sets to evaluate candidate features and select the most informative ones. The algorithm of feature selection returns the selected features and the membership values of samples to the lower approximations of their classes. These values reflect the distances of the samples to classification boundary and are used to compute probabilities of samples to be subsampled. Then we introduce a weighted Parzen window technique to estimate the probability from the weighted subsampled data. Thus we can not only reduce features and samples in original data, but also do not lose the semantics of probability. Finally, the memberships of samples to lower and upper approximations of decisions are interpreted as certainty and possibility degrees of samples belonging to the corresponding decisions, respectively. So the weighted averages with probability of the memberships of samples to lower and upper approximations are outputted as the certainty and possibility degrees of unseen samples belonging to some decisions, which enrich the semantics of KNN. Numerical experiments on artificial and real-world data validate the effectiveness of the proposed technique.

Fuzzy rough set based incremental attribute reduction from dynamic data with sample arriving

Abstract Attribute reduction with fuzzy rough set is an effective technique for selecting most informative attributes from a given real-valued dataset. However, existing algorithms for attribute reduction with fuzzy rough set have to re-compute a reduct from dynamic data with sample arriving where one sample or multiple samples arrive successively. This is clearly uneconomical from a computational point of view. In order to efficiently find a reduct from such datasets, this paper studies incremental attribute reduction with fuzzy rough sets. At the arrival of one sample or multiple samples, the relative discernibility relation is updated for each attribute. On the basis of the updated relation, an insight into the incremental process of attribute reduction with fuzzy rough sets is gained to reveal how to add new attributes into the current reduct and delete existing attributes from the current reduct. Applying the incremental process, two incremental algorithms for attribute reduction with fuzzy rough sets are presented for one incoming sample and multiple incoming samples, respectively. Experimental comparisons with several non-incremental algorithms and the proposed incremental algorithm for one incoming sample show that our proposed incremental algorithm for multiple incoming samples can efficiently find one reduct with a comparable classification accuracy.

Optimal scale selection in dynamic multi-scale decision tables based on sequential three-way decisions

Abstract It has been recognized that optimal scale selection in rough set theory is one of the most important problems in the study of multi-scale decision tables. Recently, much attention has been paid to this issue and quite a few appealing results have been obtained. However, the existing results are not applicable to the situation where the objects or attributes in a multi-scale decision table are sequentially updated, although this situation is frequently encountered in many real-world problems. Motivated by the fact that sequential three-way decisions are an effective mathematical tool in dealing with the data with information sequentially updated, we therefore use this methodology to investigate the optimal scale selection problem in a dynamic multi-scale decision table. Specifically, a sequential three-way decision model is first developed in multi-scale information tables, which can be viewed as multi-granularity of the universe of discourse. Then, this model is employed to present an optimal scale selection approach for such multi-scale decision tables that the number of objects is increasing. Finally, numerical experiments are conducted to evaluate the performance of the proposed optimal scale selection approach. Compared to the existing methods, the current approach does not need to consider the consistent and the inconsistent multi-scale decision tables separately and is especially suitable for updating the optimal scales of the multi-scale decision tables with new objects added.

Data-Distribution-Aware Fuzzy Rough Set Model and its Application to Robust Classification

Fuzzy rough sets (FRSs) are considered to be a powerful model for analyzing uncertainty in data. This model encapsulates two types of uncertainty: 1) fuzziness coming from the vagueness in human concept formation and 2) roughness rooted in the granulation coming with human cognition. The rough set theory has been widely applied to feature selection, attribute reduction, and classification. However, it is reported that the classical FRS model is sensitive to noisy information. To address this problem, several robust models have been developed in recent years. Nevertheless, these models do not consider a statistical distribution of data, which is an important type of uncertainty. Data distribution serves as crucial information for designing an optimal classification or regression model. Thus, we propose a data-distribution-aware FRS model that considers distribution information and incorporates it in computing lower and upper fuzzy approximations. The proposed model considers not only the similarity between samples, but also the probability density of classes. In order to demonstrate the effectiveness of the proposed model, we design a new sample evaluation index for prototype-based classification based on the model, and a prototype selection algorithm is developed using this index. Furthermore, a robust classification algorithm is constructed with prototype covering and nearest neighbor classification. Experimental results confirm the robustness and effectiveness of the proposed model.

Monotonic classification extreme learning machine

Monotonic classification problems mean that both feature values and class labels are ordered and monotonicity relationships exist between some features and the decision label. Extreme Learning Machine (ELM) is a single-hidden layer feedforward neural network with fast training rate and good generalization capability, but due to the existence of training error, ELM cannot be directly used to handle monotonic classification problems. This work proposes a generalization of ELM for processing the monotonic classification, named as Monotonic Classification Extreme Learning Machine (MCELM) in which the monotonicity constraints are imposed to the original ELM model. Mathematically, MCELM is a quadratic programming problem in which the monotonicity relationships are considered as constraints and the training error is the objective to be minimized. The mathematical model of MCELM not only can make the generated classifier monotonic but also can minimize the classification error. MCELM does not need to tune parameters iteratively, and therefore, keeps the advantage of extremely fast training which is the essential characteristic of ELM. MCELM does not require that the monotonic relationships existing between features and the output are consistent, which essentially relaxes the assumption of consistent monotonicity used in most existing approaches to handling monotonic classification problems. In comparison with exiting approaches to handling monotonic classification, MCELM can indeed generate a monotonicity-reserving classifier which experimentally shows a much better generalization capability on both artificial and real world datasets.