Zhaohong Deng

Enhanced soft subspace clustering integrating within-cluster and between-cluster information

While within-cluster information is commonly utilized in most soft subspace clustering approaches in order to develop the algorithms, other important information such as between-cluster information is seldom considered for soft subspace clustering. In this study, a novel clustering technique called enhanced soft subspace clustering (ESSC) is proposed by employing both within-cluster and between-class information. First, a new optimization objective function is developed by integrating the within-class compactness and the between-cluster separation in the subspace. Based on this objective function, the corresponding update rules for clustering are then derived, followed by the development of the novel ESSC algorithm. The properties of this algorithm are investigated and the performance is evaluated experimentally using real and synthetic datasets, including synthetic high dimensional datasets, UCI benchmarking datasets, high dimensional cancer gene expression datasets and texture image datasets. The experimental studies demonstrate that the accuracy of the proposed ESSC algorithm outperforms most existing state-of-the-art soft subspace clustering algorithms.

Collaborative Fuzzy Clustering From Multiple Weighted Views

Clustering with multiview data is becoming a hot topic in data mining, pattern recognition, and machine learning. In order to realize an effective multiview clustering, two issues must be addressed, namely, how to combine the clustering result from each view and how to identify the importance of each view. In this paper, based on a newly proposed objective function which explicitly incorporates two penalty terms, a basic multiview fuzzy clustering algorithm, called collaborative fuzzy c-means (Co-FCM), is firstly proposed. It is then extended into its weighted view version, called weighted view collaborative fuzzy c-means (WV-Co-FCM), by identifying the importance of each view. The WV-Co-FCM algorithm indeed tackles the above two issues simultaneously. Its relationship with the latest multiview fuzzy clustering algorithm Collaborative Fuzzy K-Means (Co-FKM) is also revealed. Extensive experimental results on various multiview datasets indicate that the proposed WV-Co-FCM algorithm outperforms or is at least comparable to the existing state-of-the-art multitask and multiview clustering algorithms and the importance of different views of the datasets can be effectively identified.

Scalable TSK Fuzzy Modeling for Very Large Datasets Using Minimal-Enclosing-Ball Approximation

In order to overcome the difficulty in Takagi-Sugeno-Kang (TSK) fuzzy modeling for large datasets, scalable TSK (STSK) fuzzy-model training is investigated in this study based on the core-set-based minimal-enclosing-ball (MEB) approximation technique. The specified L2-norm penalty-based -insensitive criterion is first proposed for TSK-model training, and it is found that such TSK fuzzy-model training can be equivalently expressed as a center-constrained MEB problem. With this finding, an STSK fuzzy-model-training algorithm, which is called STSK, for large or very large datasets is then proposed by using the core-set-based MEB-approximation technique. The proposed algorithm has two distinctive advantages over classical TSK fuzzy-model training algorithms: The maximum space complexity for training is not reliant on the size of the training dataset, and the maximum time complexity for training is linear with the size of the training dataset, as confirmed by extensive experiments on both synthetic and real-world regression datasets.

Multitask TSK Fuzzy System Modeling by Mining Intertask Common Hidden Structure

The classical fuzzy system modeling methods implicitly assume data generated from a single task, which is essentially not in accordance with many practical scenarios where data can be acquired from the perspective of multiple tasks. Although one can build an individual fuzzy system model for each task, the result indeed tells us that the individual modeling approach will get poor generalization ability due to ignoring the intertask hidden correlation. In order to circumvent this shortcoming, we consider a general framework for preserving the independent information among different tasks and mining hidden correlation information among all tasks in multitask fuzzy modeling. In this framework, a low-dimensional subspace (structure) is assumed to be shared among all tasks and hence be the hidden correlation information among all tasks. Under this framework, a multitask Takagi-Sugeno-Kang (TSK) fuzzy system model called MTCS-TSK-FS (TSK-FS for multiple tasks with common hidden structure), based on the classical L2-norm TSK fuzzy system, is proposed in this paper. The proposed model can not only take advantage of independent sample information from the original space for each task, but also effectively use the intertask common hidden structure among multiple tasks to enhance the generalization performance of the built fuzzy systems. Experiments on synthetic and real-world datasets demonstrate the applicability and distinctive performance of the proposed multitask fuzzy system model in multitask regression learning scenarios.

Knowledge-Leverage-Based TSK Fuzzy System Modeling

Classical fuzzy system modeling methods consider only the current scene where the training data are assumed to be fully collectable. However, if the data available from the current scene are insufficient, the fuzzy systems trained by using the incomplete datasets will suffer from weak generalization capability for the prediction in the scene. In order to overcome this problem, a knowledge-leverage-based fuzzy system (KL-FS) is studied in this paper from the perspective of transfer learning. The KL-FS intends to not only make full use of the data from the current scene in the learning procedure, but also effectively leverage the existing knowledge from the reference scenes. Specifically, a knowledge-leverage-based Takagi-Sugeno-Kang-type Fuzzy System (KL-TSK-FS) is proposed by integrating the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique is evaluated through experiments on synthetic and real-world datasets. The results demonstrate that KL-TSK-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenes with insufficient data.Classical fuzzy system modeling methods consider only the current scene where the training data are assumed to be fully collectable. However, if the data available from the current scene are insufficient, the fuzzy systems trained by using the incomplete datasets will suffer from weak generalization capability for the prediction in the scene. In order to overcome this problem, a knowledge-leverage-based fuzzy system (KL-FS) is studied in this paper from the perspective of transfer learning. The KL-FS intends to not only make full use of the data from the current scene in the learning procedure, but also effectively leverage the existing knowledge from the reference scenes. Specifically, a knowledge-leverage-based Takagi-Sugeno-Kang-type Fuzzy System (KL-TSK-FS) is proposed by integrating the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique is evaluated through experiments on synthetic and real-world datasets. The results demonstrate that KL-TSK-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenes with insufficient data.

Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods

Inductive transfer learning has attracted increasing attention for the training of effective model in the target domain by leveraging the information in the source domain. However, most transfer learning methods are developed for a specific model, such as the commonly used support vector machine, which makes the methods applicable only to the adopted models. In this regard, the generalized hidden-mapping ridge regression (GHRR) method is introduced in order to train various types of classical intelligence models, including neural networks, fuzzy logical systems and kernel methods. Furthermore, the knowledge-leverage based transfer learning mechanism is integrated with GHRR to realize the inductive transfer learning method called transfer GHRR (TGHRR). Since the information from the induced knowledge is much clearer and more concise than that from the data in the source domain, it is more convenient to control and balance the similarity and difference of data distributions between the source and target domains. The proposed GHRR and TGHRR algorithms have been evaluated experimentally by performing regression and classification on synthetic and real world datasets. The results demonstrate that the performance of TGHRR is competitive with or even superior to existing state-of-the-art inductive transfer learning algorithms.

FRSDE: Fast reduced set density estimator using minimal enclosing ball approximation

Reduced set density estimator (RSDE) is an important technique that can be used to replace the classical Parzen window estimator (PW) for saving the computational cost. Though RSDE demonstrates a nicer performance in the density accuracy and the computational time compared with several existing methods, it still faces the critical challenge for practical applications because of its high time complexity (no less than O(N^2)) and space complexity (O(N^2)) in training the model weighting coefficients on large data sets. In order to overcome this shortcoming, a fast reduced set density estimator algorithm (FRSDE) is proposed in this study. First, the relationship between RSDE and the minimal enclosing ball problems (MEB) in computational geometry is revealed. Then, the finding that RSDE is equivalent to a special MEB problem is derived. With this finding, the fast core-set based MEB approximation algorithm is introduced to develop the proposed algorithm FRSDE. Compared with RSDE, FRSDE has the following distinctive advantage: it can guarantee that the upper bound of the time complexity is linear with the size N of a large data set and the upper bound of the space complexity is independent of N. Our experimental results show that the proposed FRSDE has a competitive performance in the density accuracy and an overwhelming advantage over RSDE for large data sets in the data condensation rate and the training time for the weighting coefficients.

From Minimum Enclosing Ball to Fast Fuzzy Inference System Training on Large Datasets

While fuzzy inference systems (FISs) have been extensively studied in the past decades, the minimum enclosing ball (MEB) problem was recently introduced to develop fast and scalable methods in pattern classification and machine learning. In this paper, the relationship between these two apparently different data modeling techniques is explored. First, based on the reduced-set density estimator, a bridge between the MEB problem and the FIS is established. Then, an important finding that the Mamdani-Larsen FIS (ML-FIS) can be translated into a special kernelized MEB problem, i.e., a center-constrained MEB problem under some conditions, is revealed. Thus, fast kernelized MEB approximation algorithms can be adopted to construct ML-FIS in an efficient manner. Here, we propose the use of a core vector machine (CVM), which is a fast kernelized MEB approximation algorithm for support vector machine (SVM) training, to accomplish this task. The proposed fast ML-FIS training algorithm has the following merits: (1) the number of fuzzy rules can be automatically determined by the CVM training and (2) fast ML-FIS training on large datasets can be achieved as the upper bound on the time complexity of learning the parameters in ML-FIS is linear with the dataset size N and the upper bound on the corresponding space complexity is theoretically independent of N. Our experiments on simulated and real datasets confirm these advantages of the proposed training method, and demonstrate its superior robustness as well. This paper not only represents a very first study of the relationship between MEB and FIS, but it also points out the mutual transformation between kernel methods and FISs under the framework of the Gaussian mixture model and MEB.

T2FELA: Type-2 Fuzzy Extreme Learning Algorithm for Fast Training of Interval Type-2 TSK Fuzzy Logic System

A challenge in modeling type-2 fuzzy logic systems is the development of efficient learning algorithms to cope with the ever increasing size of real-world data sets. In this paper, the extreme learning strategy is introduced to develop a fast training algorithm for interval type-2 Takagi-Sugeno-Kang fuzzy logic systems. The proposed algorithm, called type-2 fuzzy extreme learning algorithm (T2FELA), has two distinctive characteristics. First, the parameters of the antecedents are randomly generated and parameters of the consequents are obtained by a fast learning method according to the extreme learning mechanism. In addition, because the obtained parameters are optimal in the sense of minimizing the norm, the resulting fuzzy systems exhibit better generalization performance. The experimental results clearly demonstrate that the training speed of the proposed T2FELA algorithm is superior to that of the existing state-of-the-art algorithms. The proposed algorithm also shows competitive performance in generalization abilities.

Knowledge-Leverage-Based Fuzzy System and Its Modeling

The classical fuzzy system modeling methods only consider the current scene where the training data are assumed fully collectable. However, if the available data from that scene are insufficient, the fuzzy systems trained will suffer from weak generalization for the modeling task in this scene. In order to overcome this problem, a fuzzy system with knowledge-leverage capability, which is known as a knowledge-leverage-based fuzzy system (KL-FS), is proposed in this paper. The KL-FS not only makes full use of the data from the current scene in the learning procedure but can effectively make leverage on the existing knowledge from the reference scene, e.g., the parameters of a fuzzy system obtained from a reference scene, as well. Specifically, a knowledge-leverage-based Mamdani-Larsen-type fuzzy system (KL-ML-FS) is proposed by using the reduced set density estimation technique integrating with the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique has been verified by experiments on synthetic and real-world datasets, where KL-ML-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenarios with insufficient data.