Yizhang Jiang

Collaborative Fuzzy Clustering From Multiple Weighted Views

Clustering with multiview data is becoming a hot topic in data mining, pattern recognition, and machine learning. In order to realize an effective multiview clustering, two issues must be addressed, namely, how to combine the clustering result from each view and how to identify the importance of each view. In this paper, based on a newly proposed objective function which explicitly incorporates two penalty terms, a basic multiview fuzzy clustering algorithm, called collaborative fuzzy c-means (Co-FCM), is firstly proposed. It is then extended into its weighted view version, called weighted view collaborative fuzzy c-means (WV-Co-FCM), by identifying the importance of each view. The WV-Co-FCM algorithm indeed tackles the above two issues simultaneously. Its relationship with the latest multiview fuzzy clustering algorithm Collaborative Fuzzy K-Means (Co-FKM) is also revealed. Extensive experimental results on various multiview datasets indicate that the proposed WV-Co-FCM algorithm outperforms or is at least comparable to the existing state-of-the-art multitask and multiview clustering algorithms and the importance of different views of the datasets can be effectively identified.

Multitask TSK Fuzzy System Modeling by Mining Intertask Common Hidden Structure

The classical fuzzy system modeling methods implicitly assume data generated from a single task, which is essentially not in accordance with many practical scenarios where data can be acquired from the perspective of multiple tasks. Although one can build an individual fuzzy system model for each task, the result indeed tells us that the individual modeling approach will get poor generalization ability due to ignoring the intertask hidden correlation. In order to circumvent this shortcoming, we consider a general framework for preserving the independent information among different tasks and mining hidden correlation information among all tasks in multitask fuzzy modeling. In this framework, a low-dimensional subspace (structure) is assumed to be shared among all tasks and hence be the hidden correlation information among all tasks. Under this framework, a multitask Takagi-Sugeno-Kang (TSK) fuzzy system model called MTCS-TSK-FS (TSK-FS for multiple tasks with common hidden structure), based on the classical L2-norm TSK fuzzy system, is proposed in this paper. The proposed model can not only take advantage of independent sample information from the original space for each task, but also effectively use the intertask common hidden structure among multiple tasks to enhance the generalization performance of the built fuzzy systems. Experiments on synthetic and real-world datasets demonstrate the applicability and distinctive performance of the proposed multitask fuzzy system model in multitask regression learning scenarios.

Knowledge-Leverage-Based TSK Fuzzy System Modeling

Classical fuzzy system modeling methods consider only the current scene where the training data are assumed to be fully collectable. However, if the data available from the current scene are insufficient, the fuzzy systems trained by using the incomplete datasets will suffer from weak generalization capability for the prediction in the scene. In order to overcome this problem, a knowledge-leverage-based fuzzy system (KL-FS) is studied in this paper from the perspective of transfer learning. The KL-FS intends to not only make full use of the data from the current scene in the learning procedure, but also effectively leverage the existing knowledge from the reference scenes. Specifically, a knowledge-leverage-based Takagi-Sugeno-Kang-type Fuzzy System (KL-TSK-FS) is proposed by integrating the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique is evaluated through experiments on synthetic and real-world datasets. The results demonstrate that KL-TSK-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenes with insufficient data.Classical fuzzy system modeling methods consider only the current scene where the training data are assumed to be fully collectable. However, if the data available from the current scene are insufficient, the fuzzy systems trained by using the incomplete datasets will suffer from weak generalization capability for the prediction in the scene. In order to overcome this problem, a knowledge-leverage-based fuzzy system (KL-FS) is studied in this paper from the perspective of transfer learning. The KL-FS intends to not only make full use of the data from the current scene in the learning procedure, but also effectively leverage the existing knowledge from the reference scenes. Specifically, a knowledge-leverage-based Takagi-Sugeno-Kang-type Fuzzy System (KL-TSK-FS) is proposed by integrating the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique is evaluated through experiments on synthetic and real-world datasets. The results demonstrate that KL-TSK-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenes with insufficient data.

Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods

Inductive transfer learning has attracted increasing attention for the training of effective model in the target domain by leveraging the information in the source domain. However, most transfer learning methods are developed for a specific model, such as the commonly used support vector machine, which makes the methods applicable only to the adopted models. In this regard, the generalized hidden-mapping ridge regression (GHRR) method is introduced in order to train various types of classical intelligence models, including neural networks, fuzzy logical systems and kernel methods. Furthermore, the knowledge-leverage based transfer learning mechanism is integrated with GHRR to realize the inductive transfer learning method called transfer GHRR (TGHRR). Since the information from the induced knowledge is much clearer and more concise than that from the data in the source domain, it is more convenient to control and balance the similarity and difference of data distributions between the source and target domains. The proposed GHRR and TGHRR algorithms have been evaluated experimentally by performing regression and classification on synthetic and real world datasets. The results demonstrate that the performance of TGHRR is competitive with or even superior to existing state-of-the-art inductive transfer learning algorithms.

Knowledge-Leverage-Based Fuzzy System and Its Modeling

The classical fuzzy system modeling methods only consider the current scene where the training data are assumed fully collectable. However, if the available data from that scene are insufficient, the fuzzy systems trained will suffer from weak generalization for the modeling task in this scene. In order to overcome this problem, a fuzzy system with knowledge-leverage capability, which is known as a knowledge-leverage-based fuzzy system (KL-FS), is proposed in this paper. The KL-FS not only makes full use of the data from the current scene in the learning procedure but can effectively make leverage on the existing knowledge from the reference scene, e.g., the parameters of a fuzzy system obtained from a reference scene, as well. Specifically, a knowledge-leverage-based Mamdani-Larsen-type fuzzy system (KL-ML-FS) is proposed by using the reduced set density estimation technique integrating with the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique has been verified by experiments on synthetic and real-world datasets, where KL-ML-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenarios with insufficient data.

Distance metric learning for soft subspace clustering in composite kernel space

Soft subspace clustering algorithms have been successfully used for high dimensional data in recent years. However, the existing algorithms often utilize only one distance function to evaluate the distance between data items on each feature, which cannot deal with datasets with complex inner structures. In this paper, a composite kernel space (CKS) is constructed based on a set of basis kernels and a novel framework of soft subspace clustering is proposed by integrating distance metric learning in the CKS. Two soft subspace clustering algorithms, i.e., entropy weighting fuzzy clustering in CKS for kernel space (CKS-EWFC-K) and feature space (CKS-EWFC-F) are thus developed. In both algorithms, the prototype in the feature space is mapped into the CKS by multiple simultaneous mappings, one mapping for each cluster, which is distinct from existing kernel-based clustering algorithms. By evaluating the distance on each feature in the CKS, both CKS-EWFC-K and CKS-EWFC-F learn the distance function adaptively during the clustering process. Experimental results have demonstrated that the proposed algorithms in general outperform classical clustering algorithms and are immune to ineffective kernels and irrelevant features in soft subspace. The composite kernel space is constructed based on a set of basis kernels.The general form of soft subspace clustering in CKS is presented.CKS-EWFC-K and CKS-EWFC-F are proposed under the framework of CKS-SSC.The properties of CKS-EWFC-K and CKS-EWFC-F are investigated.Both CKS-EWFC-K and CKS-EWFC-F are immune to ineffective kernels.

A survey on soft subspace clustering

Subspace clustering (SC) is a promising technology involving clusters that are identified based on their association with subspaces in high-dimensional spaces. SC can be classified into hard subspace clustering (HSC) and soft subspace clustering (SSC). While HSC algorithms have been studied extensively and are well accepted by the scientific community, SSC algorithms are relatively new. However, as they are said to be more adaptable than their HSC counterparts, SSC algorithms have been attracting more attention in recent years. A comprehensive survey of existing SSC algorithms and recent developments in the field are presented in this paper. SSC algorithms have been systematically classified into three main categories: conventional SSC (CSSC), independent SSC (ISSC), and extended SSC (XSSC). The characteristics of these algorithms are highlighted and potential future developments in the area of SSC are discussed. Through a comprehensive review of SSC, this paper aims to provide readers with a clear profile of existing SSC methods and to foster the development of more effective clustering technologies and significant research in this area.

Minimax Probability TSK Fuzzy System Classifier: A More Transparent and Highly Interpretable Classification Model

When an intelligent model is used for medical diagnosis, it is desirable to have a high level of interpretability and transparent model reliability for users. Compared with most of the existing intelligence models, fuzzy systems have shown a distinctive advantage in their interpretabilities. However, how to determine the model reliability of a fuzzy system trained for a recognition task is still an unsolved problem at present. In this study, a minimax probability Takagi-Sugeno-Kang (TSK) fuzzy system classifier called MP-TSK-FSC is proposed to train a fuzzy system classifier and determine the model reliability simultaneously. For the proposed MP-TSK-FSC, a lower bound of correct classification can be presented to the users to characterize the reliability of the trained fuzzy classifier. Thus, the obtained classifier has the distinctive characteristics of both a high level of interpretability and transparent model reliability inherited from the fuzzy system and minimax probability learning strategy, respectively. Our experiments on synthetic datasets and several real-world datasets for medical diagnosis have confirmed the distinctive characteristics of the proposed method.

Feedforward kernel neural networks, generalized least learning machine, and its deep learning with application to image classification

The feedforward kernel neural networks called FKNN are proposed.FKNN can work in both generalized-least-learning and deep-learning ways through implicit or explicit KPCAs.FKNNs deep learning framework DLP is justified by experiments about image classification. In this paper, the architecture of feedforward kernel neural networks (FKNN) is proposed, which can include a considerably large family of existing feedforward neural networks and hence can meet most practical requirements. Different from the common understanding of learning, it is revealed that when the number of the hidden nodes of every hidden layer and the type of the adopted kernel based activation functions are pre-fixed, a special kernel principal component analysis (KPCA) is always implicitly executed, which can result in the fact that all the hidden layers of such networks need not be tuned and their parameters can be randomly assigned and even may be independent of the training data. Therefore, the least learning machine (LLM) is extended into its generalized version in the sense of adopting much more error functions rather than mean squared error (MSE) function only. As an additional merit, it is also revealed that rigorous Mercer kernel condition is not required in FKNN networks. When the proposed architecture of FKNN networks is constructed in a layer-by-layer way, i.e., the number of the hidden nodes of every hidden layer may be determined only in terms of the extracted principal components after the explicit execution of a KPCA, we can develop FKNNs deep architecture such that its deep learning framework (DLF) has strong theoretical guarantee. Our experimental results about image classification manifest that the proposed FKNNs deep architecture and its DLF based learning indeed enhance the classification performance.

Transfer Prototype-Based Fuzzy Clustering

Traditional prototype-based clustering methods, such as the well-known fuzzy c-means (FCM) algorithm, usually need sufficient data to find a good clustering partition. If available data are limited or scarce, most of them are no longer effective. While the data for the current clustering task may be scarce, there is usually some useful knowledge available in the related scenes/domains. In this study, the concept of transfer learning is applied to prototype-based fuzzy clustering (PFC). Specifically, the idea of leveraging knowledge from the source domain is exploited to develop a set of transfer PFC algorithms. First, two representative PFC algorithms, namely, FCM and fuzzy subspace clustering, have been chosen to incorporate with knowledge leveraging mechanisms to develop the corresponding transfer clustering algorithms based on an assumption that there are the same number of clusters between the target domain (current scene) and the source domain (related scene). Furthermore, two extended versions are also proposed to implement the transfer learning for the situation that there are different numbers of clusters between two domains. The novel objective functions are proposed to integrate the knowledge from the source domain with the data in the target domain for the clustering in the target domain. The proposed algorithms have been validated on different synthetic and real-world datasets. Experimental results demonstrate their effectiveness in comparison with both the original PFC algorithms and the related clustering algorithms like multitask clustering and coclustering.