Fu-Lai Chung

Positive and negative fuzzy rule system, extreme learning machine and image classification

We often use the positive fuzzy rules only for image classification in traditional image classification systems, ignoring the useful negative classification information. Thanh Minh Nguyen and QMJonathan Wu introduced the negative fuzzy rules into the image classification, and proposed combination of positive and negative fuzzy rules to form the positive and negative fuzzy rule system, and then applied it to remote sensing image/natural image classification. Their experiments demonstrated that their proposed method has achieved promising results. However, since their method was realized using the feedforward neural network model which requires adjusting the weights in the gradient descent way, the training speed is very slow. Extreme learning machine (ELM) is a single hidden layer feedforward neural network (SLFNs) learning algorithm, which has distinctive advantages such as quick learning, good generalization performance. In this paper, the equivalence between ELM and the positive and negative fuzzy rule system is revealed, so ELM can be naturally used for training the positive and negative fuzzy rule system quickly for image classification. Our experimental results indicate this claim.

Robust fuzzy clustering-based image segmentation

The fuzzy clustering algorithm fuzzy c-means (FCM) is often used for image segmentation. When noisy image segmentation is required, FCM should be modified such that it can be less sensitive to noise in an image. In this correspondence, a robust fuzzy clustering-based segmentation method for noisy images is developed. The contribution of the study here is twofold: (1) we derive a robust modified FCM in the sense of a novel objective function. The proposed modified FCM here is proved to be equivalent to the modified FCM given by Hoppner and Klawonn [F. Hoppner, F. Klawonn, Improved fuzzy partitions for fuzzy regression models, Int. J. Approx. Reason. 32 (2) (2003) 85-102]. (2) We explore the very applicability of the proposed modified FCM for noisy image segmentation. Our experimental results indicate that the proposed modified FCM here is very suitable for noisy image segmentation.

Collaborative Fuzzy Clustering From Multiple Weighted Views

Clustering with multiview data is becoming a hot topic in data mining, pattern recognition, and machine learning. In order to realize an effective multiview clustering, two issues must be addressed, namely, how to combine the clustering result from each view and how to identify the importance of each view. In this paper, based on a newly proposed objective function which explicitly incorporates two penalty terms, a basic multiview fuzzy clustering algorithm, called collaborative fuzzy c-means (Co-FCM), is firstly proposed. It is then extended into its weighted view version, called weighted view collaborative fuzzy c-means (WV-Co-FCM), by identifying the importance of each view. The WV-Co-FCM algorithm indeed tackles the above two issues simultaneously. Its relationship with the latest multiview fuzzy clustering algorithm Collaborative Fuzzy K-Means (Co-FKM) is also revealed. Extensive experimental results on various multiview datasets indicate that the proposed WV-Co-FCM algorithm outperforms or is at least comparable to the existing state-of-the-art multitask and multiview clustering algorithms and the importance of different views of the datasets can be effectively identified.

Scalable TSK Fuzzy Modeling for Very Large Datasets Using Minimal-Enclosing-Ball Approximation

In order to overcome the difficulty in Takagi-Sugeno-Kang (TSK) fuzzy modeling for large datasets, scalable TSK (STSK) fuzzy-model training is investigated in this study based on the core-set-based minimal-enclosing-ball (MEB) approximation technique. The specified L2-norm penalty-based -insensitive criterion is first proposed for TSK-model training, and it is found that such TSK fuzzy-model training can be equivalently expressed as a center-constrained MEB problem. With this finding, an STSK fuzzy-model-training algorithm, which is called STSK, for large or very large datasets is then proposed by using the core-set-based MEB-approximation technique. The proposed algorithm has two distinctive advantages over classical TSK fuzzy-model training algorithms: The maximum space complexity for training is not reliant on the size of the training dataset, and the maximum time complexity for training is linear with the size of the training dataset, as confirmed by extensive experiments on both synthetic and real-world regression datasets.

Multitask TSK Fuzzy System Modeling by Mining Intertask Common Hidden Structure

The classical fuzzy system modeling methods implicitly assume data generated from a single task, which is essentially not in accordance with many practical scenarios where data can be acquired from the perspective of multiple tasks. Although one can build an individual fuzzy system model for each task, the result indeed tells us that the individual modeling approach will get poor generalization ability due to ignoring the intertask hidden correlation. In order to circumvent this shortcoming, we consider a general framework for preserving the independent information among different tasks and mining hidden correlation information among all tasks in multitask fuzzy modeling. In this framework, a low-dimensional subspace (structure) is assumed to be shared among all tasks and hence be the hidden correlation information among all tasks. Under this framework, a multitask Takagi-Sugeno-Kang (TSK) fuzzy system model called MTCS-TSK-FS (TSK-FS for multiple tasks with common hidden structure), based on the classical L2-norm TSK fuzzy system, is proposed in this paper. The proposed model can not only take advantage of independent sample information from the original space for each task, but also effectively use the intertask common hidden structure among multiple tasks to enhance the generalization performance of the built fuzzy systems. Experiments on synthetic and real-world datasets demonstrate the applicability and distinctive performance of the proposed multitask fuzzy system model in multitask regression learning scenarios.

Knowledge-Leverage-Based TSK Fuzzy System Modeling

Classical fuzzy system modeling methods consider only the current scene where the training data are assumed to be fully collectable. However, if the data available from the current scene are insufficient, the fuzzy systems trained by using the incomplete datasets will suffer from weak generalization capability for the prediction in the scene. In order to overcome this problem, a knowledge-leverage-based fuzzy system (KL-FS) is studied in this paper from the perspective of transfer learning. The KL-FS intends to not only make full use of the data from the current scene in the learning procedure, but also effectively leverage the existing knowledge from the reference scenes. Specifically, a knowledge-leverage-based Takagi-Sugeno-Kang-type Fuzzy System (KL-TSK-FS) is proposed by integrating the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique is evaluated through experiments on synthetic and real-world datasets. The results demonstrate that KL-TSK-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenes with insufficient data.Classical fuzzy system modeling methods consider only the current scene where the training data are assumed to be fully collectable. However, if the data available from the current scene are insufficient, the fuzzy systems trained by using the incomplete datasets will suffer from weak generalization capability for the prediction in the scene. In order to overcome this problem, a knowledge-leverage-based fuzzy system (KL-FS) is studied in this paper from the perspective of transfer learning. The KL-FS intends to not only make full use of the data from the current scene in the learning procedure, but also effectively leverage the existing knowledge from the reference scenes. Specifically, a knowledge-leverage-based Takagi-Sugeno-Kang-type Fuzzy System (KL-TSK-FS) is proposed by integrating the corresponding knowledge-leverage mechanism. The new fuzzy system modeling technique is evaluated through experiments on synthetic and real-world datasets. The results demonstrate that KL-TSK-FS has better performance and adaptability than the traditional fuzzy modeling methods in scenes with insufficient data.

Evolutionary time series segmentation for stock data mining

Stock data in the form of multiple time series are difficult to process, analyze and mine. However, when they can be transformed into meaningful symbols like technical patterns, it becomes easier. Most recent work on time series queries concentrates only on how to identify a given pattern from a time series. Researchers do not consider the problem of identifying a suitable set of time points for segmenting the time series in accordance with a given set of pattern templates (e.g., a set of technical patterns for stock analysis). On the other hand, using fixed length segmentation is a primitive approach to this problem; hence, a dynamic approach (with high controllability) is preferred so that the time series can be segmented flexibly and effectively according to the needs of users and applications. In view of the fact that such a segmentation problem is an optimization problem and evolutionary computation is an appropriate tool to solve it, we propose an evolutionary time series segmentation algorithm. This approach allows a sizeable set of stock patterns to be generated for mining or query. In addition, defining the similarity between time series (or time series segments) is of fundamental importance in fitness computation. By identifying perceptually important points directly from the time domain, time series segments and templates of different lengths can be compared and intuitive pattern matching can be carried out in an effective and efficient manner. Encouraging experimental results are reported from tests that segment the time series of selected Hong Kong stocks.

On minimum distribution discrepancy support vector machine for domain adaptation

Domain adaptation learning (DAL) is a novel and effective technique to address pattern classification problems where the prior information for training is unavailable or insufficient. Its effectiveness depends on the discrepancy between the two distributions that respectively generate the training data for the source domain and the testing data for the target domain. However, DAL may not work so well when only the distribution mean discrepancy between source and target domains is considered and minimized. In this paper, we first construct a generalized projected maximum distribution discrepancy (GPMDD) metric for DAL on reproducing kernel Hilbert space (RKHS) based domain distributions by simultaneously considering both the projected maximum distribution mean and the projected maximum distribution scatter discrepancy between the source and the target domain. In the sequel, based on both the structure risk and the GPMDD minimization principle, we propose a novel domain adaptation kernelized support vector machine (DAKSVM) with respect to the classical SVM, and its two extensions called LS-DAKSVM and @m-DAKSVM with respect to the least-square SVM and the v-SVM, respectively. Moreover, our theoretical analysis justified that the proposed GPMDD metric could effectively measure the consistency not only between the RKHS embedding domain distributions but also between the scatter information of source and target domains. Hence, the proposed methods are distinctive in that the more consistency between the scatter information of source and target domains can be achieved by tuning the kernel bandwidth, the better the convergence of GPMDD metric minimization is and thus improving the scalability and generalization capability of the proposed methods for DAL. Experimental results on artificial and real-world problems indicate that the performance of the proposed methods is superior to or at least comparable with existing benchmarking methods.

Feature based robust watermarking using image normalization

This paper presents a robust watermarking scheme based on feature point detection and image normalization. Firstly some stable feature points are detected from the original image using the proposed multiresolution feature point detection filter. Then, image normalization is applied to the disks centered at these feature points. The watermark is embedded in the subband coefficients of DFT domain of each disk separately. And the watermark detection uses the correlation between the watermark embedding coefficients and the original watermark, and does not need the original image. The proposed scheme combines the advantages of feature point detection and image normalization, which can achieve strong robustness to signal processing and geometrical distortions. The experimental results also demonstrate good performance of the proposed scheme.

From Minimum Enclosing Ball to Fast Fuzzy Inference System Training on Large Datasets

While fuzzy inference systems (FISs) have been extensively studied in the past decades, the minimum enclosing ball (MEB) problem was recently introduced to develop fast and scalable methods in pattern classification and machine learning. In this paper, the relationship between these two apparently different data modeling techniques is explored. First, based on the reduced-set density estimator, a bridge between the MEB problem and the FIS is established. Then, an important finding that the Mamdani-Larsen FIS (ML-FIS) can be translated into a special kernelized MEB problem, i.e., a center-constrained MEB problem under some conditions, is revealed. Thus, fast kernelized MEB approximation algorithms can be adopted to construct ML-FIS in an efficient manner. Here, we propose the use of a core vector machine (CVM), which is a fast kernelized MEB approximation algorithm for support vector machine (SVM) training, to accomplish this task. The proposed fast ML-FIS training algorithm has the following merits: (1) the number of fuzzy rules can be automatically determined by the CVM training and (2) fast ML-FIS training on large datasets can be achieved as the upper bound on the time complexity of learning the parameters in ML-FIS is linear with the dataset size N and the upper bound on the corresponding space complexity is theoretically independent of N. Our experiments on simulated and real datasets confirm these advantages of the proposed training method, and demonstrate its superior robustness as well. This paper not only represents a very first study of the relationship between MEB and FIS, but it also points out the mutual transformation between kernel methods and FISs under the framework of the Gaussian mixture model and MEB.