Chen-Chia Chuang

Robust support vector regression networks for function approximation with outliers

Support vector regression (SVR) employs the support vector machine (SVM) to tackle problems of function approximation and regression estimation. SVR has been shown to have good robust properties against noise. When the parameters used in SVR are improperly selected, overfitting phenomena may still occur. However, the selection of various parameters is not straightforward. Besides, in SVR, outliers may also possibly be taken as support vectors. Such an inclusion of outliers in support vectors may lead to seriously overfitting phenomena. In this paper, a novel regression approach, termed as the robust support vector regression (RSVR) network, is proposed to enhance the robust capability of SVR. In the approach, traditional robust learning approaches are employed to improve the learning performance for any selected parameters. From the simulation results, our RSVR can always improve the performance of the learned systems for all cases. Besides, it can be found that even the training lasted for a long period, the testing errors would not go up. In other words, the overfitting phenomenon is indeed suppressed.

Genetic algorithm with ant colony optimization (GA-ACO) for multiple sequence alignment

Multiple sequence alignment, known as NP-complete problem, is among the most important and challenging tasks in computational biology. For multiple sequence alignment, it is difficult to solve this type of problems directly and always results in exponential complexity. In this paper, we present a novel algorithm of genetic algorithm with ant colony optimization for multiple sequence alignment. The proposed GA-ACO algorithm is to enhance the performance of genetic algorithm (GA) by incorporating local search, ant colony optimization (ACO), for multiple sequence alignment. In the proposed GA-ACO algorithm, genetic algorithm is conducted to provide the diversity of alignments. Thereafter, ant colony optimization is performed to move out of local optima. From simulation results, it is shown that the proposed GA-ACO algorithm has superior performance when compared to other existing algorithms.

Support vector interval regression networks for interval regression analysis

In this paper, the support vector interval regression networks (SVIRNs) are proposed for the interval regression analysis. The SVIRNs consist of two radial basis function networks. One network identifies the upper side of data interval, and the other network identifies the lower side of data intervals. Because the support vector regression (SVR) approach is equivalent to solving a linear constrained quadratic programming problem, the number of hidden nodes and the initial values of adjustable parameters can be easily obtained. Since the selection of a parameter e in the SVR approach may seriously affect the modeling performance, a two-step approach is proposed to properly select the e value. After the SVR approach with the selected e, an initial structure of SVIRNs can be obtained. Besides, outliers will not significantly affect the upper and lower bound interval obtained through the proposed two-step approach. Consequently, a traditional back-propagation (BP) learning algorithm can be used to adjust the initial structure networks of SVIRNs under training data sets without or with outliers. Due to the better initial structure of SVIRNs are obtained by the SVR approach, the convergence rate of SVIRNs is faster than the conventional networks with BP learning algorithms or with robust BP learning algorithms for interval regression analysis. Four examples are provided to show the validity and applicability of the proposed SVIRNs.

Robust TSK fuzzy modeling for function approximation with outliers

The Takagi-Sugeno-Kang (TSK) type of fuzzy models has attracted a great attention of the fuzzy modeling community due to their good performance in various applications. Most approaches for modeling TSK fuzzy rules define their fuzzy subspaces based on the idea of training data being close enough instead of having similar functions. Besides, training data sets algorithms often contain outliers, which seriously affect least-square error minimization clustering and learning algorithms. A robust TSK fuzzy modeling approach is presented. In the approach, a clustering algorithm termed as robust fuzzy regression agglomeration (RFRA) is proposed to define fuzzy subspaces in a fuzzy regression manner with robust capability against outliers. To obtain a more precision model, a robust fine-tuning algorithm is then employed. Various examples are used to verify the effectiveness of the proposed approach. From the simulation results, the proposed robust TSK fuzzy modeling indeed showed superior performance over other approaches.

Annealing robust radial basis function networks for function approximation with outliers

Abstract In this paper, the annealing robust radial basis function networks (ARRBFNs) are proposed to improve the problems of the robust radial basis function networks (RBFNs) for function approximation with outliers. Firstly, a support vector regression (SVR) approach is proposed to determine an initial structure of ARRBFNs in this paper. Because an SVR approach is equivalent to solving a linear constrained quadratic programming problem under a fixed structure of SVR, the number of hidden nodes, initial parameters and initial weights of the ARRBFNs are easily obtained. Secondly, the results of SVR are used as the initial structure in ARRBFNs. At the same time, an annealing robust learning algorithm (ARLA) is used as the learning algorithm for ARRBFNs, and applied to adjust the parameters as well as weights of ARRBFNs. That is, an ARLA is proposed to overcome the problems of initialization and the cut-off points in the robust learning algorithm. Hence, when an initial structure of ARRBFNs is determined by an SVR approach, the ARRBFNs with ARLA have fast convergence speed and are robust against outliers. Simulation results are provided to show the validity and applicability of the proposed ARRBFNs.

Hybrid compensation control for affine TSK fuzzy control systems

The paper proposes a way of designing state feedback controllers for affine Takagi-Sugeno-Kang (TSK) fuzzy models. In the approach, by combining two different control design methodologies, the proposed controller is designed to compensate all rules so that the desired control performance can appear in the overall system. Our approach treats all fuzzy rules as variations of a nominal rule and such variations are individually dealt with in a Lyapunov sense. Previous approaches have proposed a similar idea but the variations are dealt with as a whole in a robust control sense. As a consequence, when fuzzy rules are distributed in a wide range, the stability conditions may not be satisfied. In addition, the control performance of the closed-loop system cannot be anticipated in those approaches. Various examples were conducted in our study to demonstrate the effectiveness of the proposed control design approach. All results illustrate good control performances as desired.

Fuzzy Weighted Support Vector Regression With a Fuzzy Partition

The problem of the traditional support vector regression (SVR) approach, referred to as the global SVR approach, is the incapability of interpreting local behavior of the estimated models. An approach called the local SVR approach was proposed in the literature to cope with this problem. Although the local SVR approach can indeed model local behavior of models better than the global SVR approach does, the local SVR approach still has the problem of boundary effects, which may generate a large bias at the boundary and also need more time to calculate. In this paper, the fuzzy weighted SVR with a fuzzy partition is proposed. Because the concept of locally weighted regression is not used in the proposed approach, the boundary effects will not appear. The proposed method first employs the fuzzy c-mean clustering algorithm to split training data into several training subsets. Then, the local-regression models (LRMs) are independently obtained by the SVR approach for each training subset. Finally, those LRMs are combined by a fuzzy weighted mechanism to form the output. Experimental results show that the proposed approach needs less computational time than the local SVR approach and can have more accurate results than the local/global SVR approaches does

Radial Basis Function Networks With Linear Interval Regression Weights for Symbolic Interval Data

This paper introduces a new structure of radial basis function networks (RBFNs) that can successfully model symbolic interval-valued data. In the proposed structure, to handle symbolic interval data, the Gaussian functions required in the RBFNs are modified to consider interval distance measure, and the synaptic weights of the RBFNs are replaced by linear interval regression weights. In the linear interval regression weights, the lower and upper bounds of the interval-valued data as well as the center and range of the interval-valued data are considered. In addition, in the proposed approach, two stages of learning mechanisms are proposed. In stage 1, an initial structure (i.e., the number of hidden nodes and the adjustable parameters of radial basis functions) of the proposed structure is obtained by the interval competitive agglomeration clustering algorithm. In stage 2, a gradient-descent kind of learning algorithm is applied to fine-tune the parameters of the radial basis function and the coefficients of the linear interval regression weights. Various experiments are conducted, and the average behavior of the root mean square error and the square of the correlation coefficient in the framework of a Monte Carlo experiment are considered as the performance index. The results clearly show the effectiveness of the proposed structure.

Hybrid robust approach for TSK fuzzy modeling with outliers

This study proposes a hybrid robust approach for constructing Takagi-Sugeno-Kang (TSK) fuzzy models with outliers. The approach consists of a robust fuzzy C-regression model (RFCRM) clustering algorithm in the coarse-tuning phase and an annealing robust back-propagation (ARBP) learning algorithm in the fine-tuning phase. The RFCRM clustering algorithm is modified from the fuzzy C-regression models (FCRM) clustering algorithm by incorporating a robust mechanism and considering input data distribution and robust similarity measure into the FCRM clustering algorithm. Due to the use of robust mechanisms and the consideration of input data distribution, the fuzzy subspaces and the parameters of functions in the consequent parts are simultaneously identified by the proposed RFCRM clustering algorithm and the obtained model will not be significantly affected by outliers. Furthermore, the robust similarity measure is used in the clustering process to reduce the redundant clusters. Consequently, the RFCRM clustering algorithm can generate a better initialization for the TSK fuzzy models in the coarse-tuning phase. Then, an ARBP algorithm is employed to obtain a more precise model in the fine-tuning phase. From our simulation results, it is clearly evident that the proposed robust TSK fuzzy model approach is superior to existing approaches in learning speed and in approximation accuracy.

Adaptive fuzzy regression clustering algorithm for TSK fuzzy modeling

The TSK type of fuzzy models has attracted a great attention of the fuzzy modeling community due to their good performance in various applications. Some approaches for modeling TSK fuzzy rules have been proposed in the literature. Most of them define their fuzzy subspaces bases based on the idea of training data being close enough instead of having similar functions. In addition, the fuzzy C-regression model (FCRM) clustering algorithm is proposed to construct TSK fuzzy models. However, this approach does not take into account the data distribution. In this paper, a novel TSK fuzzy modeling approach is presented. In this approach, adaptive fuzzy regression clustering (AFRC) algorithm is proposed to simultaneously define fuzzy subspaces and find the parameters in the consequent parts of TSK rules. In addition, the similarity measure is used to reduce the redundant rules in the clustering process. To obtain a more precise model, a gradient descent algorithm is employed. From the simulation results, the proposed TSK fuzzy model approach indeed showed superior performance.