Yan-Qing Zhang

SVMs Modeling for Highly Imbalanced Classification

Traditional classification algorithms can be limited in their performance on highly unbalanced data sets. A popular stream of work for countering the problem of class imbalance has been the application of a sundry of sampling strategies. In this paper, we focus on designing modifications to support vector machines (SVMs) to appropriately tackle the problem of class imbalance. We incorporate different ldquorebalancerdquo heuristics in SVM modeling, including cost-sensitive learning, and over- and undersampling. These SVM-based strategies are compared with various state-of-the-art approaches on a variety of data sets by using various metrics, including G-mean, area under the receiver operating characteristic curve, F-measure, and area under the precision/recall curve. We show that we are able to surpass or match the previously known best algorithms on each data set. In particular, of the four SVM variations considered in this paper, the novel granular SVMs-repetitive undersampling algorithm (GSVM-RU) is the best in terms of both effectiveness and efficiency. GSVM-RU is effective, as it can minimize the negative effect of information loss while maximizing the positive effect of data cleaning in the undersampling process. GSVM-RU is efficient by extracting much less support vectors and, hence, greatly speeding up SVM prediction.

A genetic algorithm-based method for feature subset selection

As a commonly used technique in data preprocessing, feature selection selects a subset of informative attributes or variables to build models describing data. By removing redundant and irrelevant or noise features, feature selection can improve the predictive accuracy and the comprehensibility of the predictors or classifiers. Many feature selection algorithms with different selection criteria has been introduced by researchers. However, it is discovered that no single criterion is best for all applications. In this paper, we propose a framework based on a genetic algorithm (GA) for feature subset selection that combines various existing feature selection methods. The advantages of this approach include the ability to accommodate multiple feature selection criteria and find small subsets of features that perform well for a particular inductive learning algorithm of interest to build the classifier. We conducted experiments using three data sets and three existing feature selection methods. The experimental results demonstrate that our approach is a robust and effective approach to find subsets of features with higher classification accuracy and/or smaller size compared to each individual feature selection algorithm.

Parallel granular neural networks for fast credit card fraud detection

A parallel granular neural network (GNN) is developed to speed up data mining and knowledge discovery process for credit card fraud detection. The entire system is parallelized on the Silicon Graphics Origin 2000, which is a shared memory multiprocessor system consisting of 24-CPU, 4G main memory, and 200 GB hard-drive. In simulations, the parallel fuzzy neural network running on a 24-processor system is trained in parallel using training data sets, and then the trained parallel fuzzy neural network discovers fuzzy rules for future prediction. A parallel learning algorithm is implemented in C. The data are extracted into a flat file from an SQL server database containing sample Visa Card transactions and then preprocessed for applying in fraud detection. The data are classified into three categories: first for training, second for prediction, and third for fraud detection. After learning from training data, the GNN is used to predict on a second set of data and later the third set of data is applied for fraud detection. GNN gives fewer average training errors with larger amount of past training data. The higher the fraud detection error is, the greater the possibility of that transaction being actually fraudulent.

Granular neural networks for numerical-linguistic data fusion and knowledge discovery

In this paper, we present a neural-networks-based knowledge discovery and data mining (KDDM) methodology based on granular computing, neural computing, fuzzy computing, linguistic computing, and pattern recognition. The major issues include 1) how to make neural networks process both numerical and linguistic data in a data base, 2) how to convert fuzzy linguistic data into related numerical features, 3) how to use neural networks to do numerical-linguistic data fusion, 4) how to use neural networks to discover granular knowledge from numerical-linguistic data bases, and 5) how to use discovered granular knowledge to predict missing data. In order to answer the above concerns, a granular neural network (GNN) is designed to deal with numerical-linguistic data fusion and granular knowledge discovery in numerical-linguistic databases. From a data granulation point of view, the GNN can process granular data in a database. From a data fusion point of view, the GNN makes decisions based on different kinds of granular data. From a KDDM point of view, the GNN is able to learn internal granular relations between numerical-linguistic inputs and outputs, and predict new relations in a database. The GNN is also capable of greatly compressing low-level granular data to high-level granular knowledge with some compression error and a data compression rate. To do KDDM in huge data bases, parallel GNN and distributed GNN will be investigated in the future.

Stability analysis of fuzzy control systems

In this paper, we present a general review of the stability issue as related to fuzzy control systems. The concept of stability and the general criterion used for fuzzy control systems are discussed. The intuitive and nonlinear system stability analysis of fuzzy control systems are given. Popovs technique is proposed to test the stability of fuzzy systems.

Support vector machines with genetic fuzzy feature transformation for biomedical data classification

Abstract In this paper, we present a genetic fuzzy feature transformation method for support vector machines (SVMs) to do more accurate data classification. Given data are first transformed into a high feature space by a fuzzy system, and then SVMs are used to map data into a higher feature space and then construct the hyperplane to make a final decision. Genetic algorithms are used to optimize the fuzzy feature transformation so as to use the newly generated features to help SVMs do more accurate biomedical data classification under uncertainty. The experimental results show that the new genetic fuzzy SVMs have better generalization abilities than the traditional SVMs in terms of prediction accuracy.

Compensatory Genetic Fuzzy Neural Networks and Their Applications

From the Publisher: This book presents a powerful hybrid intelligent system based on fuzzy logic, neural networks, genetic algorithms, and related intelligent techniques. The new compensatory genetic fuzzy neural networks have been widely used in fuzzy control, nonlinear system modeling, compression of a fuzzy rule base, expansion of a sparse fuzzy rule base, fuzzy knowledge discovery, time series prediction, fuzzy games, and pattern recognition. The proposed soft computing system is effective in performing both linguistic-word-level fuzzy reasoning and numerical-data-level information processing. The book also presents various novel soft computing techniques.

Type-2 fuzzy logic-based classifier fusion for support vector machines

As a machine-learning tool, support vector machines (SVMs) have been gaining popularity due to their promising performance. However, the generalization abilities of SVMs often rely on whether the selected kernel functions are suitable for real classification data. To lessen the sensitivity of different kernels in SVMs classification and improve SVMs generalization ability, this paper proposes a fuzzy fusion model to combine multiple SVMs classifiers. To better handle uncertainties existing in real classification data and in the membership functions (MFs) in the traditional type-1 fuzzy logic system (FLS), we apply interval type-2 fuzzy sets to construct a type-2 SVMs fusion FLS. This type-2 fusion architecture takes considerations of the classification results from individual SVMs classifiers and generates the combined classification decision as the output. Besides the distances of data examples to SVMs hyperplanes, the type-2 fuzzy SVMs fusion system also considers the accuracy information of individual SVMs. Our experiments show that the type-2 based SVM fusion classifiers outperform individual SVM classifiers in most cases. The experiments also show that the type-2 fuzzy logic-based SVMs fusion model is better than the type-1 based SVM fusion model in general.

Evolutionary fuzzy neural networks for hybrid financial prediction

In this paper, an evolutionary fuzzy neural network using fuzzy logic, neural networks (NNs), and genetic algorithms (GAs) is proposed for financial prediction with hybrid input data sets from different financial domains. A new hybrid iterative evolutionary learning algorithm initializes all parameters and weights in the five-layer fuzzy NN, then uses GA to optimize these parameters, and finally applies the gradient descent learning algorithm to continue the optimization of the parameters. Importantly, GA and the gradient descent learning algorithm are used alternatively in an iterative manner to adjust the parameters until the error is less than the required value. Unlike traditional methods, we not only consider the data of the prediction factor, but also consider the hybrid factors related to the prediction factor. Bank prime loan rate, federal funds rate and discount rate are used as hybrid factors to predict future financial values. The simulation results indicate that hybrid iterative evolutionary learning combining both GA and the gradient descent learning algorithm is more powerful than the previous separate sequential training algorithm described in.

Statistical fuzzy interval neural networks for currency exchange rate time series prediction

In this paper, the statistical fuzzy interval neural network with statistical interval input and output values is proposed to perform statistical fuzzy knowledge discovery and the currency exchange rate prediction. Time series data sets are grouped into time series data granules with statistical intervals. The statistical interval data sets including week-based averages, maximum errors of estimate and standard deviations are used to train the fuzzy interval neural network to discover fuzzy IF-THEN rules. The output of the fuzzy interval neural network is an interval value with certain percent confidence. Simulations are completed in terms of the exchange rates between US Dollar and other three currencies (Japanese Yen, British Pound and Hong Kong Dollar). The simulation results show that the fuzzy interval neural network can provide more tolerant prediction results.