Xizhao Wang

Fuzziness based sample categorization for classifier performance improvement

This paper investigates a relationship between the fuzziness of a classifier and the misclassification rate of the classifier on a group of samples. For a given trained classifier that outputs a membership vector, we demonstrate experimentally that samples with higher fuzziness outputted by the classifier mean a bigger risk of misclassification. We then propose a fuzziness category based divide-and-conquer strategy which separates the high-fuzziness samples from the low fuzziness samples. A particular technique is used to handle the high-fuzziness samples for promoting the classifier performance. The reasonability of the approach is theoretically explained and its effectiveness is experimentally demonstrated.

Fuzziness based semi-supervised learning approach for intrusion detection system

Countering cyber threats, especially attack detection, is a challenging area of research in the field of information assurance. Intruders use polymorphic mechanisms to masquerade the attack payload and evade the detection techniques. Many supervised and unsupervised learning approaches from the field of machine learning and pattern recognition have been used to increase the efficacy of intrusion detection systems (IDSs). Supervised learning approaches use only labeled samples to train a classifier, but obtaining sufficient labeled samples is cumbersome, and requires the efforts of domain experts. However, unlabeled samples can easily be obtained in many real world problems. Compared to supervised learning approaches, semi-supervised learning (SSL) addresses this issue by considering large amount of unlabeled samples together with the labeled samples to build a better classifier. This paper proposes a novel fuzziness based semi-supervised learning approach by utilizing unlabeled samples assisted with supervised learning algorithm to improve the classifiers performance for the IDSs. A single hidden layer feed-forward neural network (SLFN) is trained to output a fuzzy membership vector, and the sample categorization (low, mid, and high fuzziness categories) on unlabeled samples is performed using the fuzzy quantity. The classifier is retrained after incorporating each category separately into the original training set. The experimental results using this technique of intrusion detection on the NSL-KDD dataset show that unlabeled samples belonging to low and high fuzziness groups make major contributions to improve the classifiers performance compared to existing classifiers e.g., naive bayes, support vector machine, random forests, etc.

Fuzzy nonlinear regression analysis using a random weight network

Modeling a fuzzy-in fuzzy-out system where both inputs and outputs are uncertain is of practical and theoretical importance. Fuzzy nonlinear regression (FNR) is one of the approaches used most widely to model such systems. In this study, we propose the use of a Random Weight Network (RWN) to develop a FNR model called FNRRWN, where both the inputs and outputs are triangular fuzzy numbers. Unlike existing FNR models based on back-propagation (BP) and radial basis function (RBF) networks, FNRRWN does not require iterative adjustment of the network weights and biases. Instead, the input layer weights and hidden layer biases of FNRRWN are selected randomly. The output layer weights for FNRRWN are calculated analytically based on a derived updating rule, which aims to minimize the integrated squared error between α-cut sets that correspond to the predicted fuzzy outputs and target fuzzy outputs, respectively. In FNRRWN, the integrated squared error is solved approximately by Riemann integral theory. The experimental results show that the proposed FNRRWN method can effectively approximate a fuzzy-in fuzzy-out system. FNRRWN obtains better prediction accuracy in a lower computational time compared with existing FNR models based on BP and RBF networks.

Exact and approximate algorithms for discounted {0-1} knapsack problem

The Discounted {0-1} Knapsack Problem (D{0-1}KP) is an extension of the classical 0-1 knapsack problem (0-1 KP) that consists of selecting a set of item groups where each group includes three items and at most one of the three items can be selected. The D{0-1}KP is more challenging than the 0-1 KP because four choices of items in an item group diversify the selection of the items. In this paper, we systematically studied the exact and approximate algorithms for solving D{0-1}KP. Firstly, a new exact algorithm based on the dynamic programming and its corresponding fully polynomial time approximation scheme were designed. Secondly, a 2-approximation algorithm for D{0-1}KP was developed. Thirdly, a greedy repair algorithm for handling the infeasible solutions of D{0-1}KP was proposed and we further studied how to use binary particle swarm optimization and greedy repair algorithm to solve the D{0-1}KP. Finally, we used four different kinds of instances to compare the approximate rate and solving time of the exact and approximate algorithms. The experimental results and theoretical analysis showed that the approximate algorithms worked well for D{0-1}KP instances with large value, weight, and size coefficients, while the exact algorithm was good at solving D{0-1}KP instances with small value, weight, and size coefficients.

Voting-based instance selection from large data sets with MapReduce and random weight networks

Instance selection is an important preprocessing step in machine learning. By choosing a subset of a data set, it achieves the same performance of a machine learning algorithm as if the whole data set is used, and it enables a machine learning algorithm to be feasible for and to work effectively with large data sets. Based on voting mechanism, this paper proposes a large data sets instance selection algorithm with MapReduce and random weight networks (RWNs). Firstly, the proposed algorithm employs the Map of MapReduce to partition the large data sets into some small subsets, and deploys them to different cloud computing nodes. Secondly, the informative instances are selected in parallel with an instance selection algorithm. Thirdly, the Reduce of MapReduce is used to collect the selected instances from different cloud computing nodes and a selected instance subset is obtained. The previous three processes are repeated p times (p is a parameter defined by the user), and p instance subsets are obtained. Finally, the voting method is used to select the most informative instances from the p subsets. The random weight network classifier is trained with the selected instance subset, and the testing accuracy is verified on the testing set. The proposed algorithm is experimentally compared with three state-of-the-art approaches which are CNN, ENN and RNN. The experimental results show that the proposed algorithm is effective and efficient.

A review on neural networks with random weights

In big data fields, with increasing computing capability, artificial neural networks have shown great strength in solving data classification and regression problems. The traditional training of neural networks depends generally on the error back propagation method to iteratively tune all the parameters. When the number of hidden layers increases, this kind of training has many problems such as slow convergence, time consuming, and local minima. To avoid these problems, neural networks with random weights (NNRW) are proposed in which the weights between the hidden layer and input layer are randomly selected and the weights between the output layer and hidden layer are obtained analytically. Researchers have shown that NNRW has much lower training complexity in comparison with the traditional training of feed-forward neural networks. This paper objectively reviews the advantages and disadvantages of NNRW model, tries to reveal the essence of NNRW, gives our comments and remarks on NNRW, and provides some useful guidelines for users to choose a mechanism to train a feed-forward neural network.

A novel binary artificial bee colony algorithm for the set-union knapsack problem

Abstract This article investigates how to employ artificial bee colony algorithm to solve Set-Union Knapsack Problem (SUKP). A mathematical model of SUKP, which is to be easily solved by evolutionary algorithms, is developed. A novel binary artificial bee colony algorithm (BABC) is also proposed by adopting a mapping function. Furthermore, a greedy repairing and optimization algorithm (S-GROA) for handling infeasible solutions by employing evolutionary technique to solve SUKP is proposed. The consolidation of S-GROA and BABC brings about a new approach to solving SUKP. Extensive experiments are conducted upon benchmark datasets for evaluating the performance of our proposed models. The results verify that the proposed approach is significantly superior to the baseline evolutionary algorithms for solving SUKP such as A-SUKP, ABC bin and binDE in terms of both time complexity and solution performance.

A ranking-based adaptive artificial bee colony algorithm for global numerical optimization

Abstract The artificial bee colony (ABC) algorithm is a powerful population-based metaheuristic for global numerical optimization and has been shown to compete with other swarm-based algorithms. However, ABC suffers from a slow convergence speed. To address this issue, the natural phenomenon in which good individuals always have good genes and thus should have more opportunities to generate offspring is the inspiration for this paper. We propose a ranking-based adaptive ABC algorithm (ARABC). Specifically, in ARABC, food sources are selected by bees to search, and the parent food sources used in the solution search equation are all chosen based on their rankings. The higher a food source is ranked, the more opportunities it will have to be selected. Moreover, the selection probability of the food source is based on the corresponding ranking, which is adaptively adjusted according to the status of the population evolution. To evaluate the performance of ARABC, we compare ARABC with other ABC variants and state-of-the-art differential evolution and particle swarm optimization algorithms based on a number of benchmark functions. The experimental results show that ARABC is significantly better than the algorithms to which it was compared.

Monotonic classification extreme learning machine

Monotonic classification problems mean that both feature values and class labels are ordered and monotonicity relationships exist between some features and the decision label. Extreme Learning Machine (ELM) is a single-hidden layer feedforward neural network with fast training rate and good generalization capability, but due to the existence of training error, ELM cannot be directly used to handle monotonic classification problems. This work proposes a generalization of ELM for processing the monotonic classification, named as Monotonic Classification Extreme Learning Machine (MCELM) in which the monotonicity constraints are imposed to the original ELM model. Mathematically, MCELM is a quadratic programming problem in which the monotonicity relationships are considered as constraints and the training error is the objective to be minimized. The mathematical model of MCELM not only can make the generated classifier monotonic but also can minimize the classification error. MCELM does not need to tune parameters iteratively, and therefore, keeps the advantage of extremely fast training which is the essential characteristic of ELM. MCELM does not require that the monotonic relationships existing between features and the output are consistent, which essentially relaxes the assumption of consistent monotonicity used in most existing approaches to handling monotonic classification problems. In comparison with exiting approaches to handling monotonic classification, MCELM can indeed generate a monotonicity-reserving classifier which experimentally shows a much better generalization capability on both artificial and real world datasets.

Segmenting time series with connected lines under maximum error bound

The error-bounded Piecewise Linear Approximation (PLA) is to approximate the stream data by lines such that the approximation error at each point does not exceed a pre-defined error. In this paper, we focus on the version of PLA problem that generates connected lines in the segmentation for smooth approximation. We provide a new linear-time algorithm for the problem that outperform two of the existing methods with less number of connected segments. Our extensive experiments, on both real and synthetic data sets, indicate that our proposed algorithms are practically efficient.