D.S. Yeung

Network intrusion detection in covariance feature space

Detecting multiple and various network intrusions is essential to maintain the reliability of network services. The problem of network intrusion detection can be regarded as a pattern recognition problem. Traditional detection approaches neglect the correlation information contained in groups of network traffic samples which leads to their failure to improve the detection effectiveness. This paper directly utilizes the covariance matrices of sequential samples to detect multiple network attacks. It constructs a covariance feature space where the correlation differences among sequential samples are evaluated. Two statistical supervised learning approaches are compared: a proposed threshold based detection approach and a traditional decision tree approach. Experimental results show that both achieve high performance in distinguishing multiple known attacks while the threshold based detection approach offers an advantage of identifying unknown attacks. It is also pointed out that utilizing statistical information in groups of samples, especially utilizing the covariance information, will benefit the detection effectiveness.

Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees

Most fuzzy learning algorithms can generate simple fuzzy rules from a set of examples having the same type of attributes. However, the case for fuzzy rules in which their knowledge representation power has been enhanced by the inclusion of several parameters such as weight and certainty factor has not been addressed. It is important but difficult to automatically acquire fuzzy rules with parameters from a set of examples with mixed attributes. The paper presents an approach to handle mixed attributes and introduces the concept of degree of importance of each attribute-value contributing to the consequent of a given rule. Based on this new concept, a new heuristic for generating fuzzy decision trees with parameters is given and a set of weighted fuzzy rules with local weight and certainty factor are extracted from the trees. The advantages of the learning is initially verified by the Iris classification problem.

Learning capability in fuzzy Petri nets

Petri nets (PNs) have been widely used in modeling and analyzing many real applications such as computers, automatic control and management information systems, etc. The power of PNs comes from their ability to model and analyze the behaviors and states of systems (events) concurrently. Neural networks (NNs), on the other hand, were developed to handle and solve many linear and nonlinear complex problems by forming an association (relationship) between its input and output training patterns. It will be advantageous if a learning capability is incorporated into a fuzzy Petri net (FPN) which has the capability of both systems. In this paper, a FPN model which has learning capability is proposed. The purpose of including a learning facility in FPNs is that many parameters of a fuzzy expert system, included in fuzzy production rules (FPRs), once when it has been modeled by a FPN could be tuned. These parameters, including membership values, weights (local and global) and certainty factors etc., play important roles in capturing and representing complex domain expert knowledge. By comparing the artificial neural networks (ANN) with FPNs having learning capability, we have advantages such as: a) FPNs provide a transparent modeling and analyzing capability whereas ANN provides a black-box learning and no-analysis capability; b) FPNs representing a fuzzy expert system could be used to analyze the different inference states step-by-step; c) FPNs could tune parameters in a fuzzy expert system so that the overall system performance is improved.

Improving learning accuracy of fuzzy decision trees by hybrid neural networks

In the process of learning from examples with fuzzy representation, the higher learning accuracy is always expected. The paper proposes using a hybrid neural network to improve the learning accuracy of the fuzzy ID3 algorithm which is a popular and powerful method of fuzzy rule extraction without much computational effort. The proposed hybrid neural network corresponds to a fuzzy reasoning method in which the concept of local weights and global weights is employed. The time to consult with domain experts to adjust the weights for improving the learning accuracy will be greatly reduced due to the learning capability of the hybrid neural network. The synergy between fuzzy decision tree induction and a hybrid neural network offers new insight into the construction of hybrid intelligent systems.

Optimizing fuzzy knowledge base by genetic algorithms and neural networks

Fuzzy logic invented by L.A. Zadeh has been used to handle and represent information which is vague, uncertain and imprecise. Many fuzzy control systems and expert systems have been developed to capture operator knowledge and domain expert knowledge respectively. Fuzzy production rules (FPRs) have been used to capture and represent domain expert knowledge for years. To draw an accurate, reasonable and reliable conclusion in a fuzzy expert system, the knowledge base plays an important role and is the heart of this system. Once a fuzzy expert system (FES) has been built, we are faced with a large number of parameters which need to be tuned in order to improve the system performance in terms of the results (conclusions) obtained. Many approaches have been proposed to tune the parameters of this system. The parameters include membership functions, weights (local and global), and certainty factors, etc. In this paper, a method is proposed to tune some of these parameters using a genetic algorithm (GA) and a neural network (NN). The neural network is used to model and capture parameters on its connection weights and provide initial values of these parameters for the genetic algorithm to optimize. The result of such tuning is that the overall system performance is greatly improved and the tuning task could be done automatically. A fuzzy expert system which provides expert advice for computer professionals and computer science graduates in selecting an appropriate job is used to test the proposed method.

A problem of selecting optimal subset of fuzzy-valued features

Feature subset selection refers to a data mining enhancement technique which aims to reduce the number of features to be used. This reduction is expected to improve the performance of data mining algorithms to be used, in aspects of speed, accuracy and simplicity. Although there has been some work on feature subset selection, research into the theoretically computational complexity of this problem and on the optimal selection of fuzzy-valued feature subsets has not been carried out. This paper focuses on a problem called optimal fuzzy-valued feature subset selection (OFFSS) which is regarded as being important but difficult in machine learning and pattern recognition. The measure of the quality of a set of features is defined by the overall overlapping degree between two classes of examples and the size of feature subset. The main contributions of this paper are that: (1) the concept of fuzzy extension matrix is introduced; (2) the computational complexity of OFFSS is proved to be NP-hard; (3) a simple but powerful heuristic algorithm for OFFSS is given; and (4) the feasibility and simplicity of the proposed algorithm are demonstrated via applications of OFFSS to input selection of neuro-fuzzy systems and to fuzzy decision tree induction.

Fault Tolerant Differential Evolution Based Optimal Reactive Power Flow

Differential evolution (DE) is a new branch of evolutionary algorithms (EAs) and has been successfully applied to solve the optimal reactive power flow (ORPF) problems in power systems. Although DE can avoid premature convergence, large population is needed and the application of DE is limited in large-scale power systems. Grid computing, as a prevalent paradigm for resource-intensive scientific application, is expected to provide a computing platform with tremendous computational power to speed up the optimization process of DE. When implanting DE based ORPF on grid system, fault tolerance due to unstable environment and variation of grid is a significant issue to be considered. In this paper, a fault tolerant DE-based ORPF method is proposed. In this method, when the individuals are distributed to the grid for fitness evaluation, a proportion of individuals, which returns from the grid slowly or fails to return, are replaced with new individuals generated randomly according to some specific rules. This approach can deal with the fault tolerance and also maintain diversity of the population of DE. The superior performance of the proposed approach is verified by numerical simulations on the ORPF problem of the IEEE 118-bus standard power system

A rough set approach to selecting attributes for ordinal prediction

Rough set theory has been successfully applied in selecting attributes to improve the effectiveness in derivation of decision trees/rules for classification. When the classification involves ordinal classes, the rough set reduction process should take into consideration the ordering of the classes. In this paper we propose a new way of evaluating and finding reducts for ordinal classification.

Application of Fuzzy Decision Trees to Reservoir Recognition

This chapter reports a real application of fuzzy decision tree to a reservoir recognition in the logging area for oilfield exploration. Reservoir fluid recognition is an important but difficult task in providing a comprehensive explanation for logging. A good recognition method can provide reliable evidence for building a standard of explanation in a region. Since there is much vagueness in the reservoir fluid recognition and there are considerable differences of geological structure in different regions, it is very difficult to establish a uniform mathematical model to recognize the reservoir. The commonly used methods for reservoir recognition include empirical formula, synthetic evaluation, fuzzy clustering, etc. Unfortunately, these methods fail to meet many applications’ requirements. For example, the empirical formula and synthetic evaluation methods could not handle fuzzy or vague data while the fuzzy clustering could not give a good recognition of oil-water layer. By applying the fuzzy decision tree induction method to the problem of reservoir recognition in an oilfield of northern China, we find the recognition results encouraging.