Qiangfu Zhao

Scaling up fast evolutionary programming with cooperative coevolution

Evolutionary programming (EP) has been applied with success to many numerical and combinatorial optimization problems in recent years. However, most analytical and experimental results on EP have been obtained using low-dimensional problems. It is interesting to know whether the empirical results obtained from the low-dimensional problems still hold for high-dimensional cases. It was discovered that neither classical EP (CEP) nor fast EP (FEP) performed satisfactorily for some large-scale problems. The paper shows empirically that FEP with cooperative coevolution (FEPCC) can speed up convergence rates on the large-scale problems whose dimension ranges from 100 to 1000. Cooperative coevolution adopts the divide-and-conquer strategy. It divides the system into many modules, and evolves each module separately and cooperatively. The results of FEPCC on the problems investigated here are something of a surprise. The time used by FEPCC to find a near optimal solution appears to scale linearly; that is, the time used seems to go up linearly as the dimensionality of the problems studied increases.

A simple design of FIR filters with powers-of-two coefficients

A design algorithm for finite-impulse response (FIR) filters with powers-of-two coefficients (2PFIR filters) is presented. The algorithm enables the design of 2PFIR filters with sharp cutoff. The 2PFIR filters are attractive for high-speed operation and simplification of hardware. However, the optimal or suboptimal design of 2PFIR filters with sharp cutoff have been difficult in the minimax sense. The algorithm proposed is composed of two methods. The first is suboptimal design, which preserves a proportional relation between the coefficients of the conventional FIR filters and the 2PFIR filters. This method is referred to as the proportional relation-preserve (PRP) method. The second is the application of the simple symmetric-sharpening (SSS) method. The SSS method is applied when the PRP method cannot realize the given filter specifications. Using the PRP and SSS methods, the 2PFIR with filter length N>200 can be easily designed. >

Fast pattern detection using normalized neural networks and cross-correlation in the frequency domain

Neural networks have shown good results for detection of a certain pattern in a given image. In our previous work, a fast algorithm for object/face detection was presented. Such algorithm was designed based on cross-correlation in the frequency domain between the input image and the weights of neural networks. Our previous work also solved the problem of local subimage normalization in the frequency domain. In this paper, the effect of image normalization on the speedup ratio of pattern detection is presented. Simulation results show that local subimage normalization through weight normalization is faster than subimage normalization in the spatial domain. Moreover, the overall speedup ratio of the detection process is increased as the normalization of weights is done offline.

Evolutionary design of neural network tree-integration of decision tree, neural network and GA

Decision tree (DT) is one of the most popular approaches for machine learning. Using DTs, we can extract comprehensible decision rules, and make decisions based only on useful features. The drawback is that, once a DT is designed, there is no free parameter for further development. On the contrary, a neural network (NN) is adaptable or learnable, but the number of free parameters is usually too large to be determined efficiently. To have the advantages of both approaches, it is important to combine them together. Among many ways for combining NNs and DTs, this paper introduces a neural network tree (NNTree). An NNTree is a decision tree with each node being an expert neural network (ENN). The overall tree structure can be designed by following the same procedure as used in designing a conventional DT. Each node (an ENN) can be designed using genetic algorithms (GAs). Thus, the NNTree also provides a way for integrating DT, NN and GA. Through experiments with a digit recognition problem we show that NNTrees are more efficient than traditional DTs in the sense that higher recognition rate can be achieved with less nodes. Further more, if the fitness function for each node is defined properly, better generalization ability can also be achieved.

Evolutionary learning of nearest-neighbor MLP

The nearest-neighbor multilayer perceptron (NN-MLP) is a single-hidden-layer network suitable for pattern recognition. To design an NN-MLP efficiently, this paper proposes a new evolutionary algorithm consisting of four basic operations: recognition, remembrance, reduction, and review. Experimental results show that this algorithm can produce the smallest or nearly smallest networks from random initial ones.

Fast time delay neural networks.

This paper presents a new approach to speed up the operation of time delay neural networks. The entire data are collected together in a long vector and then tested as a one input pattern. The proposed fast time delay neural networks (FTDNNs) use cross correlation in the frequency domain between the tested data and the input weights of neural networks. It is proved mathematically and practically that the number of computation steps required for the presented time delay neural networks is less than that needed by conventional time delay neural networks (CTDNNs). Simulation results using MATLAB confirm the theoretical computations.

Generation of comprehensible decision trees through evolution of training data

In machine learning, decision trees (DTs) are usually considered comprehensible because a reasoning process can be given for each conclusion. When the data set is large, however, the DTs obtained may become very large, and they are no longer comprehensible. To increase the comprehensibility of DTs, we have proposed several methods. For example, we have tried to evolve DTs using genetic programming (GP), with tree size as the secondary fitness measure; we have tried to initialize GP using results obtained by C4.5; and we have also tried to introduce the divide-and-conquer concept in GP, but all results obtained are still not good enough. Up to now we have tried to design good DTs from given fixed data. In this paper, we look at the problem from a different point of view. The basic idea is to evolve a small data set that can cover the domain knowledge as good as possible. From this data set, a small but good DT can be designed. The validity of the new algorithm is verified through several experiments.

Cooperative co-evolutionary algorithm-how to evaluate a module?

When we talk about co-evolution, we often consider it as competitive co-evolution (CompCE). Examples include co-evolution of training data and neural networks, co-evolution of game players, and so on. Recently, several researchers have studied another kind of co-evolution- cooperative co-evolution (CoopCE). While CompCE tries to get more competitive individuals through evolution, the goal of CoopCE is to find individuals from which better systems can be constructed. The basic idea of CoopCE is to divide-and-conquer: divide a large system into many modules, evolve the modules separately, and then combine them together again to form the whole system. Depending on how to divide-and-conquer, different cooperative co-evolutionary algorithms (CoopCEAs) have been proposed in the literature. Results obtained so far strongly support the usefulness of CoopCEAs. To study the CoopCEAs systematically, we proposed a society model, which is a common framework of most existing CoopCEAs. From this model, we can see that there are still many open problems related to CoopCEAs. To make CoopCEAs generally useful, it is necessary to study and solve these problems. In this paper, we focus the discussion on evaluation of the modules-which is one of the key point in using CoopCEAs. To be concrete, we will apply the model to evolutionary learning of RBF-neural networks, and show the effectiveness of different evaluation methods through experiments.

A study on evolutionary design of binary decision trees

For pattern recognition, decision trees (DTs) are more efficient than neural networks (NNs) for two reasons. First, the computations in making decisions are simpler. Second, important features can be selected automatically during the design process. However, the DTs are not adaptable. This problem can be avoided by mapping a DT to an NN. This mapping not only makes a DT adaptable, but also provides a systematic way for determining the NN structure. In addition, since the features are well selected, the NN obtained from this mapping may have much fewer connections than those designed directly. The key point here is to design a DT which is as small as possible. We study the evolutionary design of the decision trees, and investigate some methods to improve the design efficiency.

Fast Neural Implementation of PCA for Face Detection

Principle component analysis (PCA) has many different important applications especially in pattern detection such as face detection / recognition. Therefore, for real time applications, the response time is required to be as very small as possible. In this paper, new implementation of PCA for fast face detection is presented. Such new implementation is designed based on cross correlation in the frequency domain between the input image and eigenvalues (weights). Simulation results show that the proposed implementation of PCA is faster than conventional one.