Man Leung Wong

Using evolutionary programming and minimum description length principle for data mining of Bayesian networks

We have developed a new approach to learning Bayesian network structures based on the minimum description length (MDL) principle and evolutionary programming. It employs a MDL metric, which is founded on information theory, and integrates a knowledge-guided genetic operator for the optimization in the search process.

Machine Learning for Direct Marketing Response Models: Bayesian Networks with Evolutionary Programming

Machine learning methods are powerful tools for data mining with large noisy databases and give researchers the opportunity to gain new insights into consumer behavior and to improve the performance of marketing operations. To model consumer responses to direct marketing, this study proposes Bayesian networks learned by evolutionary programming. Using a large direct marketing data set, we tested the endogeneity bias in the recency, frequency, monetary value (RFM) variables using the control function approach; compared the results of Bayesian networks with those of neural networks, classification and regression tree (CART), and latent class regression; and applied a tenfold cross-validation. The results suggest that Bayesian networks have distinct advantages over the other methods in accuracy of prediction, transparency of procedures, interpretability of results, and explanatory insight. Our findings lend strong support to Bayesian networks as a robust tool for modeling consumer response and other marketing problems and for assisting management decision making.

Evolutionary Computing on Consumer Graphics Hardware

We propose implementing a parallel EA on consumer graphics cards, which we can find in many PCs. This lets more people use our parallel algorithm to solve large-scale, real-world problems such as data mining. Parallel evolutionary algorithms run on consumer-grade graphics hardware achieve better execution times than ordinary evolutionary algorithms and offer greater accessibility than those run on high-performance computers

An efficient data mining method for learning Bayesian networks using an evolutionary algorithm-based hybrid approach

Given the explosive growth of data collected from current business environment, data mining can potentially discover new knowledge to improve managerial decision making. This paper proposes a novel data mining approach that employs an evolutionary algorithm to discover knowledge represented in Bayesian networks. The approach is applied successfully to handle the business problem of finding response models from direct marketing data. Learning Bayesian networks from data is a difficult problem. There are two different approaches to the network learning problem. The first one uses dependency analysis, while the second one searches good network structures according to a metric. Unfortunately, both approaches have their own drawbacks. Thus, we propose a novel hybrid algorithm of the two approaches, which consists of two phases, namely, the conditional independence (CI) test and the search phases. In the CI test phase, dependency analysis is conducted to reduce the size of the search space. In the search phase, good Bayesian network models are generated by using an evolutionary algorithm. A new operator is introduced to further enhance the search effectiveness and efficiency. In a number of experiments and comparisons, the hybrid algorithm outperforms MDLEP, our previous algorithm which uses evolutionary programming (EP) for network learning, and other network learning algorithms. We then apply the approach to two data sets of direct marketing and compare the performance of the evolved Bayesian networks obtained by the new algorithm with those by MDLEP, the logistic regression models, the na/spl inodot//spl uml/ve Bayesian classifiers, and the tree-augmented na/spl inodot//spl uml/ve Bayesian network classifiers (TAN). In the comparison, the new algorithm outperforms the others.

Parallel evolutionary algorithms on graphics processing unit

Evolutionary algorithms (EAs) are effective and robust methods for solving many practical problems such as feature selection, electrical circuit synthesis, and data mining. However, they may execute for a long time for some difficult problems, because several fitness evaluations must be performed. A promising approach to overcome this limitation is to parallelize these algorithms. In this paper, we propose to implement a parallel EA on consumer-level graphics cards. We perform experiments to compare our parallel EA with an ordinary EA and demonstrate that the former is much more effective than the latter. Since consumer-level graphics cards are available in ubiquitous personal computers and these computers are easy to use and manage, more people are able to use our parallel algorithm to solve their problems encountered in real-world applications.

An efficient self-organizing map designed by genetic algorithms for the traveling salesman problem

As a typical combinatorial optimization problem, the traveling salesman problem (TSP) has attracted extensive research interest. In this paper, we develop a self-organizing map (SOM) with a novel learning rule. It is called the integrated SOM (ISOM) since its learning rule integrates the three learning mechanisms in the SOM literature. Within a single learning step, the excited neuron is first dragged toward the input city, then pushed to the convex hull of the TSP, and finally drawn toward the middle point of its two neighboring neurons. A genetic algorithm is successfully specified to determine the elaborate coordination among the three learning mechanisms as well as the suitable parameter setting. The evolved ISOM (eISOM) is examined on three sets of TSP to demonstrate its power and efficiency. The computation complexity of the eISOM is quadratic, which is comparable to other SOM-like neural networks. Moreover, the eISOM can generate more accurate solutions than several typical approaches for TSP including the SOM developed by Budinich, the expanding SOM, the convex elastic net, and the FLEXMAP algorithm. Though its solution accuracy is not yet comparable to some sophisticated heuristics, the eISOM is one of the most accurate neural networks for the TSP.

A new type of nonlinear integrals and the computational algorithm

In information fusion, aggregations with various backgrounds require a variety of integrals to handle. These integrals are generally nonlinear since the set functions used are nonadditive in many real problems. In this study, the set functions considered are nonnegative and vanishing at the empty set. They are a class of set functions including fuzzy measures and even imprecise probabilities. A new type of nonlinear integrals with respect to such a set function for nonnegative functions is introduced and its primary properties are detailed. These type of integrals has a natural explanation and, therefore, has wide applicability. We also show a comparison between the newly introduced nonlinear integral and other nonlinear integrals, such as the Choquet integral, the natural extension in the theory of imprecise probabilities, and the common pan-integral. With a flowchart, the algorithm for calculating the integral is given in this paper when the universe of discourse (the set of all information sources) is finite.

Parallel multi-objective evolutionary algorithms on graphics processing units

Most real-life optimization problems or decision-making problems are multi-objective in nature, since they normally have several (possibly conflicting) objectives that must be satisfied at the same time. Multi-Objective Evolutionary Algorithms (MOEAs) have been gaining increasing attention among researchers and practitioners. However, they may execute for a long time for some difficult problems, because several evaluations must be performed. Moreover, the non-dominance checking and the non-dominated selection procedures are also very time consuming. From our experiments, more than 99% of the execution time is used in performing the two procedures. A promising approach to overcome this limitation is to parallelize these algorithms. In this paper, we propose a parallel MOEA on consumer-level Graphics Processing Units (GPU). We perform many experiments on two-objective and three-objective benchmark problems to compare our parallel MOEA with a sequential MOEA and demonstrate that the former is much more efficient than the latter.

Expanding self-organizing map for data visualization and cluster analysis

The Self-Organizing Map (SOM) is a powerful tool in the exploratory phase of data mining. It is capable of projecting high-dimensional data onto a regular, usually 2- dimensional grid of neurons with good neighborhood preservation between two spaces. However, due to the dimensional conflict, the neighborhood preservation cannot always lead to perfect topology preservation. In this paper, we establish an Expanding SOM (ESOM) to preserve better topology between the two spaces. Besides the neighborhood relationship, our ESOM can detect and preserve an ordering relationship using an expanding mechanism. The computational complexity of the ESOM is comparable with that of the SOM. Our experiment results demonstrate that the ESOM constructs better mappings than the classic SOM, especially, in terms of the topological error. Furthermore, clustering results generated by the ESOM are more accurate than those obtained by the SOM.

Medical data mining using evolutionary computation.

In this paper, we introduce a system for discovering medical knowledge by learning Bayesian networks and rules. Evolutionary computation is used as the search algorithm. The Bayesian networks can provide an overall structure of the relationships among the attributes. The rules can capture detailed and interesting patterns in the database. The system is applied to real-life medical databases for limb fracture and scoliosis. The knowledge discovered provides insights to and allows better understanding of these two medical domains.