Zongyuan Zhao

Investigation and improvement of multi-layer perceptron neural networks for credit scoring

We present an Average Random Choosing method which increases 0.04 classification accuracy.Investigate different MLP models and get the best model with accuracy of 87%.Accuracy increases when the model has more hidden neurons. Multi-Layer Perceptron (MLP) neural networks are widely used in automatic credit scoring systems with high accuracy and efficiency. This paper presents a higher accuracy credit scoring model based on MLP neural networks that have been trained with the back propagation algorithm. Our work focuses on enhancing credit scoring models in three aspects: (i) to optimise the data distribution in datasets using a new method called Average Random Choosing; (ii) to compare effects of training-validation-test instance numbers; and (iii) to find the most suitable number of hidden units. We trained 34 models 20 times with different initial weights and training instances. Each model has 6 to 39 hidden units with one hidden layer. Using the well-known German credit dataset we provide test results and a comparison between models, and we get a model with a classification accuracy of 87%, which is higher by 5% than the best result reported in the relevant literature of recent years. We have also proved that our optimisation of dataset structure can increase a models accuracy significantly in comparison with traditional methods. Finally, we summarise the tendency of scoring accuracy of models when the number of hidden units increases. The results of this work can be applied not only to credit scoring, but also to other MLP neural network applications, especially when the distribution of instances in a dataset is imbalanced.

A New Evolutionary Algorithm for Extracting a Reduced Set of Interesting Association Rules

Data mining techniques involve extracting useful, novel and interesting patterns from large data sets. Traditional association rule mining algorithms generate a huge number of unnecessary rules because of using support and confidence values as a constraint for measuring the quality of generated rules. Recently, several studies defined the process of extracting association rules as a multi-objective problem allowing researchers to optimize different measures that can present in different degrees depending on the data sets used. Applying evolutionary algorithms to noisy data of a large data set, is especially useful for automatic data processing and discovering meaningful and significant association rules. From the beginning of the last decade, multi-objective evolutionary algorithms are gradually becoming more and more useful in data mining research areas. In this paper, we propose a new multi-objective evolutionary algorithm, MBAREA, for mining useful Boolean association rules with low computational cost. To accomplish this our proposed method extends a recent multi-objective evolutionary algorithm based on a decomposition technique to perform evolutionary learning of a fitness value of each rule, while introducing a best population and a class based mutation method to store all the best rules obtained at some point of intermediate generation of a population and improving the diversity of the obtained rules. Moreover, this approach maximizes two objectives such as performance and interestingness for getting rules which are useful, easy to understand and interesting. This proposed algorithm is applied to different real world data sets to demonstrate the effectiveness of the proposed approach and the result is compared with existing evolutionary algorithm based approaches.

Comparative analysis of genetic based approach and Apriori algorithm for mining maximal frequent item sets

In the data mining research area, discovering frequent item sets is an important issue and key factor for mining association rules. For large datasets, a huge amount of frequent patterns are generated for a low support value, which is a major challenge in frequent pattern mining tasks. A Maximal frequent pattern mining task helps to resolve this problem since a maximal frequent pattern contains information about a large number of small frequent sub patterns. For this study we have developed a genetic based approach to find maximal frequent patterns using a user defined threshold value as a constraint. To optimize the search problems, a genetic algorithm is one of the best choices which mimics the natural selection procedure and considers global search mechanism which is good for searching solution especially when the search space is large. The use of evolutionary algorithm is also effective for undetermined solutions. Therefore, this approach uses a genetic algorithm to find maximal frequent item sets from different sorts of data sets. A low support value generates some large patterns which contain the information about huge amount of small frequent sub patterns that could be useful for mining association rules. We have applied this genetic based approach for different real data sets as well as synthetic data sets. The experimental results show that our proposed approach evaluates less nodes than the number of candidate item sets considered by Apriori algorithm, especially when the support value is set low.

A new multiple seeds based genetic algorithm for discovering a set of interesting Boolean association rules

A new multiple seeds based genetic algorithm is proposed.This method relies on generating multiple seeds from different domains.This scheme introduces m-domain model and m-seeds selection process.Multiple seeds are used to generate an effective initial population.The experiments were conducted to show the effeciency of the proposed method. Association rule mining algorithms mostly use a randomly generated single seed to initialize a population without paying attention to the effectiveness of that population in evolutionary learning. Recently, research has shown significant impact of the initial population on the production of good solutions over several generations of a genetic algorithm. Single seed based genetic algorithms suffer from the following major challenges (1) solutions of a genetic algorithm are varied, since different seeds generate different initial population, (2) difficulty in defining a good seed for a specific application. To avoid these problems, in this paper we propose the MSGA, a new multiple seeds based genetic algorithm which generates multiple seeds from different domains of a solution space to discover high quality rules from a large data set. This scheme introduces m-domain model and m-seeds selection process through which the whole solution space is subdivided into m- number of same size domains, selecting a seed from each domain. Use of these seeds enables this method to generate an effective initial population for evolutionary learning of the fitness value of each rule. As a result, strong searching efficiency is obtained at the beginning of the evolution, achieving fast convergence. The MSGA is tested with different mutation and crossover operators for mining interesting Boolean association rules from four real world data sets. The results are compared to different single seeds based genetic algorithms under the same conditions.

A New approach to utilize augmented reality on precision livestock farming

This paper proposes a new method that utilizes AR to assist pasture-based dairy farmers identify and locate animal within large herds. Our proposed method uses GPS collars on cows and digital camera and on-board GPS on a mobile device to locate a selected cow and show the behavioral and other associated key metrics on our mobile application. The augmented cow’s information shown on real scene video steam will help users (farmers) manage their animals with respect to welfare, health, and management interventions. By integrating GPS data with computer vision (CV) and machine learning, our mobile AR application has two major functions: 1. Searching a cow by its unique ID, and 2. Displaying information associated with a selected cow visible on screen. Our proof-of-concept application shows the potential of utilizing AR in precision livestock farming.

Multiple Seeds Based Evolutionary Algorithm for Mining Boolean Association Rules

Most of the association rule mining algorithms use a single seed for initializing a population without paying attention to the effectiveness of an initial population in an evolutionary learning. Recently, researchers show that an initial population has significant effects on producing good solutions over several generations of a genetic algorithm. There are two significant challenges raised by single seed based genetic algorithms for real world applications: 1 solutions of a genetic algorithm are varied, since different seeds generate different initial populations, 2 it is a hard process to define an effective seed for a specific application. To avoid these problems, in this paper we propose a new multiple seeds based genetic algorithm MSGA which generates multiple seeds from different domains of a solution space to discover high quality rules from a large data set. This approach introduces m-domain model and m-seeds selection process through which the whole solution space is subdivided into m-number of same size domains and from each domain it selects a seed. By using these seeds, this method generates an effective initial population to perform an evolutionary learning of the fitness value of each rule. As a result, this method obtains strong searching efficiency at the beginning of the evolution and achieves fast convergence along with the evolution. MSGA is tested with different mutation and crossover operators for mining interesting Boolean association rules from different real world data sets and compared the results with different single seeds based genetic algorithms.

Discovery of Interesting Association Rules Using Genetic Algorithm with Adaptive Mutation

Association rule mining is the process of discovering useful and interesting rules from large datasets. Traditional association rule mining algorithms depend on a user specified minimum support and confidence values. These constraints introduce two major challenges in real world applications: exponential search space and a dataset dependent minimum support value. Data analyzers must specify suitable dataset dependent minimum support value for mining tasks although they might have no knowledge regarding the dataset and these algorithms generate a huge number of unnecessary rules. To overcome these kinds of problems, recently several researchers framed association rule mining problem as a multi objective problem. In this paper, we propose ARMGAAM, a new evolutionary algorithm, which generates a reduced set of association rules and optimizes several measures that are present in different degrees based on the datasets are used. To accomplish this, our method extends the existing ARMGA model for performing an evolutionary learning, while introducing a reinitialization process along with an adaptive mutation method. Moreover, this approach maximizes conditional probability, lift, net confidence and performance in order to obtain a set of rules which are interesting, useful and easy to comprehend. The effectiveness of the proposed method is validated over a few real world datasets.