Leandro M. Almeida

A multi-objective memetic and hybrid methodology for optimizing the parameters and performance of artificial neural networks

The use of artificial neural networks implies considerable time spent choosing a set of parameters that contribute toward improving the final performance. Initial weights, the amount of hidden nodes and layers, training algorithm rates and transfer functions are normally selected through a manual process of trial-and-error that often fails to find the best possible set of neural network parameters for a specific problem. This paper proposes an automatic search methodology for the optimization of the parameters and performance of neural networks relying on use of Evolution Strategies, Particle Swarm Optimization and concepts from Genetic Algorithms corresponding to the hybrid and global search module. There is also a module that refers to local searches, including the well-known Multilayer Perceptrons, Back-propagation and the Levenberg-Marquardt training algorithms. The methodology proposed here performs the search using the aforementioned parameters in an attempt to optimize the networks and performance. Experiments were performed and the results proved the proposed method to be better than trial-and-error and other methods found in the literature.

Hybrid Training Method for MLP: Optimization of Architecture and Training

The performance of an artificial neural network (ANN) depends upon the selection of proper connection weights, network architecture, and cost function during network training. This paper presents a hybrid approach (GaTSa) to optimize the performance of the ANN in terms of architecture and weights. GaTSa is an extension of a previous method (TSa) proposed by the authors. GaTSa is based on the integration of the heuristic simulated annealing (SA), tabu search (TS), genetic algorithms (GA), and backpropagation, whereas TSa does not use GA. The main advantages of GaTSa are the following: a constructive process to add new nodes in the architecture based on GA, the ability to escape from local minima with uphill moves (SA feature), and faster convergence by the evaluation of a set of solutions (TS feature). The performance of GaTSa is investigated through an empirical evaluation of 11 public-domain data sets using different cost functions in the simultaneous optimization of the multilayer perceptron ANN architecture and weights. Experiments demonstrated that GaTSa can also be used for relevant feature selection. GaTSa presented statistically relevant results in comparison with other global and local optimization techniques.

Topology optimization for artificial neural networks using differential evolution

Backpropagation (BP) training algorithm is the main algorithm for training feedforward artificial neural networks (ANNs). BP is based on gradient descent, thus it converges to a local optimum in the region of the initial solution. Meanwhile, the evolutionary algorithms (EAs) always look for global optimum, however their ability of local search is not as good as the BP algorithm. This paper presents a hybrid system that uses differential evolution with global and local neighborhoods (DEGL), which is a variant of differential evolution (DE), to search for a suitable architecture and a near-optimal set of initial connection weights, and then performs the Levenberg-Marquadt training algorithm, which is a more robust variation of BP, to perform local search from these initial weights. Finally, it is performed a comparison of the performance of the hybrid system DEGL+ANN with the hybrid system DE+ANN and the raw RNA, for classification problems using machine learning benchmarks.

An Evolutionary Approach for Tuning Artificial Neural Network Parameters

The widespread use of artificial neural networks and the difficult work regarding the correct specification (tuning) of parameters for a given problem are the main aspects that motivated the approach purposed in this paper. This approach employs an evolutionary search to perform the simultaneous tuning of initial weights, transfer functions, architectures and learning rules (learning algorithm parameters). Experiments were performed and the results demonstrate that the method is able to find efficient networks with satisfactory generalization in a shorter search time.

An improved method for automatically searching near-optimal artificial Neural Networks

This paper describes an improved version of a method that automatically searches near-optimal multi-layer feedforward artificial neural networks using genetic algorithms. This method employs an evolutionary search for simultaneous choices of initial weights, transfer functions, architectures and learning rules. Experimental results have shown that the developed method can produce compact, efficient networks with a satisfactory generalization power and with shorter training times when compared to other methods found in the literature.

An Intelligent Monitoring System for Natural Gas Odorization

In this paper, we present the design of an intelligent monitoring system consisting of physical sensors and intelligent software for the automatic identification of the concentration of natural gas odorants in the environment. An optical-based sensor array was proposed comprising the hardware module. The software module employs wavelets filters and artificial neural networks to recognize the concentration of odorant in a natural gas sample. The objective is to help the natural gas odorization process by means of end point monitoring through the recognizing of the odorant concentration. The recognizing process uses a benchmark index, which measures the degrees of human perception of gas in the environment. In this way, the proposed system tries to mimic the human perception of a natural gas leak and helps one to indicate if more or less amount of odorant should be added into the gas pipeline. Experiments were conducted comparing the performance of the system with human performance, which is normally used to deal with this problem. The proposed system demonstrated promising results and improvements are presented.

Clustering and Selection Using Grouping Genetic Algorithms for Blockmodeling to Construct Neural Network Ensembles

The choice of a Committee of Classifiers is based on the idea that two or more classifiers can make a better decision than a single one. In the literature there are several methodologies for construction of committees and among them Classifier Selection that determines the best or a subset with the most efficient classifiers in each region of the feature space. Blockmodeling is a useful tool for describing the fundamental structure of social networks, but it is used in this work in a non-social data. As shown in the literature, clustering data and then choosing a classifier for each cluster can increase the committee performance and Evolutionary Algorithms can increase even more the performance. Thus this paper proposes BMGGAVS using a combination of Blockmodeling and Genetic Algorithms in order to cluster data and through a simple vote system assign a Neural Network for each cluster. Results from experiments in 9 databases indicate that BMGGAVS is able to obtain a good performance.

A Two Stage Clustering Method Combining Self-Organizing Maps and Ant K-Means

This paper proposes a clustering method SOMAK, which is composed by Self-Organizing Maps (SOM) followed by the Ant K-means (AK) algorithm. SOM is an Artificial Neural Network (ANN), which has one of its characteristics, the nonlinear projection from a high dimensionality of the sensorial space. AK is based in the Ant Colony Optimization (ACO), which is a recently proposed meta-heuristic approach for solving hard combinatorial optimization problems. The AK algorithm modifies the K-means on locating the objects and these are then clustered according to the probabilities which in turn are updated by the pheromone. The SOMAK has a good performance when compared with some clustering techniques and reduces the computational time.

Tuning Artificial Neural Networks Parameters Using an Evolutionary Algorithm

This paper describes a method to automatically tuning artificial neural networks parameters for a specific problem using an evolutionary algorithm. The method employs an evolutionary search to perform simultaneous tuning of initial weights, transfer functions, architectures and learning rules (learning algorithms parameters). Experiments were performed and the results demonstrate that the method in a shorter time of search, is able to find efficient networks with satisfactory generalization capabilities.

Odor recognition systems for natural gas odorization monitoring

This paper presents a system consisting of physical sensors and intelligent software for the automatic identification of the concentration of natural gas odorants and details the development of the sensor and pattern recognition systems. The sensor system uses spectroscopic technology and the pattern recognition system uses wavelet and artificial neural network technology. The aim is to determine the concentration of a natural gas odorant in the environment and associate this concentration with the benchmark index, which measures the degree of human perception to the presence of gas in the environment. Experiments were conducted comparing the performance of the system with human performance, which is normally used to deal with this problem. The proposed system demonstrated promising results.