Eduardo Masato Iyoda

Feedforward neural network initialization: an evolutionary approach

The initial set of weights to be used in supervised learning for multilayer neural networks has a strong influence in the learning speed and in the quality of the solution obtained after convergence. An inadequate initial choice of the weight values may cause the training process to get stuck in a poor local minimum or to face abnormal numerical problems. There are several proposed techniques that try to avoid both local minima and numerical instability, only by means of a proper definition of the initial set of weights. This paper focuses on the application of genetic algorithms (GA) as a tool to analyze the space of weights, in order to achieve good initial conditions for supervised learning. GAs almost-global sampling compliments connectionist local search techniques well, and allows one to find some very important characteristics in the initial set of weights for multilayer networks. The results presented are compared, for a set of benchmarks, with that produced by other approaches found in the literature.

Evolutionary hybrid composition of activation functions in feedforward neural networks

Considering computational algorithms available in the literature, associated with supervised learning in feedforward neural networks, a wide range of distinct approaches can be identified While the adjustment of the connection weights represents an omnipresent stage, the algorithms differ in three basic aspects: the technique chosen to determine the dimension of the multilayer neural network, the procedure adopted to determine the activation function of each neuron, and the kind of composition of the hidden activations used to produce the output. The advanced learning algorithms are designed to treat all these three aspects during learning, guiding to dedicated solutions. In this paper, an evolutionary hybrid learning algorithm is presented to deal simultaneously with these three aspects. The essence of this approach is the existence of a search procedure based on a synergy between genetic algorithms and conjugate gradient optimization.

Evolutionary design of neurofuzzy networks for pattern classification

We consider a neural network based fuzzy system model whose basic processing unit consists of two types of generic logic (OR and AND) neurons. The net is structured into a multilayer topology and trained by a competitive learning algorithm, together with a genetic algorithm approach to select the most suitable triangular norms and co-norms that model the logic neurons. The main features of the system include: automatic rule generation and selection, learning capability, processing time independent of the input space partition, and automatic selection of the t-norms and s-norms that model the basic logic operators (OR, AND) encountered in the theory of fuzzy sets. Four benchmark problems are considered to compare the performance of the proposed method with those produced by alternative strategies.

Solving Truck Delivery Problems Using Integrated Evaluation Criteria Based on Neighborhood Degree and Evolutionary Algorithm

To solve a real-world truck delivery and dispatch problem (TDDP) that involves multiple mutually conflicting objectives, such as running and loading costs, a concept of neighborhood degree (ND) and an integrated evaluation criteria (IEC) of the solution based on ND are proposed. The IEC makes the weight setting easier than by using conventional methods. To find a high-quality solution to a TDDP in practical computational time, an evolutionary algorithm is proposed. It involves 3 components: (i) a simulated annealing (SA)-based method for finding an optimal or a suboptimal route for each vehicle; (ii) an evolutionary computation (EC)-based method for finding an optimal schedule for a group of vehicles; and (iii) threshold-based evolutionary operations, utilizing the ND concept. The TDDP viewed from real-world application is formulated and the proposed algorithm is implemented on a personal computer using C++. The proposed algorithm is evaluated in 2 experiments involving real-world data representative of the TDDP, and applied to food product delivery to a chain of 46 convenience stores in Saitama Prefecture. In the 2 experiments, our proposed algorithm resulted in a better schedule (with 80%-90% shorter computational time) than a schedule produced by an expert. By incorporating application-specific evaluation criteria, the proposed algorithm is applied to problems such as homedelivery of parcels or mail, and to problems of multidepot delivery and dispatch.

Redes Neurais Construtivas: Uma Abordagem Comparativa

Until recently, the determination of a proper dimension for an artificial neural network in a given application task usually involved only the designer’s experience in implementing trial and error procedures. Nowadays, automatic design of neural network architectures is becoming part of the training process, by means of a more efficient exploration of the available information for supervised learning. Among the already proposed alternatives to automatic design, this paper emphasizes methods founded on the constructive paradigm. We chose three constructive algorithms for comparison: A* heuristic search (A*), CascadeCorrelation with different activation functions (CASCOR) and Projection Pursuit Learning (PPL). A brief review of these constructive strategies in the solution of regression problems is presented, and their performance in terms of the parsimony of the resultant architecture is verified in benchmark problems.