Antônio de Pádua Braga

Improving generalization of MLPs with multi-objective optimization

Abstract This paper presents a new learning scheme for improving generalization of multilayer perceptrons. The algorithm uses a multi-objective optimization approach to balance between the error of the training data and the norm of network weight vectors to avoid overfitting. The results are compared with support vector machines and standard backpropagation.

Intelligent Thermographic Diagnostic Applied to Surge Arresters: A New Approach

This paper describes a methodology that aims to extract information to enable the detection and diagnosis of faults in surge arresters, using the thermovision technique. Thermovision is a non-destructive technique used in diverse services of maintenance, having the advantage not to demand the disconnection of the equipment. The methodology uses a digital image processing algorithm based on the Watershed Transform to get the segmentation of the surge arresters. By applying the methodology is possible to classify surge arresters operative condition in: faulty, normal, light, and suspicious. The computational system generated train its neuro-fuzzy network by using a historical thermovision data. During the train phase, a heuristic is proposed in order to set the number of networks in the diagnosis system. This system was validated by a database with a hundreds of different faulty sceneries. The validation error of the set of neuro-fuzzy and the automatic digital thermovision image processing was about 10%t. The diagnosis system described has been successfully used by Electric Energy Research Center as a decision making tool for surge arresters fault diagnosis.

Novel Cost-Sensitive Approach to Improve the Multilayer Perceptron Performance on Imbalanced Data

Traditional learning algorithms applied to complex and highly imbalanced training sets may not give satisfactory results when distinguishing between examples of the classes. The tendency is to yield classification models that are biased towards the overrepresented (majority) class. This paper investigates this class imbalance problem in the context of multilayer perceptron (MLP) neural networks. The consequences of the equal cost (loss) assumption on imbalanced data are formally discussed from a statistical learning theory point of view. A new cost-sensitive algorithm (CSMLP) is presented to improve the discrimination ability of (two-class) MLPs. The CSMLP formulation is based on a joint objective function that uses a single cost parameter to distinguish the importance of class errors. The learning rule extends the Levenberg-Marquadts rule, ensuring the computational efficiency of the algorithm. In addition, it is theoretically demonstrated that the incorporation of prior information via the cost parameter may lead to balanced decision boundaries in the feature space. Based on the statistical analysis of results on real data, our approach shows a significant improvement of the area under the receiver operating characteristic curve and G-mean measures of regular MLPs.

Improving generalization of MLPs with sliding mode control and the Levenberg-Marquardt algorithm

A variation of the well-known Levenberg-Marquardt for training neural networks is proposed in this work. The algorithm presented restricts the norm of the weights vector to a preestablished norm value and finds the minimum error solution for that norm value. The norm constrain controls the neural networks degree of freedom. The more the norm increases, the more flexible is the neural model. Therefore, more fitted to the training set. A range of different norm solutions is generated and the best generalization solution is selected according to the validation set error. The results show the efficiency of the algorithm in terms of generalization performance.

Reinforcement learning of a simple control task using the spike response model

Abstract In this work, we propose a variation of a direct reinforcement learning algorithm, suitable for usage with spiking neurons based on the spike response model (SRM). The SRM is a biologically inspired, flexible model of spiking neuron based on kernel functions that describe the effect of spike reception and emission on the membrane potential of the neuron. In our experiments, the spikes emitted by a SRM neuron are used as input signals in a simple control task. The reinforcement signal obtained from the environment is used by the direct reinforcement learning algorithm, that modifies the synaptic weights of the neuron, adjusting the spiking firing times in order to obtain a better performance at the given problem. The obtained results are comparable to those from classic methods based on value function approximation and temporal difference, for simple control tasks.

Training neural networks with a multi-objective sliding mode control algorithm

Abstract This paper presents a new sliding mode control algorithm that is able to guide the trajectory of a multi-layer perceptron within the plane formed by the two objective functions: training set error and norm of the weight vectors. The results show that the neural networks obtained are able to generate an approximation to the Pareto set, from which an improved generalization performance model is selected.

An efficient multi-objective learning algorithm for RBF neural network

Most of modern multi-objective machine learning methods are based on evolutionary optimization algorithms. They are known to be global convergent, however, usually deliver nondeterministic results. In this work we propose the deterministic global solution to a multi-objective problem of supervised learning with the methodology of nonlinear programming. As the result, the proposed multi-objective algorithm performs a global search of Pareto-optimal hypotheses in the space of RBF networks, determining their weights and basis functions. In combination with the Akaike and Bayesian information criteria, the algorithm demonstrates a high generalization efficiency on several synthetic and real-world benchmark problems.

A multi-objective approach to RBF network learning

The problem of inductive supervised learning is discussed in this paper within the context of multi-objective (MOBJ) optimization. The smoothness-based apparent (effective) complexity measure for RBF networks is considered. For the specific case of RBF network, bounds on the complexity measure are formally described. As the synthetic and real-world data experiments show, the proposed MOBJ learning method is capable of efficient generalization control along with network size reduction.

Evolutionary Radial Basis Functions for Credit Assessment

Credit analysts generally assess the risk of credit applications based on their previous experience. They frequently employ quantitative methods to this end. Among the methods used, Artificial Neural Networks have been particularly successful and have been incorporated into several computational tools. However, the design of efficient Artificial Neural Networks is largely affected by the definition of adequate values for their free parameters. This article discusses a new approach to the design of a particular Artificial Neural Networks model, RBF networks, through Genetic Algorithms. It presents an overall view of the problems involved and the different approaches employed to optimize Artificial Neural Networks genetically. For such, several methods proposed in the literature for optimizing RBF networks using Genetic Algorithms are discussed. Finally, the model proposed by the authors is described and experimental results using this model for a credit risk assessment problem are presented.

Black and Gray-Box Identification of a Hydraulic Pumping System

The use of auxiliary information during the identification of nonlinear systems can be handled in different ways and at different levels. In this brief, static information of a 15 kW hydraulic pumping system is used as a priori knowledge in the parameters estimation of polynomial models which are compared to polynomial and neural models obtained by black-box techniques. The aim is to find models with good performance in both transient and steady-state regimes. This brief presents a novel bi-objective problem that uses free-run simulation and a new decision-maker. The optimization problem is solved using a genetic algorithm. Compared with other techniques, the proposed approach can lead to models with better dynamic and static performance.