Hermenegildo Alejandro Ceccatto

Neural network ensembles: evaluation of aggregation algorithms

Ensembles of artificial neural networks show improved generalization capabilities that outperform those of single networks. However, for aggregation to be effective, the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. We present here an extensive evaluation of several algorithms for ensemble construction, including new proposals and comparing them with standard methods in the literature. We also discuss a potential problem with sequential aggregation algorithms: the non-frequent but damaging selection through their heuristics of particularly bad ensemble members. We introduce modified algorithms that cope with this problem by allowing individual weighting of aggregate members. Our algorithms and their weighted modifications are favorably tested against other methods in the literature, producing a sensible improvement in performance on most of the standard statistical databases used as benchmarks.

Weed seeds identification by machine vision

Abstract The implementation of new methods for reliable and fast identification and classification of seeds is of major technical and economical importance in the agricultural industry. As in ocular inspection, the automatic classification of seeds should be based on knowledge of seed size, shape, color and texture. In this work, we assess the discriminating power of these characteristics for the unique identification of seeds of 57 weed species. Using the performance of a naive Bayes classifier as selection criterion, we identified a nearly optimal set of 12 (six morphological+four color+two textural) seed characteristics to be used as classification parameters. We found that, as expected, size and shape characteristics have larger discriminating power than color and textural ones. However, all these features are required to reach an identification performance acceptable for practical applications. In spite of its simplicity, the naive Bayes classifier reveals itself surprisingly good for the identification of seed species. This might be due to the careful selection of the feature set, leading to nearly independent parameters as assumed by this method. We also found that, using the same feature set, a more sophisticated classifier based on an artificial neural network committee performs only slightly better than this simple Bayesian approach.

Aggregation algorithms for neural network ensemble construction

How to generate and aggregate base learners to have optimal ensemble generalization capabilities is an important questions in building composite regression/classification machines. We present here an evaluation of several algorithms for artificial neural networks aggregation in the regression settings, including new proposals and comparing them with standard methods in the literature. We also discuss a potential problem with sequential algorithms: the non frequent but damaging selection through their heuristics of particularly bad ensemble members. We show that one can cope with this problem by allowing individual weighting of aggregate members. Our algorithms and their weighted modifications are favorably tested against other methods in the literature, producing a performance improvement on the standard statistical databases used as benchmarks.

Selecting diverse members of neural network ensembles

Ensembles of artificial neural networks have been used as classification/regression machines, showing improved generalization capabilities that outperform those of single networks. However, it has been recognized that for aggregation to be effective the individual network must be as accurate and diverse as possible. An important problem is, then, how to choose the aggregate members in order to have an optimal compromise between these two conflicting conditions. We propose here a new method for selecting members of regression/classification ensembles that leads to small aggregates with few but very diverse individual predictors. Using artificial neural networks as individual learners, the algorithm is favorably tested against other methods in the literature, producing a remarkable performance improvement on the standard statistical databases used as benchmarks. In addition, and as a concrete application, we study the sunspot time series and predict the remaining of the current cycle 23 of solar activity.

Symmetric-embedding prediction of the CATS benchmark

We present a general strategy for filling the missing data of the CATS benchmark time series prediction competition. Our approach builds upon a time-symmetric embedding of this time series and the use of a one-shot forecasting for each missing value inside the gaps from distant-enough delayed and forwarded predictors. In the extrapolation region we perform standard, non-iterated forward predictions. For modeling purposes we consider bagging of multi-layer perceptrons (MLPs). We discuss two different implementations of this strategy: The first one is based on a simultaneous modeling of both large- and short-scale dynamics information, using (suitably delayed and forwarded) original CATS values and their first differences as inputs to MLPs. The second one follows a two-stage strategy, in which behaviors at different scales are modeled separately. First, the overall behavior at large scales is fitted with a smooth curve obtained by repeated application of a Savitzky-Golay filter. Then, the remaining short-scale variability is approximated using bagged MLPs. Expected error levels for these two implementations are provided according to performance on test data.