Antonio J. Tallón-Ballesteros

A two-stage algorithm in evolutionary product unit neural networks for classification

This paper presents a procedure to add broader diversity at the beginning of the evolutionary process. It consists of creating two initial populations with different parameter settings, evolving them for a small number of generations, selecting the best individuals from each population in the same proportion and combining them to constitute a new initial population. At this point the main loop of an evolutionary algorithm is applied to the new population. The results show that our proposal considerably improves both the efficiency of previous methodologies and also, significantly, their efficacy in most of the data sets. We have carried out our experimentation on twelve data sets from the UCI repository and two complex real-world problems which differ in their number of instances, features and classes.

Improving the accuracy of a two-stage algorithm in evolutionary product unit neural networks for classification by means of feature selection

This paper introduces a methodology that improves the accuracy of a two-stage algorithm in evolutionary product unit neural networks for classification tasks by means of feature selection. A couple of filters have been taken into consideration to try out the proposal. The experimentation has been carried out on seven data sets from the UCI repository that report test mean accuracy error rates about twenty percent or above with reference classifiers such as C4.5 or 1-NN. The study includes an overall empirical comparison between the models obtained with and without feature selection. Also several classifiers have been tested in order to illustrate the performance of the different filters considered. The results have been contrasted with nonparametric statistical tests and show that our proposal significantly improves the test accuracy of the previous models for the considered data sets. Moreover, the current proposal is much more efficient than a previous methodology developed by us; lastly, the reduction percentage in the number of inputs is above a fifty five, on average.

Merging subsets of attributes to improve a hybrid consistency-based filter: a case of study in product unit neural networks

ABSTRACT This paper presents a quality enhancement of the selected features by a hybrid filter-based jointly on feature ranking and feature subset selection (FR-FSS) using a consistency-based measure via merging new features which are obtained applying other FR-FSS evaluated with a correlation metric. The goal is to overcome the accuracy of a neural network classifier containing product units as hidden nodes combined with a feature selection pre-processing step by means of a single consistency-based FR-FSS filter. Neural models are trained with a refined evolutionary programming approach called two-stage evolutionary algorithm. The experimentation has been carried out in eight complex classification problems, seven out of them from UCI (University of California at Irvine) repository and one real-world problem, with high test error rates (around 20%) with powerful classifiers such as 1-nearest neighbour or C4.5. Non-parametric statistical tests revealed that the new proposal significantly improves the accuracy of the neural models.

Tackling Ant Colony Optimization Meta-Heuristic as Search Method in Feature Subset Selection Based on Correlation or Consistency Measures

This paper introduces the use of an ant colony optimization (ACO) algorithm, called Ant System, as a search method in two well-known feature subset selection methods based on correlation or consistency measures such as CFS (Correlation-based Feature Selection) and CNS (Consistency-based Feature Selection). ACO guides the search using a heuristic evaluator. Empirical results on twelve real-world classification problems are reported. Statistical tests have revealed that InfoGain is a very suitable heuristic for CFS or CNS feature subset selection methods with ACO acting as search method. The use of InfoGain is shown to be the significantly better heuristic over a range of classifiers. The results achieved by means of ACO-based feature subset selection with the suitable heuristic evaluator are better for most of the problems comparing with those obtained with CFS or CNS combined with Best First search.

Deleting or keeping outliers for classifier training

This paper introduces two statistical outlier detection approaches by classes. Experiments on binary and multi-class classification problems reveal that the partial removal of outliers improves significantly one or two performance measures for C4.5 and 1-nearest neighbour classifiers. Also, a taxonomy of problems according to the amount of outliers is proposed.

Low Dimensionality or Same Subsets as a Result of Feature Selection: An In-Depth Roadmap.

This paper addresses the situation that may happen after the application of feature subset selection in terms of a reduced number of selected features or even same solutions obtained by different algorithms. The data mining community has been working for a long time with the assumption that meaningful attributes are either highly correlated with the class or represent a consistent subset, that is, with no inconsistencies. We have analysed around a hundred data sets very varied with a number of attributes below one hundred, a number of instances not greater than fifty thousand and a number of classes below fifty. Basically, in the first round we applied two different feature subset selection methods to pick up the figures in terms of reduced dimensionality. After that, we divided them into different groups according to the number of selected attributes. Next, we deepened the analysis in every category and we added a new feature selection procedure. Finally, we assessed the performance of the original problem and the reduced subsets with four classifiers providing some prospective directions.

Simplifying pattern recognition problems via a scatter search algorithm

ABSTRACT This article presents an approach to simplify pattern recognition problems via a scatter search algorithm that is applied as feature subset selector (FSS). Experimentation on five high-dimensionality problems, with a feature space in the range 2308–16063 and feature-to-pattern ratios greater than 27, revealed that the most appropriate feature selector is based on correlation. Moreover, the most accurate way is to combine the new proposal with a correlation-based attribute evaluator with a Naive Bayes Tree classifier; their performance has been compared with a reference FSS and sheds light on very interesting results in terms of accuracy and problem reduction.

Medium and high-dimensionality attribute selection in Bayes-type classifiers

This paper introduces the application of attribute selection methods along with Bayes classifiers. The proposal has been evaluated in eleven binary and multi-class real data sets with a number of instances lower than a thousand and a number of attributes between eight and sixteen thousand. Among them, five data sets belong to the Bioinformatics area. Experiments show that, in general terms, the most convenient attribute selector is based on a correlation measure. According to the reported results, the best classifier in high-dimensional data sets is Naive Bayes.

Accuracy Increase on Evolving Product Unit Neural Networks via Feature Subset Selection

A framework that combines feature selection with evolutionary artificial neural networks is presented. This paper copes with neural networks that are applied in classification tasks. In machine learning area, feature selection is one of the most common techniques for pre-processing the data. A set of filters have been taken into consideration to assess the proposal. The experimentation has been conducted on nine data sets from the UCI repository that report test error rates about fifteen percent or above with reference classifiers such as C4.5 or 1-NN. The new proposal significantly improves the baseline framework, both approaches based on evolutionary product unit neural networks. Also several classifiers have been tried in order to illustrate the performance of the different methods considered.

Featuring the Attributes in Supervised Machine Learning

This paper introduces an approach to feature subset selection which is able to characterise the attributes of a supervised machine learning problem into two categories: essential and important features. Additionally, the fusion of both kinds of features yields to an overcoming in the prediction task, where some measures such as accuracy and Receiver Operating Characteristic curve (ROC) have been reported. The test-bed is composed of eight binary and multi-class classification problems with up to five hundred of attributes. Several classification algorithms such as Ridor, PART, C4.5 and NBTree have been tested to assess the proposal.