Alicia Morales-Reyes

Term-weighting learning via genetic programming for text classification

A new method for learning term-weighting schemes is proposed.A genetic program searches for the scheme that maximizes classification performance.The method is evaluated in text and image categorization, and authorship attribution. This paper describes a novel approach to learning term-weighting schemes (TWSs) in the context of text classification. In text mining a TWS determines the way in which documents will be represented in a vector space model, before applying a classifier. Whereas acceptable performance has been obtained with standard TWSs (e.g., Boolean and term-frequency schemes), the definition of TWSs has been traditionally an art. Further, it is still a difficult task to determine what is the best TWS for a particular problem and it is not clear yet, whether better schemes, than those currently available, can be generated by combining known TWS. We propose in this article a genetic program that aims at learning effective TWSs that can improve the performance of current schemes in text classification. The genetic program learns how to combine a set of basic units to give rise to discriminative TWSs. We report an extensive experimental study comprising data sets from thematic and non-thematic text classification as well as from image classification. Our study shows the validity of the proposed method; in fact, we show that TWSs learned with the genetic program outperform traditional schemes and other TWSs proposed in recent works. Further, we show that TWSs learned from a specific domain can be effectively used for other tasks.

Simultaneous generation of prototypes and features through genetic programming

Nearest-neighbor (NN) methods are highly effective and widely used pattern classification techniques. There are, however, some issues that hinder their application for large scale and noisy data sets; including, its high storage requirements, its sensitivity to noisy instances, and the fact that test cases must be compared to all of the training instances. Prototype (PG) and feature generation (FG) techniques aim at alleviating these issues to some extent; where, traditionally, both techniques have been implemented separately. This paper introduces a genetic programming approach to tackle the simultaneous generation of prototypes and features to be used for classification with a NN classifier. The proposed method learns to combine instances and attributes to produce a set of prototypes and a new feature space for each class of the classification problem via genetic programming. An heterogeneous representation is proposed together with ad-hoc genetic operators. The proposed approach overcomes some limitations of NN without degradation in its classification performance. Experimental results are reported and compared with several other techniques. The empirical assessment provides evidence of the effectiveness of the proposed approach in terms of classification accuracy and instance/feature reduction.

Genetic Programming of Prototypes for Pattern Classification

This paper introduces a genetic programming approach to the generation of classification prototypes. Prototype-based classification is a pattern recognition methodology in which the training set of a classification problem is represented by a small subset of instances. The assignment of labels to test instances is usually done by a 1NN rule. We propose a new prototype generation method, based on genetic programming, in which examples of each class are automatically combined to generate highly effective classification prototypes. The genetic program aims to maximize an estimate of the generalization performance of a 1NN classifier using the prototypes. We report experimental results on a benchmark for the evaluation of prototype generation methods. Experimental results show the validity of our approach: the proposed method outperforms most of the state of the art techniques when using both small and large data sets. Better results are obtained for data sets with numeric attributes only, although the performance of our method on mixed data is very competitive as well.

MOPG: a multi-objective evolutionary algorithm for prototype generation

Prototype generation deals with the problem of generating a small set of instances, from a large data set, to be used by KNN for classification. The two key aspects to consider when developing a prototype generation method are: (1) the generalization performance of a KNN classifier when using the prototypes; and (2) the amount of data set reduction, as given by the number of prototypes. Both factors are in conflict because, in general, maximizing data set reduction implies decreasing accuracy and viceversa. Therefore, this problem can be naturally approached with multi-objective optimization techniques. This paper introduces a novel multi-objective evolutionary algorithm for prototype generation where the objectives are precisely the amount of reduction and an estimate of generalization performance achieved by the selected prototypes. Through a comprehensive experimental study we show that the proposed approach outperforms most of the prototype generation methods that have been proposed so far. Specifically, the proposed approach obtains prototypes that offer a better tradeoff between accuracy and reduction than alternative methodologies.

PGGP: Prototype Generation via Genetic Programming

Abstract Prototype generation (PG) methods aim to find a subset of instances taken from a large training data set, in such a way that classification performance (commonly, using a 1NN classifier) when using prototypes is equal or better than that obtained when using the original training set. Several PG methods have been proposed so far, most of them consider a small subset of training instances as initial prototypes and modify them trying to maximize the classification performance on the whole training set. Although some of these methods have obtained acceptable results, training instances may be under-exploited, because most of the times they are only used to guide the search process. This paper introduces a PG method based on genetic programming in which many training samples are combined through arithmetic operators to build highly effective prototypes. The genetic program aims to generate prototypes that maximize an estimate of the generalization performance of an 1NN classifier. Experimental results are reported on benchmark data to assess PG methods. Several aspects of the genetic program are evaluated and compared to many alternative PG methods. The empirical assessment shows the effectiveness of the proposed approach outperforming most of the state of the art PG techniques when using both small and large data sets. Better results were obtained for data sets with numeric attributes only, although the performance of the proposed technique on mixed data was very competitive as well.

Evolving weighting schemes for the Bag of Visual Words

AbstractThe Bag of Visual Words (BoVW) is an established representation in computer vision. Taking inspiration from text mining, this representation has proved to be very effective in many domains. However, in most cases, standard term-weighting schemes are adopted (e.g., term-frequency or TF-IDF). It remains open the question of whether alternative weighting schemes could boost the performance of methods based on BoVW. More importantly, it is unknown whether it is possible to automatically learn and determine effective weighting schemes from scratch. This paper brings some light into both of these unknowns. On the one hand, we report an evaluation of the most common weighting schemes used in text mining, but rarely used in computer vision tasks. Besides, we propose an evolutionary algorithm capable of automatically learning weighting schemes for computer vision problems. We report empirical results of an extensive study in several computer vision problems. Results show the usefulness of the proposed method.

A scalable and customizable processor array for implementing cellular genetic algorithms

Architectures design for Genetic Algorithms (GAs) has proved its effectiveness to tackle hard real time constrained problems that require an optimization mechanism in one of their phases. Most of these approaches are problem dependent and cannot be easily adapted to other problems. Moreover, GAs based architectures preserve the algorithmic structure of a panmictic population in a sequential GA and therefore they are similar to a software implementation. Recently, combination of GAs, both sequential and parallel and reconfigurable devices such as FPGAs have been merged to create GAs based parallel hardware architectures. This study proposes a novel hardware architectural framework that implements a fine grained or cellular GAs while maintaining toroidal connection among individuals within the population. Achieving massive parallelism is limited by available resources; therefore, the proposed architectural design implements a segmentation strategy that partitions the entire decentralized population while maintaining original algorithmic interaction among solutions. The proposed architecture aims at preserving fine grained GAs algorithmic structure while improving resources usage. It also allows flexibility in terms of population and solutions representation size and the evaluation module containing the objective function is interchangeable.

Accelerating the construction of BRIEF descriptors using an FPGA-based architecture

BRIEF emerged as a novel alternative to conventional floating-point-based descriptors such as SIFT or SURF. In contrast to these descriptors, BRIEF is a descriptor represented by a binary number offering two main advantages: low memory footprint and fast descriptor comparison. These qualities make it a suitable descriptor to be implemented on a hardware architecture, where the comparison operation can be implemented efficiently via a parallel scheme. However, the construction of BRIEF involves a sequential operation in the form of a set of pairwise tests on the image intensities, and as consequence, sequential memory access is necessary. In this paper, we propose a novel way to construct the BRIEF descriptor by arranging the pairwise tests such that data retrieval from memory is exploited, thus accelerating the descriptor construction up to 4 times when compared to the sequential way.

Genetic Programming of Heterogeneous Ensembles for Classification

The ensemble classification paradigm is an effective way to improve the performance and stability of individual predictors. Many ways to build ensembles have been proposed so far, most notably bagging and boosting based techniques. Evolutionary algorithms EAs also have been widely used to generate ensembles. In the context of heterogeneous ensembles EAs have been successfully used to adjust weights of base classifiers or to select ensemble members. Usually, a weighted sum is used for combining classifiers outputs in both classical and evolutionary approaches. This study proposes a novel genetic program that learns a fusion function for combining heterogeneous-classifiers outputs. It evolves a population of fusion functions in order to maximize the classification accuracy. Highly non-linear functions are obtained with the proposed method, subsuming the existing weighted-sum formulations. Experimental results show the effectiveness of the proposed approach, which can be used not only with heterogeneous classifiers but also with homogeneous-classifiers and under bagging/boosting based formulations.

Structurally Layered Representation Learning: Towards Deep Learning Through Genetic Programming

We introduce a novel method for representation learning based on genetic programming (GP). Inspired into the way that deep neural networks learn descriptive/discriminative representations from raw data, we propose a structurally layered representation that allows GP to learn a feature space from large scale and high dimensional data sets. Previous efforts from the GP community for feature learning have focused on small data sets with a few input variables, also, most approaches rely on domain expert knowledge to produce useful representations. In this paper, we introduce the structurally layered GP formulation, together with an efficient scheme to explore the search space and show that this framework can be used to learn representations from large data sets of high dimensional raw data. As case of study we describe the implementation and experimental evaluation of an autoencoder developed under the proposed framework. Results evidence the benefits of the proposed framework and pave the way for the development of deep genetic programming.