Raul Fonseca Neto

Perceptron models for online structured prediction

Our structured prediction problem is formulated as a convex optimization problem of maximal margin [5-6], quite similar to the formulation of multiclass support vector machines (MSVM) [8]. It is applied to predict costs among states of paths. Predicting them properly is very important, because the problem of paths planning depends on its correctness. Ratliff [4] showed a maximum margin approach which allows the prediction of costs in different environments using subgradient method. As a contribution of this work, we developed new solution methods: the first one, called Structured Perceptron, has similarities with the correction scheme proposed by [1] and the second one is called Structured IMA. It is derived from the work presented by [2]. Both use the Perceptron model. The proposed algorithms were more efficient in terms of computational effort and similar in prediction quality when compared with [4].

A dual method for solving the nonlinear structured prediction problem

A primal structured perceptron method to solve the linear structured prediction problem.A kernel method to solve the nonlinear structured prediction problem based on sampling.An alternative approach to solve the inverse reinforcement problem.An efficient solution for the path planning prediction costs problem. In this paper, we present a perceptron-based algorithm and have developed a dual formulation to solve the nonlinear structured prediction problem, which we called Dual Structured Incremental Margin Algorithm (DSIMA). The proposed formulation allows the introduction of kernel functions enabling the efficient solution of nonlinear problems. In order to verify the correctness and applicability of the algorithm, we consider an inverse approach to the path planning problem. The problem mapped on a grid environment can be solved by a search process that essentially depends on the definition of the transition costs between states. In this context, we develop and apply a learning algorithm that is able to perform the reverse path, i.e., the prediction of these costs in a direct space for the linear form. However, considering the nonlinear form, the problem is solved in a space of high dimension and where it is possible to learn a path instead of the transition costs. This learning problem is usually formulated as a convex optimization problem of maximum margin. Several tests to solve the costs prediction problem were carried out and the results compared to other structured prediction techniques. The proposed algorithm demonstrated greater efficiency in terms of computational effort and quality of prediction.

Incremental p-margin algorithm for classification with arbitrary norm

This paper presents a new algorithm to approximate large margin solutions in binary classification problems with arbitrary q-norm or p-margin, where p and q are Holder conjugates. We begin by presenting the online fixed p-margin perceptron algorithm (FMPp) that solves linearly separable classification problems in primal variables and consists of a generalization of the fixed margin perceptron algorithm (FMP). This algorithm is combined with an incremental margin strategy called IMAp, which computes an approximation of the maximal p-margin. To achieve this goal, IMAp executes FMPp several times with increasing p-margin values. One of the main advantages of this approach is its flexibility, which allows the use of different p-norms in the same primal formulation. For non-linearly separable problems, FMPp can be used with a soft margin in primal variables. The incremental learning strategy always guarantees a good approximation of the optimal p-margin and avoids the use of linear or higher order programming methods. IMAp was tested in different datasets obtaining similar results when compared to classical L1 and L ∞ linear programming formulations. Also, the algorithm was compared to ALMAp and presents superior results. HighlightsWe propose a novel algorithm for large p-margin classification problems, for 1 � p � ∞ .The approach is based on an unified perceptron-based formulation.Soft-margin in primal variables is introduced for non-linearly separable problems.An efficient incremental strategy is used to construct the large p-margin solution.

A Novel Ensemble Approach Based on Balanced Perceptrons Applied to Microarray Datasets

Recently, ensemble learning theory has received much attention in the machine learning community, since it has been demonstrated as a great alternative to generate more accurate predictors with higher generalization abilities. The improvement of generalization performance of an ensemble is directly related to the diversity and accuracy of the individual classifiers. Thus, contributions in this scenario are still relevant. In this paper, we propose a novel ensemble approach based on balanced Perceptrons. In order to improve the accuracy of each individual classifier, we balance the final hyper plane solution. Also, we introduce the dissimilarity measure which is employed in order to maximize the diversity of the ensemble. This strategy accepts a new component in the ensemble only if it holds a minimum predetermined distance from the other components. We conduct our experimental study on micro array datasets and assess the performance of the proposed method combined by averaging and unweighted voting. Reported results show that our method outperforms other ensemble approaches, such as Random Averaging and AdaBoost, in all considered datasets. Also, we overcome Support Vector Machines in almost all cases. We perform statistical tests to check for the significance of our results.

Um Classificador Kernel Composto por um Comitê de Perceptrons Balanceados

Resumo—Recentemente, abordagens baseadas em comitês de classificadores e métodos ensemble têm sido bastante exploradas por serem uma alternativa simples e eficaz para a construção de classificadores mais acurados. A melhoria da capacidade de generalização de um ensemble está diretamente relacionada a acurácia de cada classificador individual, bem como a diversidade dos classificadores que o compõem. Sendo assim, contribuições nesse escopo ainda são relevantes. Nesse trabalho, é apresentada uma extensão de um modelo ensemble baseado em Perceptrons balanceados que permite a inclusão de funções kernel e a solução de problemas não-linearmente separáveis. Visando a melhoria da acurácia do classificador individual, o hiperplano solução é balanceado. Além disso, uma medida de dissimilaridade é introduzida com intuito de maximizar a diversidade do ensemble. Essa estratégia permite a aceitação de um novo componente no comitê se, e somente se, uma distância mínima pré-estabelecida é mantida entre o novo candidato e todos os outros componentes. Um estudo experimental foi conduzido em bases de dados nãolinearmente separáveis. Os resultados obtidos mostraram que o método proposto foi capaz de superar outros algoritmos avaliados, como o AdaBoost e o SVM, na maior parte dos casos testados. Além disso, o método proposto superou consistentemente o classificador de base empregado.

Um Classificador para Seleção de Características Aplicado a Problemas Não-Linearmente Separáveis

Resumo—Este artigo apresenta uma abordagem robusta para o problema de seleção de caracterı́sticas aplicado a conjuntos de dados não-linearmente separáveis. Neste sentido, foram realizados estudos comparando-se resultados relativos a utilização de classificadores baseados em funções kernel, os quais produzem o processo de seleção em um espaço de mais alta dimensão, denominado espaço kernel, com classificadores robustos, que utilizam o conceito de margem flexı́vel e permitem uma tolerância a erros de classificação promovendo a seleção de caracterı́sticas diretamente no espaço de entrada. A introdução do processo de flexibilização da margem possibilita a correta classificação de dados, que não sejam linearmente separáveis no espaço de entrada, refletindo em uma melhora do poder de generalização. Tal fato pode ser comprovado pela redução de erros nos testes experimentais. Também, a opção pela minimização da norma L1 do vetor normal ao hiperplano separador, tornou possı́vel a construção de hipóteses com alto grau de esparsidade. De fato, esta forma de otimização, que apresenta um processo de regularização interna, contribui de forma significativa para uma melhor eficiência do processo de seleção de caracterı́sticas. Para a seleção dos melhores subconjuntos, os classificadores foram associados a um algoritmo de busca ordenada que utiliza os valores de margem como medida de avaliação dos subconjuntos candidatos. Foram realizados experimentos para a comprovação da proposta apresentada, tendo-se obtido resultados bastante significativos.