Héctor Allende

M-estimators with asymmetric influence functions: the distribution case

Many applications consecrate the use of asymmetric distributions, and practical situations often require robust parametric inference. This paper presents the derivation of M-estimators with asymmetric influence functions, motivated by the distribution. This law, regarded as the universal model for speckled imagery, can be highly skewed and maximum likelihood estimation can be severely hampered by small percentages of outliers. These outliers appear mainly because the hypothesis of independence and equal distribution of observations are seldom satisfied in practice; for instance, in the process of filtering, some pixels within a window frequently come from regions with different underlying distributions. Traditional robust estimation methods, on the basis of symmetric robustifying functions, assume that the distribution is symmetric, but when the data distribution is asymmetric, these methods yield biased estimators. Empirical influence functions for maximum likelihood estimators are computed, and based on this information we propose the asymmetric M-estimator (AM-estimator), an M-estimator with asymmetric redescending functions. The performance of AM estimators is assessed, and it is shown that they either compete with or outperform both maximum likelihood and Huber-type M-estimators.

A non-parametric filter for digital image restoration, using cluster analysis

We introduce a method to restore digital images with contaminated pixels. One particular characteristic of this method is that it does not change the pixels that are not considered contaminated, thus avoiding excessive intervening of the original image. Each pixel is analyzed by studying its eight point neighborhood. A cluster analysis is performed on the group of eight pixels contained in the neighborhood. After deciding how many clusters there are in the neighborhood, a decision is made whether the center pixel is an outlier or not. If so, to assign a new value, another decision is made, on a probabilistic basis, as to which cluster it belongs.This method can be applied to black and white images as well as to color and multiphase images.

A novel Frank-Wolfe algorithm. Analysis and applications to large-scale SVM training

Recently, there has been a renewed interest in the machine learning community for variants of a sparse greedy approximation procedure for concave optimization known as {the Frank-Wolfe (FW) method}. In particular, this procedure has been successfully applied to train large-scale instances of non-linear Support Vector Machines (SVMs). Specializing FW to SVM training has allowed to obtain efficient algorithms but also important theoretical results, including convergence analysis of training algorithms and new characterizations of model sparsity. In this paper, we present and analyze a novel variant of the FW method based on a new way to perform away steps, a classic strategy used to accelerate the convergence of the basic FW procedure. Our formulation and analysis is focused on a general concave maximization problem on the simplex. However, the specialization of our algorithm to quadratic forms is strongly related to some classic methods in computational geometry, namely the Gilbert and MDM algorithms. On the theoretical side, we demonstrate that the method matches the guarantees in terms of convergence rate and number of iterations obtained by using classic away steps. In particular, the method enjoys a linear rate of convergence, a result that has been recently proved for MDM on quadratic forms. On the practical side, we provide experiments on several classification datasets, and evaluate the results using statistical tests. Experiments show that our method is faster than the FW method with classic away steps, and works well even in the cases in which classic away steps slow down the algorithm. Furthermore, these improvements are obtained without sacrificing the predictive accuracy of the obtained SVM model.

Fusion of self organizing maps

An important issue in data-mining is to find effective and optimal forms to learn and preserve the topological relations of highly dimensional input spaces and project the data to lower dimensions for visualization purposes. In this paper we propose a novel ensemble method to combine a finite number of Self Organizing Maps, we called this model Fusion-SOM. In the fusion process the nodes with similar Voronoi polygons are merged in one fused node and the neighborhood relation is given by links that measures the similarity between these fused nodes. The aim of combining the SOM is to improve the quality and robustness of the topological representation of the single model. Computational experiments show that the Fusion-SOM model effectively preserves the topology of the input space and improves the representation of the single SOM. We report the performance results using synthetic and real datasets, the latter obtained from a benchmark site.

Robust image modeling on image processing

Abstract This paper is concerned with robust models for representing images. The robust methods in image models are also applied to some important image processing situations such as segmentation by texture and image restoration in the presence of outliers. We consider a non-symmetric half plane (NSHP) autoregressive image model, where the image intensity at a point is a linear combination of the intensities of the eight nearest points located on one quadrant of the coordinate plane, plus an innovation process. Robust estimation algorithms for different outlier processes in causal autoregressive models are developed. These algorithms are based on robust generalized M (GM) estimators. Theoretical properties of the robust estimation algorithms are presented. The robust estimation algorithm for causal autoregressive models is applied to image restoration. The restoration method based on robust image model cleans out the outliers without involving any blurring of the image. Experimental results show that the quality of images restored by the model-based method is superior to the images restored by other conventional methods.

AD-SVMs: A light extension of SVMs for multicategory classification

The margin maximization principle implemented by binary Support Vector Machines (SVMs) has been shown to be equivalent to find the hyperplane equidistant to the closest points belonging to the convex hulls that enclose each class of examples. In this paper, we propose an extension of SVMs for multicategory classification which generalizes this geometric formulation. The obtained method preserves the form and complexity of the binary case, optimizing a single convex quadratic program where each new class introduces just one additional constraint. Reduced convex hulls and non-linear kernels, used in the binary case to deal with the non-linearly separable case, can be also implemented by our algorithm to obtain additional flexibility. Experimental results in well known datasets are presented, comparing our method with two widely used multicategory SVMs extensions.

Robust neural gas for the analysis of data with outliers

Learning the structure of real world data is difficult both to recognize and describe. The structure may contain high dimensional clusters that are related in complex ways. Furthermore, real data sets may contain several outliers. Vector quantization techniques has been successfully applied as a data mining tool. In particular the neural gas (NG) is a variant of the self organizing map (SOM) where the neighborhoods are adaptively defined during training through the ranking order of the distance of prototypes from the given training sample. Unfortunately, the learning algorithm of the NG is sensitive to the presence of outliers as we show in this paper. Due to the influence of the outliers in the learning process, the topology of the employed network does not conserve the topology of the manifold of the data which is presented. In this paper, we propose to robustify the learning algorithm where the parameter estimation process is resistant to the presence of outliers in the data. We call this algorithm robust neural gas (RNG). We illustrate our technique on synthetic and real data sets.

Self-Organizing Neuro-Fuzzy Inference System

The architectural design of neuro-fuzzy models is one of the major concern in many important applications. In this work we propose an extension to Rogerss ANFIS model by providing it with a selforganizing mechanism. The main purpose of this mechanism is to adapt the architecture during the training process by identifying the optimal number of premises and consequents needed to satisfy a users performance criterion. Using both synthetic and real data, our proposal yields remarkable results compared to the classical ANFIS.

Robust growing hierarchical self organizing map

The Growing Hierarchical Self Organizing Map (GHSOM) was introduced as a dynamical neural network model that adapts its architecture during its unsupervised training process to represents the hierarchical relation of the data. However, the dynamical algorithm of the GHSOM is sensitive to the presence of noise and outliers, and the model will no longer preserve the topology of the data space as we will show in this paper. The outliers introduce an influence to the GHSOM model during the training process by locating prototypes far from the majority of data and generating maps for few samples data. Therefore, the network will not effectively represent the topological structure of the data under study. In this paper, we propose a variant to the GHSOM algorithm that is robust under the presence of outliers in the data by being resistant to these deviations. We call this algorithm Robust GHSOM (RGHSOM). We will illustrate our technique on synthetic and real data sets.

Robust self-organizing maps

The Self Organizing Map (SOM) model is an unsupervised learning neural network that has been successfully applied as a data mining tool. The advantages of the SOMs are that they preserve the topology of the data space, they project high dimensional data to a lower dimension representation scheme, and are able to find similarities in the data.