Marie Cottrell

Theoretical Aspects of the SOM Algorithm

Abstract The SOM algorithm is very astonishing. On the one hand, it is very simple to write down and to simulate, its practical properties are clear and easy to observe. However, on the other hand, its theoretical properties still remain without proof in the general case, despite the tremendous efforts of several authors. In this paper, we briefly review the previous results and provide some conjectures for future work.

Large deviations and rare events in the study of stochastic algorithms

New asymptotics formulas for the mean exit time from an almost stable domain of a discrete-time Markov process are obtained. An original fast simulation method is also proposed. The mathematical background involves the large deviation theorems and approximations by a diffusion process. We are chiefly concerned with the classical Robbins-Monroe algorithm. The validity of the results are tested on examples from the ALOHA system (a satellite type communication algorithm).

Neural modeling for time series: A statistical stepwise method for weight elimination

Many authors use feedforward neural networks for modeling and forecasting time series. Most of these applications are mainly experimental, and it is often difficult to extract a general methodology from the published studies. In particular, the choice of architecture is a tricky problem. We try to combine the statistical techniques of linear and nonlinear time series with the connectionist approach. The asymptotical properties of the estimators lead us to propose a systematic methodology to determine which weights are nonsignificant and to eliminate them to simplify the architecture. This method (SSM or statistical stepwise method) is compared to other pruning techniques and is applied to some artificial series, to the famous Sunspots benchmark, and to daily electrical consumption data.

Batch and median neural gas

Neural Gas (NG) constitutes a very robust clustering algorithm given Euclidean data which does not suffer from the problem of local minima like simple vector quantization, or topological restrictions like the self-organizing map. Based on the cost function of NG, we introduce a batch variant of NG which shows much faster convergence and which can be interpreted as an optimization of the cost function by the Newton method. This formulation has the additional benefit that, based on the notion of the generalized median in analogy to Median SOM, a variant for non-vectorial proximity data can be introduced. We prove convergence of batch and median versions of NG, SOM, and k-means in a unified formulation, and we investigate the behavior of the algorithms in several experiments.

A stochastic model of retinotopy: A self organizing process

Following Kohonen and using the Hebb principle, we define a self orgaizing stochastic process, which is a simple modelization of the retinotopy, i.e. the establishment of well-ordered connexions between the retina and the cortex.We give some mathematical results about convergence of this process. These results are illustrated by computer simulations.

Statistical tools to assess the reliability of self-organizing maps

Results of neural network learning are always subject to some variability, due to the sensitivity to initial conditions, to convergence to local minima, and, sometimes more dramatically, to sampling variability. This paper presents a set of tools designed to assess the reliability of the results of self-organizing maps (SOM), i.e. to test on a statistical basis the confidence we can have on the result of a specific SOM. The tools concern the quantization error in a SOM, and the neighborhood relations (both at the level of a specific pair of observations and globally on the map). As a by-product, these measures also allow to assess the adequacy of the number of units chosen in a map. The tools may also be used to measure objectively how the SOM are less sensitive to non-linear optimization problems (local minima, convergence, etc.) than other neural network models.

The Kohonen Algorithm: A Powerful Tool for Analyzing and Representing Multidimensional Quantitative and Qualitative Data

The simultaneous analysis of quantitative and qualitative variables is not an easy task in general. When a linear model is appropriate, the Generalized Linear Models are commonly used with success. But when the intrinsic structure of the data is not at all linear, they give very poor and confusing results. In this paper, we extensively study how to use the (non linear) Kohonen maps to solve some of the interesting problems which are encountered in data analysis: how to realize a rapid and robust classification based on the quantitative variables, how to visualize the classes, their differences and homogeneity, how to cross the classification with the remaining qualitative variables to interpret the classification and put in evidence the most important explanatory variables.

Time series forecasting: Obtaining long term trends with self-organizing maps

Kohonen self-organisation maps are a well know classification tool, commonly used in a wide variety of problems, but with limited applications in time series forecasting context. In this paper, we propose a forecasting method specifically designed for multi-dimensional long-term trends prediction, with a double application of the Kohonen algorithm. Practical applications of the method are also presented.

SOM-based algorithms for qualitative variables

It is well known that the SOM algorithm achieves a clustering of data which can be interpreted as an extension of Principal Component Analysis, because of its topology-preserving property. But the SOM algorithm can only process real-valued data. In previous papers, we have proposed several methods based on the SOM algorithm to analyze categorical data, which is the case in survey data. In this paper, we present these methods in a unified manner. The first one (Kohonen Multiple Correspondence Analysis, KMCA) deals only with the modalities, while the two others (Kohonen Multiple Correspondence Analysis with individuals, KMCA_ind, Kohonen algorithm on DISJonctive table, KDISJ) can take into account the individuals, and the modalities simultaneously.

On the use of self-organizing maps to accelerate vector quantization

Self-organizing maps (SOM) are widely used for their topology preservation property: neighboring input vectors are quantified (or classified) either on the same location or on neighbor ones on a predefined grid. SOM are also widely used for their more classical vector quantization property. We show in this paper that using SOM instead of the more classical simple competitive learning (SCL) algorithm drastically increases the speed of convergence of the vector quantization process. This fact is demonstrated through extensive simulations on artificial and real examples, with specific SOM (fixed and decreasing neighborhoods) and SCL algorithms