Olli Simula

Engineering applications of the self-organizing map

The self-organizing map (SOM) method is a new, powerful software tool for the visualization of high-dimensional data. It converts complex, nonlinear statistical relationships between high-dimensional data into simple geometric relationships on a low-dimensional display. As it thereby compresses information while preserving the most important topological and metric relationships of the primary data elements on the display, it may also be thought to produce some kind of abstractions. The term self-organizing map signifies a class of mappings defined by error-theoretic considerations. In practice they result in certain unsupervised, competitive learning processes, computed by simple-looking SOM algorithms. Many industries have found the SOM-based software tools useful. The most important property of the SOM, orderliness of the input-output mapping, can be utilized for many tasks: reduction of the amount of training data, speeding up learning nonlinear interpolation and extrapolation, generalization, and effective compression of information for its transmission.

OP-ELM: Optimally Pruned Extreme Learning Machine

In this brief, the optimally pruned extreme learning machine (OP-ELM) methodology is presented. It is based on the original extreme learning machine (ELM) algorithm with additional steps to make it more robust and generic. The whole methodology is presented in detail and then applied to several regression and classification problems. Results for both computational time and accuracy (mean square error) are compared to the original ELM and to three other widely used methodologies: multilayer perceptron (MLP), support vector machine (SVM), and Gaussian process (GP). As the experiments for both regression and classification illustrate, the proposed OP-ELM methodology performs several orders of magnitude faster than the other algorithms used in this brief, except the original ELM. Despite the simplicity and fast performance, the OP-ELM is still able to maintain an accuracy that is comparable to the performance of the SVM. A toolbox for the OP-ELM is publicly available online.

TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization

In this paper an improvement of the optimally pruned extreme learning machine (OP-ELM) in the form of a L2 regularization penalty applied within the OP-ELM is proposed. The OP-ELM originally proposes a wrapper methodology around the extreme learning machine (ELM) meant to reduce the sensitivity of the ELM to irrelevant variables and obtain more parsimonious models thanks to neuron pruning. The proposed modification of the OP-ELM uses a cascade of two regularization penalties: first a L1 penalty to rank the neurons of the hidden layer, followed by a L2 penalty on the regression weights (regression between hidden layer and output layer) for numerical stability and efficient pruning of the neurons. The new methodology is tested against state of the art methods such as support vector machines or Gaussian processes and the original ELM and OP-ELM, on 11 different data sets; it systematically outperforms the OP-ELM (average of 27% better mean square error) and provides more reliable results – in terms of standard deviation of the results – while remaining always less than one order of magnitude slower than the OP-ELM.

Advanced analysis methods for 3G cellular networks

The operation and maintenance of the third generation (3G) mobile networks will be challenging. These networks will be strongly service driven, and this approach differs significantly from the traditional speech dominated in the second generation (2G) approach. Compared to 2G, in 3G, the mobile cells interact and interfere with each other more, they have hundreds of adjustable parameters, and they monitor and record data related to several hundreds of different variables in each cell. This paper shows that a neural network algorithm called the self-organizing map, together with a conventional clustering method like the k-means, can effectively be used to simplify and focus network analysis. It is shown that these algorithms help in visualizing and grouping similarly behaving cells. Thus, it is easier for a human expert to discern different states of the network. This makes it possible to perform faster and more efficient troubleshooting and optimization of the parameters of the cells. The presented methods are applicable for different radio access network technologies.

Process State Monitoring Using Self-Organizing Maps

Self-organizing map algorithm has the ability to create a model for a system that is not exactly known a priori. Using this kind of model we can classify the system states and detect the abnormal ones. In this paper, an application of the feature map to detect operational states of a device is presented. The features used in the map are the measurements from the device describing its operational and environmental parameters.

Combining linear equalization and self-organizing adaptation in dynamic discrete-signal detection

An adaptive algorithm combining traditional linear equalization techniques and a self-organizing neural learning algorithm is presented. The results show that the performance of the neural equalizer is insensitive to nonlinear learning distortions in dynamic discrete-signal detection. Stabilization of the self-organizing map during undistorted transmission has to be further considered to decrease the absolute mean-square error (MSE) rate of the neural equalizer. The error is due to oscillations in the self-organizing map, mainly caused by the neighborhood learning. The oscillations can be decreased by taking more samples to the map before adapting the mi values and by decreasing the neighborhood learning parameter β

An approach to automated interpretation of SOM

The objective of this work was to develop automatic tools for post-processing of SOMs, especially in the context of hierarchical data — data where each higher level object consists of a varying number of lower level objects. Both low and high level data is available and needs to be utilized. The information from lower levels is transferred to higher level using data histograms of lower level clusters. The clusters are formed and interpreted automatically so as to summarize the information given by the SOM, and to produce meaningful indicators that are useful also to problem domain experts. The results show that the approach works well at least in the case study of pulp and paper mills technology data.

Neural analysis of mobile radio access network

The self-organizing map (SOM) is an efficient tool for visualization and clustering of multidimensional data. It transforms the input vectors on two-dimensional grid of prototype vectors and orders them. The ordered prototype vectors are easier to visualize and explore than the original data. Mobile networks produce a huge amount of spatiotemporal data. The data consists of parameters of base stations (BS) and quality information of calls. There are two alternatives in starting the data analysis. We can build either a general one-cell-model trained using state vectors from all cells, or a model of the network using state vectors with parameters from all mobile cells. In both methods, further analysis is needed to understand the reasons for various operational states of the entire network.

Editorial: Time series prediction competition: The CATS benchmark

This paper presents the CATS Benchmark and the results of the competition organised during the IJCNN’04 conference in Budapest. Twenty-four papers and predictions have been submitted and seventeen have been selected. The goal of the competition was the prediction of 100 missing values divided into five groups of twenty consecutive values.

Visualization of fluid-bed granulation with self-organizing maps

The degree of the instrumentation of pharmaceutical unit operations has increased. This instrumentation provides information of the state of the process and can be used for both process control and research. However, on-line process data is usually multidimensional, and is difficult to study with traditional trends and scatter plots. The Self-Organizing Map (SOM) is a recognized tool for dimension reduction and process state monitoring. The basics of the SOM and the application to on-line data collected from a fluid-bed granulation process are presented. As a batch process, granulation traversed through a number of process states, which was visualized with SOM as a two-dimensional map. In addition, it is demonstrated how the differences between granulation batches can be studied. The results suggest that SOM together with new in-line process analytical solutions support the in-process control of the pharmaceutical unit operations. Further, a novel research tool for understanding the phenomena during processing is achieved.