M.L. de Andrade Netto

Cluster analysis using self-organizing maps and image processing techniques

Cluster analysis methods are used to classify R unlabeled objects in a P-dimensional space into groups based on their similarities. This paper focuses on the use of self organising maps (SOM) as a clustering tool and some of the additional procedures required to enable a meaningful clusters interpretation in the trained map. Topics discussed here include the usage of mathematical morphology segmentation method watershed to segment the neurons distance image (u-matrix). Finding good watershed markers and the modification of the u-matrix homotopy are discussed. The algorithm automatically produces labeled sets of neurons that are related to the clusters in the P-dimensional space. An example of non-spherical, complex shaped and nonlinearly separable clusters illustrate the capabilities of the method.

A new tree-structured self-organizing map for data analysis

This paper presents a new algorithm for dynamical generation of a hierarchical structure of self-organizing maps (SOM) with applications to data analysis. Different from other tree-structured SOM approaches, in this case the tree nodes are actually maps. From top to down, maps are automatically segmented by using the U-matrix information, which presents relations between neighboring neurons. The automatic map partitioning algorithm is based on mathematical morphology segmentation and it is applied to each map in each level of the hierarchy. Clusters of neurons are automatically identified and labeled and generate new sub-maps. Data are partitioned accordingly the label of its best match unit in each level of the tree. The algorithm may be seen as a recursive partition clustering method with multiple prototypes cluster representation, which enables the discoveries of clusters in a variety of geometrical shapes.

Data Clustering using Self-Organizing Maps segmented by Mathematic Morphology and Simplified Cluster Validity Indexes: an application in remotely sensed images

This paper presents a cluster analysis method which automatically finds the number of clusters as well as the partitioning of a data set without any type of interaction with the user. The data clustering is made using the self-organizing (or Kohonen) map (SOM). Different partitions of the trained SOM are obtained from different segmentations of the U-matrix (a neuron-distance image) that are generated by means of mathematical morphology techniques. The different partitions of the trained SOM produce different partitions for the data set which are evaluated by cluster validity indexes. To reduce the computational cost of the cluster analysis process this work also proposes the simplification of cluster validity indexes using the statistical properties of the SOM. The proposed methodology is applied in the cluster analysis of remotely sensed images.

Automatic data classification by a hierarchy of self-organizing maps

Clustering is the process by which discrete objects are assigned to groups that have similar characteristics. Self-organizing maps (SOM) have been widely used as a data visualization tool. Some of their advantages include information compression and density estimation while trying to preserve the topological and metric relationships of the primary data items. For using SOM as a clustering tool additional procedures are required to interpret the mapping obtained through unsupervised learning. Costa and Netto (1999) described the usage of image analysis and mathematical morphology to find automatically regions of similar neurons and their borders. The purpose of this paper is to enhance the clustering process in order to detail the underlying structure obtained in a first trial. Groups of neurons associated to clusters are further subdivided in new sub-networks, generating a tree-like structure of SOMs. Differently to other hierarchical SOM approaches, the number of sub-nets for a given SOM in a given height of the tree is not specified in advance. The process can be seen as a dynamic strategy for cluster discovery.

Unit-growing learning optimizing the solvability condition for model-free regression

The universal approximation capability exhibited by one-hidden layer neural networks is explored to produce a supervised unit-growing learning for model-free nonlinear regression. The development is based on the solvability condition, which attests that the ability to learn a specific learning set increases with the number of nodes in the hidden layer. Since the training process operates the hidden nodes individually, a pertinent activation function can be iteratively developed for each node as a function of the learning set. The optimization of the solvability condition gives rise to neural networks of minimum dimension, an important step toward improving generalization.The universal approximation capability exhibited by one-hidden layer neural networks is explored to produce a supervised unit-growing learning for model-free nonlinear regression. The development is based on the solvability condition, which attests that the ability to learn a specific learning set increases with the number of nodes in the hidden layer. Since the training process operates the hidden nodes individually, a pertinent activation function can be iteratively developed for each node as a function of the learning set. The optimization of the solvability condition gives rise to neural networks of minimum dimension, an important step toward improving generalization.

Projection pursuit and the solvability condition applied to constructive learning

Single hidden layer neural networks with supervised learning have been successfully applied to approximate unknown functions defined in compact functional spaces. The more advanced results also give rates of convergence, stipulating how many hidden neurons with a given activation function should be used to achieve a specific order of approximation. However, independently of the activation function employed, these connectionist models for function approximation suffer from a severe limitation: all hidden neurons use the same activation function. If each activation function of a hidden neuron is optimally defined for every approximation problem, then better rates of convergence will be achieved. This is exactly the purpose of constructive learning using projection pursuit techniques. Since the training process operates the hidden neurons individually, a pertinent activation function employing automatic smoothing splines can be iteratively developed for each neuron as a function of the learning set. We apply projection pursuit in association with the optimization of the solvability condition, giving rise to a more efficient and accurate computational learning algorithm.Single hidden layer neural networks with supervised learning have been successfully applied to approximate unknown functions defined in compact functional spaces. The more advanced results also give rates of convergence, stipulating how many hidden neurons with a given activation function should be used to achieve a specific order of approximation. However, independently of the activation function employed, these connectionist models for function approximation suffer from a severe limitation: all hidden neurons use the same activation function. If each activation function of a hidden neuron is optimally defined for every approximation problem, then better rates of convergence will be achieved. This is exactly the purpose of constructive learning using projection pursuit techniques. Since the training process operates the hidden neurons individually, a pertinent activation function employing automatic smoothing splines can be iteratively developed for each neuron as a function of the learning set. We apply projection pursuit in association with the optimization of the solvability condition, giving rise to a more efficient and accurate computational learning algorithm.

Automatic remotely sensed data clustering by tree-structured self-organizing maps

This work presents a clusters analysis method which automatically finds the number of clusters as well as the partitioning of data set in a remotely sensed image without any type of assistance of an image analyst. The data clustering is made using the self-organizing (or Kohonen) map (SOM) and the techniques proposed by Costa & Netto (2001) for automatic partition of trained SOM networks and for generating a hierarchy of maps based on the detected data clusters. The proposed clustering method has been applied on a LANDSAT/TM image and its performance was compared with that of K-means algorithm, conventionally used for remotely sensed images.

Parts classification in assembly lines using multilayer feedforward neural networks

The paper describes a low cost system for a position, scale, and rotation invariant classification of mechanical parts in assembly lines using multilayer feedforward neural networks. After image acquisition, moment invariants are calculated for each significant region in the input image. Different network sizes were tested for classifying these features and the authors compare these results with the traditional k-nearest neighbor (k-NN), for different k values. Hybrid strategies were adopted for training the networks. They used deterministic methods, such as conjugate gradient and Levenberg-Marquardt algorithms, combined with a stochastic method, simulated annealing. The system deals with digital images with an unknown number of unoccluded object types and poses. Results show that, in this case, artificial neural networks had better generalization capability than k-NN; despite geometrical transformations and other degradations over the images. The systems runs on low cost personal computers and can therefore be easily adapted for use even by small factories.

Exploring the nonlinear dynamic behavior of artificial neural networks

This paper explores the universal approximation capability exhibited by neural networks in the development of suitable architectures and associated training processes for nonlinear discrete-time dynamic system representation. The resulting architectures include recurrent and non recurrent multilayer neural networks and the derived training processes can be seen as optimization problems. Particular attention is given to the investigation of the dynamic behavior of a recurrent processing unit. >This paper explores the universal approximation capability exhibited by neural networks in the development of suitable architectures and associated training processes for nonlinear discrete-time dynamic system representation. The resulting architectures include recurrent and non recurrent multilayer neural networks and the derived training processes can be seen as optimization problems. Particular attention is given to the investigation of the dynamic behavior of a recurrent processing unit.<<ETX>>

A neural hybrid system for large memory association

A neural hybrid system based on Kohonen and Hopfield networks is proposed for memory association. It uses a heuristic approach to split a total set of patterns into various subsets with the aim to increase performance of the parallel architecture of Hopfield networks (PAHN). This architecture avoids several spurious states enabling a pattern storage capacity larger then permitted by a typical Hopfield network. The strategy consists of a method to sort patterns with the SOM algorithm and distribute them into these subsets in such a way that the patterns of the same subset are to be as more orthogonal as possible among themselves. The results show that the strategy employed to distribute patterns in subsets works well when compared with the random distributions and with the exhaustive approach. The results also show that the proposed heuristic lead to patterns subsets that enable more robust memory retrieval.