Kazuhiro Tokunaga

Modular network SOM

This study aims to develop a generalized framework of an SOM called a modular network SOM (mnSOM). The mnSOM has an array structure consisting of functional modules that are trainable neural networks, e.g., multi-layer perceptrons (MLPs), instead of the vector units of the conventional SOM. In the case of MLP-modules, an mnSOM learns a group of systems or functions in terms of the input-output relationships in parallel with generating a feature map of them. Thus an mnSOM with MLP modules is an SOM in function space rather than in vector space. In this paper, first, as an example, we focus on a class of mnSOM that consists of MLP modules and introduce the architecture and algorithm. Then, a more generalized framework is described. Finally, some simulation results of an MLP-module-mnSOM are presented.

Modular network SOM (mnSOM): from vector space to function space

Kohonens self-organizing map (SOM), which performs topology-preserving transformation from a high dimensional data vector space to a low-dimensional map space, provides a powerful tool for data analysis, classification and visualization in many application fields. Despite its power, SOM can only deal with vectorized data, although many expansions have been proposed for various data-type cases. This study aims to develop a novel generalization of SOM called modular network SOM (mnSOM), which enables users to deal with general data classes in a consistent manner. mnSOM has an array structure consisting of function modules that are trainable neural networks, e.g. multi-layer perceptrons (MLPs), instead of the vector units of the conventional SOM family. In the case of MLP-modules, mnSOM learns a group of systems or functions in terms of the input-output relationships, and at the same time, mnSOM generates a feature map that shows distances between the learned systems. Thus, mnSOM with MLP modules is an SOM in function space rather than in vector space. From this point of view, the conventional SOM of Kohonens can be regarded as a special case of mnSOM, the modules consisting of fixed-value bias units. In this paper, mnSOM with MLP modules is described along with some application examples.

Requirements for the learning of multiple dynamics

The aim of this work is to discover the principles of learning a group of dynamical systems. The learning mechanism, which is referred to as the Learning Algorithm of Multiple Dynamics (LAMD), is expected to satisfy the following four requirements. (i) Given a set of time-series sequences for training, estimate the dynamics and their latent variables. (ii) Order the dynamical systems according to the similarities between them. (iii) Interpolate intermediate dynamics from the given dynamics. (iv) After training, the LAMD should be able to identify or classify new sequences. For this purpose several algorithms have been proposed, such as the Recurrent Neural Network with Parametric Bias and the modular network SOM with recurrent network modules. In this paper, it is shown that these types of algorithms do not satisfy the above requirements, but can be improved by normalization of estimated latent variables. This confirms that the estimation process of latent variables plays an important role in the LAMD. Finally, we show that a fully latent space model is required to satisfy the requirements, for which purpose a SOM with a higher-rank, such as a SOM2, is best suited.

Generalization of the self-organizing map: from artificial neural networks to artificial cortexes

This paper presents a generalized framework of a self-organizing map (SOM) applicable to more extended data classes rather than vector data. A modular structure is adopted to realize such generalization; thus, it is called a modular network SOM (mnSOM), in which each reference vector unit of a conventional SOM is replaced by a functional module. Since users can choose the functional module from any trainable architecture such as neural networks, the mnSOM has a lot of flexibility as well as high data processing ability. In this paper, the essential idea is first introduced and then its theory is described.

Modular network SOM: theory, algorithm and applications

The modular network SOM (mnSOM) proposed by authors is an extension and generalization of a conventional SOM in which each nodal unit is replaced by a module such as a neural network. It is expected that the mnSOM will extend the area of applications beyond that of a conventional SOM. We set out to establish the theory and algorithm of a mnSOM, and to apply it to several research topics, to create a fundamental technology that is generally usable only in expensive studies. In this paper, the theory and the algorithm of the mnSOM are reported; moreover, the results of applications of the mnSOM are presented.

Growing graph network based on an online gaussian mixture model

In this paper, the author proposes a growing neural network based on an online Gaussian mixture model, in which mechanisms are included for growing Gaussian kernels and finding topologies between kernels using graph paths. The proposed method has the following advantages compared with conventional growing neural networks: no permanent increase in nodes (Gaussian kernels), robustness to noise, and increased speed of constructing networks. This paper presents the theory and algorithm for the proposed method and the results of verification experiments using artificial data.

Realized through a Marriage with Modular-Networks

This paper presents a new development of self-organizing maps (SOM), realized by combining them with the idea of a modular network. This we called a modular network SOM (mnSOM) in which each reference vector unit of a conventional SOM is replaced by a functional module. Since users can choose the functional module from any trainable architecture such as neural networks, the mnSOM is very flexible as well as having a high data processing ability. In this paper, we first introduce the basic idea and then describe its theory. Finally we introduce some applications of mnSOMs.