Hiroshi Wakuya

Bi-directional computing architecture for time series prediction

A number of neural network models and training procedures for time series prediction have been proposed in the technical literature. These models studied for different time-variant data sets have typically used uni-directional computation flow or its modifications. In this study, on the contrary, the concept of bi-directional computational style is proposed and applied to prediction tasks. A bi-directional neural network model consists of two subnetworks performing two types of signal transformations bi-directionally. The networks also receive complementary signals from each other through mutual connections. The model not only deals with the conventional future prediction task, but also with the past prediction, an additional task from the viewpoint of the conventional approach. An improvement of the performance is achieved through making use of the future-past information integration. Since the coupling effects help the proposed model improve its performance, it is found that the prediction score is better than with the traditional uni-directional method. The bi-directional predicting architecture has been found to perform better than the conventional one when tested with standard benchmark sunspots data.

Time series prediction by a neural network model based on bi‐directional computation style: A study on generalization performance with the computer‐generated time series “Data Set D”

The principal goal of time series prediction is the enhancement of prediction accuracy. To achieve this goal, most previous investigations have adopted the so-called uni-directional computation style, focusing only on the forward-time direction (presentfuture). This paper adopts a different approach, the bi-directional computation style, and applies it to real-time series prediction tasks. The objective of this style is to improve the accuracy of time series prediction by means of interaction between the forward-time direction processing system, which only predicts the future, and a separate backward-time processing system. Good results have previously been obtained on sunspot data, but since these involve time series that are short, there has been insufficient investigation of generalization performance. We therefore apply the proposed technique to different time series data in order to investigate further the generalization performance with untraining data. The time series data called “Data Set D” used here were artificially generated by a computer and are distinguished by their great length, consisting of 100,000 points. Computer simulations indicate that on average, an improvement in dealing with untraining data equal to that for training data is achievable. In addition, better prediction accuracy is achieved in areas where the conventional method is weak.

Time series prediction by a neural network model based on the bi-directional computation style

A number of neural network models and training procedures for time series prediction have been proposed in the technical literature. These models typically used uni-directional computation flow or its modifications. In this study a novel concept of bi-directional computation style is proposed and applied to prediction tasks. Since the coupling effects between the future prediction system and the past prediction system help the proposed model improve its performance, it is found that the prediction score is better than with the traditional uni-directional method. The bi-directional predicting architecture has been found to perform better than the conventional one when tested with standard benchmark sunspots data.

Bottom-Up Learning of Hierarchical Models in a Class of Deterministic Pomdp Environments

Abstract The theory of partially observable Markov decision processes (POMDPs) is a useful tool for developing various intelligent agents, and learning hierarchical POMDP models is one of the key approaches for building such agents when the environments of the agents are unknown and large. To learn hierarchical models, bottom-up learning methods in which learning takes place in a layer-by-layer manner from the lowest to the highest layer are already extensively used in some research fields such as hidden Markov models and neural networks. However, little attention has been paid to bottom-up approaches for learning POMDP models. In this paper, we present a novel bottom-up learning algorithm for hierarchical POMDP models and prove that, by using this algorithm, a perfect model (i.e., a model that can perfectly predict future observations) can be learned at least in a class of deterministic POMDP environments

Temporal Signal Processing by Feedback SOM: An Application to On-line Character Recognition Task

An Elman-type feedback SOM (EFSOM) is a revised version of the standard SOM for dealing with time-variant information. In order to estimate its performance, an on-line character recognition task is tried with a large-scale training data set in this paper. At the same time, an idea for increasing the number of firing neurons in the competitive layer is examined to improve its trainability by means of enriching the state information. After the EFSOM is trained successfully, some validation tests for spatial displacement and temporal elasticity are also investigated. As a result of computer simulations, it is found that the EFSOM shows good performance and has good temporal signal processing ability.

A Study on Scheduling Function of a Magnetic Parameter in a Virtual Magnetic Diminuendo Method

A virtual magnetic diminuendo method proposed recently is originally inspired by the analogy between the Hopfield network and the spin glass. It is a simple and ingenious idea for solving combinatorial optimization problems, because only a threshold of Hopfield network is controlled from a negative value to zero as a magnetic parameter, which is newly introduced in this study. According to the preliminary study, it seems to be clear experimentally that the proposed method is effective. In order to carry out further considerations, a scheduling function of the virtual magnetic parameter is investigated in this study. As a result of some computer simulations with the crossbar switch problem, it is found that changing polarity (negative i¾? positive) of the magnetic parameter must be effective to improve the score.

Acquired sensorimotor coordinated signal transformation in a bi-directional neural network model

In general, a biological neural system is divided into two major parts, one is a neural system for motor control and the other is that for sensory reception, and they work not independently but cooperatively to perform various kinds of complex activities. Therefore its disorders produce strange phenomena. In a previous study, it was observed that a sensorimotor coordinated neural network model generates a trained pattern reversely when some special condition is provided. In this paper, computer simulation studies are performed from the viewpoints of neural information processing to investigate its sensorimotor coordination which is developed throughout the training. From the results, it is found that the way for information integration developed antagonistically or not plays an important role in performing signal transformation.

A study on an integrated neural network model of motor control and sensory reception systems

In general a neural information processing system consists of a motor control system and a sensory reception one. The former has been investigated frequently by a lot of researchers but the latter has not. Moreover integrated modelling of them seems to have been done even less. In this paper an integrated neural network model of motor control and sensory reception systems is discussed to investigate its signal processing ability, adaptability, and so on. But the most important thing in this study is how to develop sensorimotor coordination throughout the training phase. Some simulation studies are performed from these points of view. As the result, it is clear that an information processing ability and its adaptability are improved with a help of information integration.

An analysis of multi-dimensional data containing emphasized items by self-organizing map and its application to sightseeing information analysis

One of the attractive features which a self-organizing map (SOM) possesses is a topology-preserving projection from the input layer to the competitive layer. Generally speaking, its correspondence is developed through training based on information representations of the applied multi-dimensional data. A developed feature map in the competitive layer enables us easy to understand some underlying rules visually. By the way, an analysis of Saga Prefectural sightseeing information by a SOM has been tried so far. According to the results of preceding studies, applied various topics are divided into several groups successfully. Nevertheless, there are some items not reflected in them at all. This fact implies that representations of the applied data are not appropriate to develop the feature map which we have intended in advance. Then, to overcome this tough problem, a simple idea to emphasize particular items depending on our interests is introduced as pre-synaptic processing. As a result of some computer simulations, it is confirmed that the developed feature maps are modified adaptively depending on the emphasized coefficient. And, it is concluded that the proposed simple method is effective.

Fragmentization of distance measure for pattern generation by a self-organizing map

A self-organizing map (SOM) can be seen as an analytical tool to discover some underlying rules in the given data set. Based on such distinctive nature called topology-preserving projection, a new method for generating intermediate patterns was proposed. According to the results of preceding studies, most developed patterns are not morphing but dissolve. Then, in order to overcome this problem, a fragmentized distance measure is introduced in this paper. As a result of computer simulations, it is confirmed that some asymmetrical patterns are developed even though only symmetrical ones are used for training. This fact reminds us that the distance measure is quite essential, because a feature map is developed through training based on the distance measure.