Kazuyoshi Tsutsumi

Cross-coupled Hopfield nets via generalized-delta-rule-based internetworks

An integrated neural network architecture is proposed in which two Hopfield networks are cross-coupled via multilayered internetworks. A Lyapunov function for storing one state in each Hopfield network leads to the necessity of the delta rule for training two-layered linear internetworks. The generalized delta rule is also derived in the case of using multilayered internetworks with nonlinear hidden units. Each internetwork is composed of forward and backward subnetworks with the same connection weights. In the backward subnetworks, the deltas for connectionist learning are computed. At the same time, their final outputs and the inputs to them are utilized effectively for network relaxation via extra paths to Hopfield networks. Simulation in robotic motion control illustrates that the network can associate the smooth motion from a key configuration to the memorized one

An artificial modular neural network and its basic dynamical characteristics

Abstract. This work contains a proposition of an artificial modular neural network (MNN) in which every module network exchanges input/output information with others simultaneously. It further studies the basic dynamical characteristics of this network through both computer simulations and analytical considerations. A notable feature of this model is that it has generic representation with regard to the number of composed modules, network topologies, and classes of introduced interactions. The information processing of the MNN is described as the minimization of a total-energy function that consists of partial-energy functions for modules and their interactions, and the activity and weight dynamics are derived from the total-energy function under the Lyapunov stability condition. This concept was realized by Cross-Coupled Hopfield Nets (CCHN) that one of the authors proposed. In this paper, in order to investigate the basic dynamical properties of CCHN, we offer a representative model called Cross-Coupled Hopfield Nets with Local And Global Interactions (CCHN-LAGI) to which two distinct classes of interactions – local and global interactions – are introduced. Through a conventional test for associative memories, it is confirmed that our energy-function-based approach gives us proper dynamics of CCHN-LAGI even if the networks have different modularity. We also discuss the contribution of a single interaction and the joint contribution of the two distinct interactions through the eigenvalue analysis of connection matrices.

Underwater Robot with a Buoyancy Control System Based on the Spermaceti Oil Hypothesis

The goal of this paper is to develop an underwater robot with a buoyancy control system based on the spermaceti oil hypothesis. Sperm whales have a spermaceti organ in their head that is filled with spermaceti oil. Spermaceti oil is high quality oil and was used as material for candles, lubricant, and so on. There is a hypothesis about spermaceti oil that insists that sperm whales melt and congeal their spermaceti oil and change the volume of the oil to control their own buoyancy. This hypothesis appears suitable for the underwater robot because no materials for the ballast, such as sea water taken in at another place and iron, are discarded in the sea. To choose the best material as a spermaceti oil substitute, we measured the densities of four materials at both liquid and solid states, and calculated their buoyancy differences between both states. From the results, we concluded that the paraffin wax was the best material because its buoyancy difference is the largest of the four and its melting point is relatively low. Next, we directly measured the buoyancy of the paraffin wax and found that a particular arrangement of nichrome wire, which heats the oil, increases the level of buoyancy. Finally, we developed an underwater robot with a buoyancy control system based on this hypothesis. We measured its buoyancy and succeeded in surfacing of the robot in a small water tank

Higher degree error backpropagation in cross-coupled Hopfield nets

The author discusses higher-degree error backpropagation in cross-coupled Hopfield nets employing exponential energy functions for cross-coupling. He constructs a Lyapunov function to derive a total network architecture and a learning algorithm for training nonlinear multilayered internetworks. In the derived architecture, each internetwork for cross-coupling has a forward subnet and a backward subnet. The backward subnet consists of multiple planes, each of which has the same connection weights as those in the forward subnet. From linear to higher degree errors respectively backpropagate in the different planes. The final outputs from the multiple planes are utilized effectively for network relaxation. At the same time, the interactions between the errors in each plane and the signals in the forward subnet contribute to the connectionistic learning. The result obtained indicates that higher-degree error backpropagation is effective for fast learning.<<ETX>>

Hopping height control of an active suspension type leg module based on reinforcement learning and a neural network

The aim of our study is to have a hopping module to control the height of hopping in an environment where the control parameters are unknown. This will lead to the development of a system for building dynamic walking robots. Assuming that a hopping module can be controlled by a spring and a DC motor, we placed a built-in learning system in the module that consists of reinforcement learning (RL) for identification and layered neural networks (NN) for generalization. By using this learning system, we simulated autonomous adjustment control in order to obtain the optimum DC motor angular velocity, which enables the module to hop to an arbitrary height. As a result, we can design a regulator that has the advantage of both RL and NN, and have laid the foundation for further developments to apply the algorithms of learning to practical walking robots.

Function-selectable tactile sensing system with morphological change

This paper presents a novel approach for active tactile sensation that utilizes soft morphological deformation. This work is inspired by human fingers wrinkles, which appear after a long time soaking in water, and has been indicated as an efficient mean for enhancement of gripping in wet environment. We created a tactile sensing system that is an integration of actuation (pneumatic actuator) and sensing elements (strain gauges). This device can change its morphology (wrinkle patterns) so that the posture of embedded sensing elements can vary, then generate different responses depending on sensing tasks. As a result, this device can actively select its sensing function depending on sensing task. In this paper, the sensing device is both sensitive to contact action and sliding action by using only one types of strain gauges. This work can be extended to a wide range of sensing elements (not only strain gauges), and considered to give impact to the field.

A Continuous-Time Model of AutoassociativeNeural Memories Utilizing the Noise-SubspaceDynamics

This paper presents a continuous-time model of Autoassociative Neural Memories (ANMs) which correspond to a modified version of pseudoinverse-type ANMs. This ANM model is derived from minimizing the energy function for a modular neural network. Through the eigendecomposition of the connection matrix, we show that the dynamical properties of the ANM are qualitatively different in the two state subspaces: a pattern-subspace and a noise-subspace. The proposed ANM has a distinctive feature in the noise-subspace dynamics. The size of basins of attraction can be varied by controlling the contribution of the noise-subspace dynamics to the whole network. The first simulation confirms this attractive feature. In the second simulation, we investigate the performance robustness of the ANM for several kinds of correlated pattern sets. These simulation results confirm the usefulness of the proposed ANM.

Relaxing in a warped space: an effect due to the cooperation of static and dynamical neurons

This paper proposes a module-based neural network composed of static and dynamical neurons, and discusses what effect can be produces by the integration of mapping and relaxation. The proposed network can be obtained when a specially-designed total energy function is minimized. Although the derivation process is similar to the case of the well-known Hopfield network, state variables in the network included, in addition to the addition to the output and the potential of dynamical neurons, another type of state variable converted from the direct output of dynamical neurons using a mapping function. If we suppose a layered network with only static neurons corresponding to the mapping function, the layered network comprises a forward subnet and a backward subnet; connection weights in the forward and backward subnets are modified based on propagated error signals through the backward subnet, and, at the same time, the final output of the backward subnet is utilized for overall network-dynamics. As a result, the proposed network offers a framework in which relaxation can be carried out in a warped space due to the cooperation of static and dynamical neurons. Furthermore, it gives an interpretation for the backpropagated error signals in the case of delta-rule based learning; although the backward subnet for the calculating the delta values is usually assumed to be virtual, it must actually exist for network relaxation in the proposed network.

Dynamics and local minima of a simple neural network for optimization

In the neural computation framework proposed by Hopfield and Tank (1985), the total energy for deriving the network dynamics is composed of multiple sub-energy functions, each of which is expressed by a quadratic form. Their work has had a great impact on various fields in science and engineering, and has produced a lot of active discussions about neural network based computation. However, problems involving local minima have not been solved yet, and so there are no methods for obtaining global optima. We can design various types of energy functions to solve an optimization problem, such as high-dimensional, low-dimensional, and/or non-linear functions. In this paper, we treat a simple total-energy function and study the nature of the minima in the dynamics. We further discuss some techniques to avoid convergence towards inadequate minima.

The estimation of cross-coupled Hopfield nets as an interactive modular neural network

This paper describes the effects on the association performance of cross-coupled Hopfield nets (CCHN) due to the increase in the number of modules M, in order to clarify the usefulness as an interactive modular neural network. The association performance of a cross-coupled Hopfield net with many-to-many mapping internetworks (CCHN-MMMI), which is an extended version of CCHN is investigated through computer simulations. The simulation results show that the performance of CCHN-MMMI is greatly improved as compared with that of a conventional Hopfield network (HN) as long as M is not too large. However, in our simulation, the performance of CCHN-MMMI with M/spl ges/16 is worse than or equal to that of HN. In order to clarify the factors on its performance deterioration, the features-of average activities in modules and interactions are separately investigated. As a result, we clarify three factors on the deterioration and suggest some approaches to solve them.<<ETX>>