Atsushi Masumori

Learning by Stimulation Avoidance as a Primary Principle of Spiking Neural Networks Dynamics

Practical implementation of the concept of reward has deep implications on what artificial-life based systems can learn and how they learn it. How can a system distinguish between useful behavior and harmful behavior? In this paper we implement reward/punishment as the removal/application of a stimulation to a recurrent spiking neural network with spiketiming dependent plasticity. This implementation embodies the concept of reward at the level of the neuron, making learning mechanisms ubiquitous to the network. We show that this low-level learning scales up to the network level: the network learns arbitrary spatio-temporal firing patterns purely by interacting with the environment, from a random initial state where virtually no knowledge is available. This approach yields fast, noise-robust results.

Emergence of Sense-Making Behavior by the Stimulus Avoidance Principle: Experiments on a Robot Behavior Controlled by Cultured Neuronal Cells.

Robot experiments using real cultured neuronal cells as controllers are a way to explore the idea of embodied cognition. Real cultured neuronal cells have innate plasticity, and a sensorimotor coupling is expected to develop a neural circuit. Previous studies have suggested that a dissociated neuronal culture has two properties: i) modifiability of connection between neurons by external stimuli and ii) stability of the connection without external stimuli. If cultured neuronal cells are embodied by coupling to an environment, they learn to avoid external stimulation. We call this mechanism a “learning by stimulation avoidance” principle. We try to demonstrate that adaptive behavior, like wall avoidance, can emerge spontaneously from embodied cultured neuronal cells. In this study, we developed a system in which a robot moves in a real environment and is controlled by cultured neuronal cells growing on a glass plate. We used a high-density complementary metal-oxide-semiconductor array to monitor the neural dynamics. We then conducted a robotic experiment using this platform. The results showed that wall-avoidance behavior by a robot can be enhanced spontaneously without giving any reward from the external environment.

Learning by stimulation avoidance: A principle to control spiking neural networks dynamics

Learning based on networks of real neurons, and learning based on biologically inspired models of neural networks, have yet to find general learning rules leading to widespread applications. In this paper, we argue for the existence of a principle allowing to steer the dynamics of a biologically inspired neural network. Using carefully timed external stimulation, the network can be driven towards a desired dynamical state. We term this principle “Learning by Stimulation Avoidance” (LSA). We demonstrate through simulation that the minimal sufficient conditions leading to LSA in artificial networks are also sufficient to reproduce learning results similar to those obtained in biological neurons by Shahaf and Marom, and in addition explains synaptic pruning. We examined the underlying mechanism by simulating a small network of 3 neurons, then scaled it up to a hundred neurons. We show that LSA has a higher explanatory power than existing hypotheses about the response of biological neural networks to external simulation, and can be used as a learning rule for an embodied application: learning of wall avoidance by a simulated robot. In other works, reinforcement learning with spiking networks can be obtained through global reward signals akin simulating the dopamine system; we believe that this is the first project demonstrating sensory-motor learning with random spiking networks through Hebbian learning relying on environmental conditions without a separate reward system.

Morphological computation on two dimensional self-assembly system

Self-assembly is a process in which components autonomously organized into structure without external direction. There are many researches of self-assembly system at molecular scale to macro scale. Such a self-assembly system may serve as the new production method with the characteristics, such as decomposability, self-repairing, self-replicating and adaptability. For developing such a new production method, an understanding of the interaction of shape and pattern, which can be called as morphological computation, is important.