Auke Jan Ijspeert

2008 Special Issue: Central pattern generators for locomotion control in animals and robots: A review

The problem of controlling locomotion is an area in which neuroscience and robotics can fruitfully interact. In this article, I will review research carried out on locomotor central pattern generators (CPGs), i.e. neural circuits capable of producing coordinated patterns of high-dimensional rhythmic output signals while receiving only simple, low-dimensional, input signals. The review will first cover neurobiological observations concerning locomotor CPGs and their numerical modelling, with a special focus on vertebrates. It will then cover how CPG models implemented as neural networks or systems of coupled oscillators can be used in robotics for controlling the locomotion of articulated robots. The review also presents how robots can be used as scientific tools to obtain a better understanding of the functioning of biological CPGs. Finally, various methods for designing CPGs to control specific modes of locomotion will be briefly reviewed. In this process, I will discuss different types of CPG models, the pros and cons of using CPGs with robots, and the pros and cons of using robots as scientific tools. Open research topics both in biology and in robotics will also be discussed.

From swimming to walking with a salamander robot driven by a spinal cord model

The transition from aquatic to terrestrial locomotion was a key development in vertebrate evolution. We present a spinal cord model and its implementation in an amphibious salamander robot that demonstrates how a primitive neural circuit for swimming can be extended by phylogenetically more recent limb oscillatory centers to explain the ability of salamanders to switch between swimming and walking. The model suggests neural mechanisms for modulation of velocity, direction, and type of gait that are relevant for all tetrapods. It predicts that limb oscillatory centers have lower intrinsic frequencies than body oscillatory centers, and we present biological data supporting this.

Movement imitation with nonlinear dynamical systems in humanoid robots

Presents an approach to movement planning, on-line trajectory modification, and imitation learning by representing movement plans based on a set of nonlinear differential equations with well-defined attractor dynamics. The resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy (CP) which is robust to strong external perturbations and that can be modified on-line by additional perceptual variables. We evaluate the system with a humanoid robot simulation and an actual humanoid robot. Experiments are presented for the imitation of three types of movements: reaching movements with one arm, drawing movements of 2-D patterns, and tennis swings. Our results demonstrate (a) that multi-joint human movements can be encoded successfully by the CPs, (b) that a learned movement policy can readily be reused to produce robust trajectories towards different targets, (c) that a policy fitted for one particular target provides a good predictor of human reaching movements towards neighboring targets, and (d) that the parameter space which encodes a policy is suitable for measuring to which extent two trajectories are qualitatively similar.

Dynamical movement primitives: Learning attractor models for motor behaviors

Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior (e.g., stable locomotion from a system of coupled oscillators under perceptual guidance). Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by means of a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander

Abstract. This article investigates the neural mechanisms underlying salamander locomotion, and develops a biologically plausible connectionist model of a central pattern generator capable of producing the typical aquatic and terrestrial gaits of the salamander. It investigates, in particular, what type of neural circuitry can produce and modulate the two locomotor programs identified within the salamanders spinal cord; namely, a traveling wave of neural activity for swimming and a standing wave for trotting. A two-dimensional biomechanical simulation of the salamanders body is developed whose muscle contraction is determined by the locomotion controller simulated as a leaky-integrator neural network. While the connectivity of the neural circuitry underlying locomotion in the salamander has not been decoded for the moment, this article presents the design of a neural circuit that has a general organization corresponding to that hypothesized by neurobiologists. In particular, the locomotion controller is based on a body central pattern generator (CPG) corresponding to a lamprey-like swimming controller, and is extended with a limb CPG for controlling the salamanders limbs. The complete controller is developed in three stages: first the development of segmental oscillators, second the development of intersegmental coupling for the making of a lamprey-like swimming CPG, and finally the development of the limb CPG and its coupling to the body CPG. A genetic algorithm is used to determine the parameters of the neural circuit for the different stages, given a high-level description of the desired state space trajectories of the different subnetworks. A controller is thus developed that can produce neural activities and locomotion gaits very similar to those observed in the real salamander. By varying the tonic (i.e. non-oscillating) excitation applied to the network, the speed, direction and type of gait can be varied.

Programmable central pattern generators: an application to biped locomotion control

We present a system of coupled nonlinear oscillators to be used as programmable central pattern generators, and apply it to control the locomotion of a humanoid robot. Central pattern generators are biological neural networks that can produce coordinated multidimensional rhythmic signals, under the control of simple input signals. They are found both in vertebrate and invertebrate animals for the control of locomotion. In this article, we present a novel system composed of coupled adaptive nonlinear oscillators that can learn arbitrary rhythmic signals in a supervised learning framework. Using adaptive rules implemented as differential equations, parameters such as intrinsic frequencies, amplitudes, and coupling weights are automatically adjusted to replicate a teaching signal. Once the teaching signal is removed, the trajectories remain embedded as the limit cycle of the dynamical system. An interesting aspect of this approach is that the learning is completely embedded into the dynamical system, and does not require external optimization algorithms. We use our system to encapsulate rhythmic trajectories for biped locomotion with a simulated humanoid robot, and demonstrate how it can be used to do online trajectory generation. The system can modulate the speed of locomotion, and even allow the reversal of direction (i.e. walking backwards). The integration of sensory feedback allows the online modulation of trajectories such as to increase the basin of stability of the gaits, and therefore the range of speeds that can be produced

Learning Movement Primitives

This paper discusses a comprehensive framework for modular motor control based on a recently developed theory of dynamic movement primitives (DMP). DMPs are a formulation of movement primitives with autonomous nonlinear differential equations, whose time evolution creates smooth kinematic control policies. Model-based control theory is used to convert the outputs of these policies into motor commands. By means of coupling terms, on-line modifications can be incorporated into the time evolution of the differential equations, thus providing a rather flexible and reactive framework for motor planning and execution. The linear parameterization of DMPs lends itself naturally to supervised learning from demonstration. Moreover, the temporal, scale, and translation invariance of the differential equations with respect to these parameters provides a useful means for movement recognition. A novel reinforcement learning technique based on natural stochastic policy gradients allows a general approach of improving DMPs by trial and error learning with respect to almost arbitrary optimization criteria. We demonstrate the different ingredients of the DMP approach in various examples, involving skill learning from demonstration on the humanoid robot DB, and learning biped walking from demonstration in simulation, including self-improvement of the movement patterns towards energy efficiency through resonance tuning.

AmphiBot I: An amphibious snake-like robot

This article presents a project that aims at constructing a biologically inspired amphibious snake-like robot. The robot is designed to be capable of anguilliform swimming like sea-snakes and lampreys in water and lateral undulatory locomotion like a snake on ground. Both the structure and the controller of the robot are inspired by elongate vertebrates. In particular, the locomotion of the robot is controlled by a central pattern generator (a system of coupled oscillators) that produces travelling waves of oscillations as limit cycle behavior. We present the design considerations behind the robot and its controller. Experiments are carried out to identify the types of travelling waves that optimize speed during lateral undulatory locomotion on ground. In particular, the optimal frequency, amplitude and wavelength are thus identified when the robot is crawling on a particular surface.

Dynamics systems vs. optimal control a unifying view

In the past, computational motor control has been approached from at least two major frameworks: the dynamic systems approach and the viewpoint of optimal control. The dynamic system approach emphasizes motor control as a process of self-organization between an animal and its environment. Nonlinear differential equations that can model entrainment and synchronization behavior are among the most favorable tools of dynamic systems modelers. In contrast, optimal control approaches view motor control as the evolutionary or development result of a nervous system that tries to optimize rather general organizational principles, e.g., energy consumption or accurate task achievement. Optimal control theory is usually employed to develop appropriate theories. Interestingly, there is rather little interaction between dynamic systems and optimal control modelers as the two approaches follow rather different philosophies and are often viewed as diametrically opposing. In this paper, we develop a computational approach to motor control that offers a unifying modeling framework for both dynamic systems and optimal control approaches. In discussions of several behavioral experiments and some theoretical and robotics studies, we demonstrate how our computational ideas allow both the representation of self-organizing processes and the optimization of movement based on reward criteria. Our modeling framework is rather simple and general, and opens opportunities to revisit many previous modeling results from this novel unifying view.

Online Optimization of Swimming and Crawling in an Amphibious Snake Robot

An important problem in the control of locomotion of robots with multiple degrees of freedom (e.g., biomimetic robots) is to adapt the locomotor patterns to the properties of the environment. This article addresses this problem for the locomotion of an amphibious snake robot, and aims at identifying fast swimming and crawling gaits for a variety of environments. Our approach uses a locomotion controller based on the biological concept of central pattern generators (CPGs) together with a gradient-free optimization method, Powells method. A key aspect of our approach is that the gaits are optimized online, i.e., while moving, rather than as an off-line optimization process. We present various experiments with the real robot and in simulation: swimming, crawling on horizontal ground, and crawling on slopes. For each of these different situations, the optimized gaits are compared with the results of systematic explorations of the parameter space. The main outcomes of the experiments are: 1) optimal gaits are significantly different from one medium to the other; 2) the optimums are usually peaked, i.e., speed rapidly becomes suboptimal when the parameters are moved away from the optimal values; 3) our approach finds optimal gaits in much fewer iterations than the systematic search; and 4) the CPG has no problem dealing with the abrupt parameter changes during the optimization process. The relevance for robotic locomotion control is discussed.