Jun Nakanishi

Movement imitation with nonlinear dynamical systems in humanoid robots

Presents an approach to movement planning, on-line trajectory modification, and imitation learning by representing movement plans based on a set of nonlinear differential equations with well-defined attractor dynamics. The resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy (CP) which is robust to strong external perturbations and that can be modified on-line by additional perceptual variables. We evaluate the system with a humanoid robot simulation and an actual humanoid robot. Experiments are presented for the imitation of three types of movements: reaching movements with one arm, drawing movements of 2-D patterns, and tennis swings. Our results demonstrate (a) that multi-joint human movements can be encoded successfully by the CPs, (b) that a learned movement policy can readily be reused to produce robust trajectories towards different targets, (c) that a policy fitted for one particular target provides a good predictor of human reaching movements towards neighboring targets, and (d) that the parameter space which encodes a policy is suitable for measuring to which extent two trajectories are qualitatively similar.

Dynamical movement primitives: Learning attractor models for motor behaviors

Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior (e.g., stable locomotion from a system of coupled oscillators under perceptual guidance). Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by means of a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

Learning from demonstration and adaptation of biped locomotion

In this paper, we report on our research for learning biped locomotion from human demonstration. Our ultimate goal is to establish a design principle of a controller in order to achieve natural human-like locomotion. We suggest dynamical movement primitives as a CPG of a biped robot, an approach we have previously proposed for learning and encoding complex human movements. Demonstrated trajectories are learned through the movement primitives by locally weighted regression, and the frequency of the learned trajectories is adjusted automatically by a novel frequency adaptation algorithm based on phase resetting and entrainment of oscillators. Numerical simulations demonstrate the effectiveness of the proposed locomotion controller.

Learning Movement Primitives

This paper discusses a comprehensive framework for modular motor control based on a recently developed theory of dynamic movement primitives (DMP). DMPs are a formulation of movement primitives with autonomous nonlinear differential equations, whose time evolution creates smooth kinematic control policies. Model-based control theory is used to convert the outputs of these policies into motor commands. By means of coupling terms, on-line modifications can be incorporated into the time evolution of the differential equations, thus providing a rather flexible and reactive framework for motor planning and execution. The linear parameterization of DMPs lends itself naturally to supervised learning from demonstration. Moreover, the temporal, scale, and translation invariance of the differential equations with respect to these parameters provides a useful means for movement recognition. A novel reinforcement learning technique based on natural stochastic policy gradients allows a general approach of improving DMPs by trial and error learning with respect to almost arbitrary optimization criteria. We demonstrate the different ingredients of the DMP approach in various examples, involving skill learning from demonstration on the humanoid robot DB, and learning biped walking from demonstration in simulation, including self-improvement of the movement patterns towards energy efficiency through resonance tuning.

Operational Space Control: A Theoretical and Empirical Comparison

Dexterous manipulation with a highly redundant movement system is one of the hallmarks of human motor skills. From numerous behavioral studies, there is strong evidence that humans employ compliant task space control, i.e. they focus control only on task variables while keeping redundant degrees-of-freedom as compliant as possible. This strategy is robust towards unknown disturbances and simultaneously safe for the operator and the environment. The theory of operational space control in robotics aims to achieve similar performance properties. However, despite various compelling theoretical lines of research, advanced operational space control is hardly found in actual robotics implementations, in particular new kinds of robots like humanoids and service robots, which would strongly profit from compliant dexterous manipulation. To analyze the pros and cons of different approaches to operational space control, this paper focuses on a theoretical and empirical evaluation of different methods that have been suggested in the literature, but also some new variants of operational space controllers. We address formulations at the velocity, acceleration, and force levels. First, we formulate all controllers in a common notational framework, including quaternion-based orientation control, and discuss some of their theoretical properties. Second, we present experimental comparisons of these approaches on a seven-degree-of-freedom anthropomorphic robot arm with several benchmark tasks. As an aside, we also introduce a novel parameter estimation algorithm for rigid body dynamics, which ensures physical consistency, as this issue was crucial for our successful robot implementations. Our extensive empirical results demonstrate that one of the simplified acceleration-based approaches can be advantageous in terms of task performance, ease of parameter tuning, and general robustness and compliance in the face of inevitable modeling errors.

Learning rhythmic movements by demonstration using nonlinear oscillators

ATR Human Information Science Laboratories, Kyoto, JapanEmail: ijspeert@usc.edu, jun@his.atr.co.jp, sschaal@usc.eduAbstractThis paper presents a new approach to the generationof rhythmic movement patterns with nonlinear dy-namical systems. Starting from a canonical limit cy-cle oscillator with well-de ned stability properties, wemodify the attractor landscape of the canonical sys-temby meansof statisticallearning methods toembedarbitrary smooth target patterns, however, withoutlosing the stability properties of the canonical sys-tem. In contrast to non-autonomous movement rep-resentations like splines, the learned pattern gener-ators remain autonomous dynamical systems whichrobustly cope with external perturbations that disruptthe time ow of the original pattern, and which canalso be modi ed on-line by additional perceptual vari-ables. A simple extension allows to cope with mul-tiple degrees-of-freedom (DOF) patterns, where allDOFs share the same fundamental frequency but,otherwise, can move in arbitrary phase and ampli-tude o sets to each other. We evaluate our meth-ods in learning from demonstration with an actual30 DOF humanoid robot. Figure-8 and drummingmovements are demonstrated by a human, recordedin joint angle space with an exoskeleton, and em-bedded in multi-dimensional rhythmic pattern gener-ators. The learned patterns can be used by the robotin various workspace locations and from arbitraryinitial conditions. Spatial and temporal invarianceof the pattern generators allow easy amplitude andspeed scaling without losing the qualitative signatureof a movement. This novel way of creating rhyth-mic patterns could tremendously facilitate rhythmicmovement generation, in particular in locomotion ofrobots and neural prosthetics in clinical applications.

Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot

In this paper we describe a learning framework for a central pattern generator (CPG)-based biped locomotion controller using a policy gradient method. Our goals in this study are to achieve CPG-based biped walking with a 3D hardware humanoid and to develop an efficient learning algorithm with CPG by reducing the dimensionality of the state space used for learning. We demonstrate that an appropriate feedback controller can be acquired within a few thousand trials by numerical simulations and the controller obtained in numerical simulation achieves stable walking with a physical robot in the real world. Numerical simulations and hardware experiments evaluate the walking velocity and stability. The results suggest that the learning algorithm is capable of adapting to environmental changes. Furthermore, we present an online learning scheme with an initial policy for a hardware robot to improve the controller within 200 iterations.

Trajectory formation for imitation with nonlinear dynamical systems

Explores an approach to learning by imitation and trajectory formation by representing movements as mixtures of nonlinear differential equations with well-defined attractor dynamics. An observed movement is approximated by finding a best fit of the mixture model to its data by a recursive least squares regression technique. In contrast to non-autonomous movement representations like splines, the resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy which is robust to strong external perturbations and that can be modified by additional perceptual variables. This movement policy remains the same for a given target, regardless of the initial conditions, and can easily be re-used for new targets. We evaluate the trajectory formation system in the context of a humanoid robot simulation that is part of the Virtual Trainer project, which aims at supervising rehabilitation exercises in stroke-patients. A typical rehabilitation exercise was collected with a Sarcos Sensuit, a device to record joint angular movement from human subjects, and approximated and reproduced with our imitation techniques. Our results demonstrate that multijoint human movements can be encoded successfully, and that this system allows robust modifications of the,movement policy through external variables.

Experimental Studies of a Neural Oscillator for Biped Locomotion with QRIO

Recently, there has been a growing interest in biologically inspired biped locomotion control with Central Pattern Generator (CPG). However, few experimental attempts on real hardware 3D humanoid robots have yet been made. Our goal in this paper is to present our achievement of 3D biped locomotion using a neural oscillator applied to a humanoid robot, QRIO. We employ reduced number of neural oscillators as the CPG model, along with a task space Cartesian coordinate system and utilizing entrainment property to establish stable walking gait. We verify robustness against lateral perturbation, through numerical simulation of stepping motion in place along the lateral plane. We then implemented it on the QRIO. It could successfully cope with unknown 3mm bump by autonomously adjusting its stepping period. Sagittal motion produced by a neural oscillator is introduced, and then overlapped with the lateral motion generator in realizing 3D biped locomotion on a QRIO humanoid robot.

A unifying framework for robot control with redundant DOFs

Abstract Recently, Udwadia (Proc. R. Soc. Lond. A 2003:1783–1800, 2003) suggested to derive tracking controllers for mechanical systems with redundant degrees-of-freedom (DOFs) using a generalization of Gauss’ principle of least constraint. This method allows reformulating control problems as a special class of optimal controllers. In this paper, we take this line of reasoning one step further and demonstrate that several well-known and also novel nonlinear robot control laws can be derived from this generic methodology. We show experimental verifications on a Sarcos Master Arm robot for some of the derived controllers. The suggested approach offers a promising unification and simplification of nonlinear control law design for robots obeying rigid body dynamics equations, both with or without external constraints, with over-actuation or underactuation, as well as open-chain and closed-chain kinematics.