Debora Clever

Learning movement primitives from optimal and dynamically feasible trajectories for humanoid walking

We present a new approach for humanoid gait generation based on movement primitives learned from optimal and dynamically feasible motion trajectories. As testing platform we consider the humanoid robot HRP-2, so far only in simulation. Training data is generated by solving a set of optimal control problems for a minimum-torque optimality criterion and five different step lengths. As the dynamic robot model with all its kinematic and dynamic constraints is considered in the optimal control problem formulation, the resulting motion trajectories are not only optimal but also dynamically feasible. For the learning process we consider the joint angle trajectories of all actuated joints, the ZMP trajectory and the pelvis trajectory, which are sufficient quantities to control the robot. From the training data we learn morphable movement primitives based on Gaussian processes and principal component analysis. We show that five morphable primitives are sufficient to generate steps with 24 different lengths, which are close enough to both dynamical feasibility and optimality to be useful for fast on-line movement generation.

A novel approach for the generation of complex humanoid walking sequences based on a combination of optimal control and learning of movement primitives

We combine optimal control and movement primitive learning in a novel way for the fast generation of humanoid walking movements and demonstrate our approach at the example of the humanoid robot HRP-2 with 36 degrees of freedom. The present framework allows for an efficient computation of long walking sequences consisting of feasible steps of different kind: starting steps from a static posture, cyclic steps or steps with varying step lengths, and stopping motions back to a static posture. Together with appropriate sensors and high level decision strategies this approach provides an excellent basis for an adaptive walking generation on challenging terrain. Our framework comprises a movement primitive model learned from a small number of example steps that are dynamically feasible and minimize an integral mean of squared torques. These training steps are computed by solving three different kinds of optimal control problems that are restricted by the whole-body dynamics of the robot and the gait cycle. The movement primitive model decomposes the joint angles, pelvis orientation and ZMP trajectories in the example data into a small number of primitives, which effectively deals with the redundancy inherent in highly articulated motion. New steps can be composed by weighted combinations of these primitives. The mappings from step parameters to weights are learned with a Gaussian process approach, the contiguity of subsequent steps is promoted by conditioning the beginning of a new step on the end of the current one. Each step can be generated in less than a second, because the expensive optimal control computations, which take several hours per step, are shifted to the precomputational off-line phase. We validate our approach in the virtual robot simulation environment OpenHRP and study the effects of different kernels and different numbers of primitives. We show that the robot can execute long walking sequences with varying step lengths without falling, and hence that feasibility is transferred from optimized to generated motions. Furthermore, we demonstrate that the generated motions are close to torque optimality on the interior parts of the steps but have higher torques than their optimized counterparts on the steps boundaries. Having passed the validation in the robot simulator, we plan to tackle the transfer of this approach to the real platform HRP-2 as a next step.

Inverse optimal control based identification of optimality criteria in whole-body human walking on level ground

Understanding the underlying concepts of human locomotion is important for many fields of research. Based on the assumption that human motions are optimal, we propose an inverse optimal control (IOC) based approach to identify the optimality criteria in human walking. To this end, human walking is modeled as a non-linear optimal control problem with a linear combination of elementary optimality functions as objective and a hybrid dynamics multi-body system as constraints. The developed IOC-framework is set up in a modular way and exploits the natural bi-level structure of the problem. It allows for a great flexibility in the choice of outer optimization techniques and inner dynamic models. In the present work, we use the developed IOC approach to identify weights of seven elementary criteria for seven walking motions captured from six different subjects. The considered optimality criteria address the minimization of joint torques for four sets of joints, head stabilization, the step length, and the step frequency. For all trials the algorithm performs successfully. Even though the identified weights differ observably between subjects, which explains the different walking styles, the correlation matrix gives rise to the hypothesis that there exists a significant correlation of optimality across subjects. The identification of optimality criteria in human walking is a very important issue for all disciplines, where a prediction of human behavior is needed. For example in medical applications to improve therapies or to develop new mobility devices, in sport science to improve training plans or in humanoid robotics to develop new walking strategies.

A new template model for optimization studies of human walking on different terrains

Human movement, as for example human gait, can be considered as an optimal realization of some given task. If the optimization criteria for different types of gait were known, this knowledge could help to improve robot motion generation and control, also for complex walking motions on slopes or stairs. Unfortunately, in general the criteria for which the naturally performed human motion is optimal, are not known. Therefore, in this article we study the relevance of different measurable quantities in human locomotion based on human motion capture data and a new template model that is able to capture the main dynamical characteristics we are interested in. To this end we introduce a three dimensional actuated walking model, with an upper body, two actuated legs and two point-masses as feet. Taking into account single and double support phases and the impact at touch down, the model is suitable to reproduce realistic three dimensional center of mass and swing foot trajectories even in constrained environments. In this article, we focus on walking up and down stairs and compare the observed quantities.

Joint torque analysis of push recovery motions during human walking

Most of their lifetime humans can recover from disturbances during walking motions very well. Our assumption is that to recover from disturbances during walking requires higher internal torques in the joints than motions without disturbances. To measure the internal joint torques in experiments is complicated and expensive. In this work we propose an optimality based simulation environment that allows to determine the internal torques in the joints of a human during disturbed walking motions. The human is represented by a two dimensional (2D) rigid multi-body model consisting of 14 segments controlled by torques in 13 rotational joints resulting in 16 degrees of freedom (DoF). The disturbance is modeled as external force acting on the model. A least-squares optimal control problem that minimizes the distance between the joint angles of the model and joint angles gained from motion capture experiments, while satisfying the dynamics and constraints of the human model, is set up. The analysis of perturbed and unperturbed walking motions shows that the torques in the joints vary according to the strength and duration of the disturbance. The calculation of the internal joint torques is important for the development of new control strategies or set up of humanoid robots and prostheses. It can also be used in the context of sport sciences to improve training or therapies.

COCoMoPL: A Novel Approach for Humanoid Walking Generation Combining Optimal Control, Movement Primitives and Learning and its Transfer to the Real Robot HRP-2

COCoMoPL is a recently developed approach combining optimal control, movement primitives and learning for the generation of humanoid walking motions (Clever et al. Robot. Auton. Syst., vol. 83, pp. 287.298, 2016). It solves optimal control problems based on detailed dynamic models of the robot for a variety of walking parameters and uses the solutions as training data to create movement primitives that are very close to feasibility and optimality. These can be employed to synthesize complex walking sequences for humanoid robots online in a very efficient way. We demonstrate, for the first time, that COCoMoPL works on a real humanoid robot, here HRP-2 with 36 DOF and 30 position controlled actuators. To this end, it was necessary to significantly extend the existing approach by including transition steps into the training data, modify the movement primitives (MP) to admit these transitions, improve the representation of the zero moment point MPs and tighten the transition conditions at the beginning and end of steps. We present a thorough validation of the method in simulation and on the real robot for a challenging sequence of movements. We also compare the characteristics of movements after each step of the methodology.

Optimization based analysis of push recovery during walking motions to support the design of lower-limb exoskeletons

Moving independently is very important for most of the people in daily life. Exoskeletons can help disabled people to gain this capability again. Designing these assistive technologies, it is important to take into account that unperturbed motions barely exist. Therefore it is crucial to develop devices that are able to capture from perturbations. To this end in this work the torques needed in a lower body exoskeleton to perform perturbed human walking motions are determined by an optimization based simulation approach. We compare the behavior of a two dimensional human model with and without an exoskeleton. Two different exoskeleton configurations are considered. The differences between perturbed and unperturbed walking motions as well as between the model with and without exoskeleton are analyzed.

Humanoid gait generation in complex environments based on template models and optimality principles learned from human beings

In this paper, we present an inverse optimal control-based transfer of motions from human experiments to humanoid robots and apply it to walking in constrained environments. To this end, we introduce a 3D template model, which describes motion on the basis of center-of-mass trajectory, foot trajectories, upper-body orientation, and phase duration. Despite its abstract architecture, with prismatic joints combined with damped series elastic actuators instead of knees, the model (including dynamics and constraints) is suitable for describing both human and humanoid locomotion with appropriate parameters. We present and apply an inverse optimal control approach to identify optimality criteria based on human motion capture experiments. The identified optimal strategy is then transferred to a humanoid robot template model for gait generation by solving an optimal control problem, which takes into account the properties of the robot and differences in the environment. The results of this step are the center-of-mass trajectory, the foot trajectories, the torso orientation, and the single and double support phase durations for a sequence of steps, allowing the humanoid robot to walk within a new environment. In a previous paper, we have already presented one computational cycle (from motion capture data to an optimized robot template motion) for the example of walking over irregular stepping stones with the aim of transferring the motion to two very different humanoid robots (iCub@Heidelberg and HRP-2@LAAS). This study represents an extension, containing an entirely new part on the transfer of the optimized template motion to the iCub robot by means of inverse kinematics in a dynamic simulation environment and also on the real robot.

Model-Based Optimization for the Design of Exoskeletons that Help Humans to Sustain Large Pushes While Walking

In order to be useful in daily life, lower limb exoskeletons have to be able to provide support not only for nominal situations, such as level ground walking, but also for the recovery from extreme situations. In this paper, we investigate which torques a lower leg exoskeleton would have to produce in order to allow a person to recover from large perturbations or pushes that may occur while walking. We propose a model-based optimization approach that takes into account dynamic models of the human and the exoskeleton as well as experimental data of humans being pushed. Using optimal control and a least squares objective function we compute the joint torques that exoskeletons of different masses and mass distributions would have to produce in order to make the person follow the recorded recovery trajectories of healthy subjects and which loads would occur in the structure. The results of these computations can serve as guidelines for the design of future lower limb exoskeletons.

Optimization-based analysis of push recovery during walking motions to support the design of rigid and compliant lower limb exoskeletons

AbstractLower limb exoskeletons provide a promising approach to allow disabled people to walk again in the future. Designing such exoskeletons and tuning the required actuators is challenging, since the full dynamics of the combined human-exoskeleton system have to be taken into account. In particular, it is important to not only consider nominal walking motions but also extreme situations such as the recovery from large perturbations. In this paper, we present an approach based on push recovery experiments while walking, multibody system models, and least-squares optimal control to analyze the required torques to be generated by the exoskeleton, assuming that the human provides no torque. We consider seven different trials with varying push locations and push magnitudes applied on the back of the subject. In a first study, we investigate the dependency of these total joint torques on the exoskeleton mass – and compare the torques required for a human without exoskeleton to the ones for the human with two...