Jemin Hwangbo

Practice Makes Perfect: An Optimization-Based Approach to Controlling Agile Motions for a Quadruped Robot

This article approaches the problem of controlling quadrupedal running and jumping motions with a parameterized, model-based, state-feedback controller. Inspired by the motor learning principles observed in nature, our method automatically fine tunes the parameters of our controller by repeatedly executing slight variations of the same motion task. This learn-through-practice process is performed in simulation to best exploit computational resources and to prevent the robot from damaging itself. To ensure that the simulation results match the behavior of the hardware platform, we introduce and validate an accurate model of the compliant actuation system. The proposed method is experimentally verified on the torque-controllable quadruped robot StarlETH by executing squat jumps and dynamic gaits, such as a running trot, pronk, and a bounding gait.

Control of a quadrotor with reinforcement learning

In this letter, we present a method to control a quadrotor with a neural network trained using reinforcement learning techniques. With reinforcement learning, a common network can be trained to directly map state to actuator command making any predefined control structure obsolete for training. Moreover, we present a new learning algorithm that differs from the existing ones in certain aspects. Our algorithm is conservative but stable for complicated tasks. We found that it is more applicable to controlling a quadrotor than existing algorithms. We demonstrate the performance of the trained policy both in simulation and with a real quadrotor. Experiments show that our policy network can react to step response relatively accurately. With the same policy, we also demonstrate that we can stabilize the quadrotor in the air even under very harsh initialization (manually throwing it upside-down in the air with an initial velocity of 5 m/s). Computation time of evaluating the policy is only 7

Concurrent Optimization of Mechanical Design and Locomotion Control of a Legged Robot

\mu

ROCK∗ — Efficient black-box optimization for policy learning

s per time step, which is two orders of magnitude less than common trajectory optimization algorithms with an approximated model.

Policy Learning with an Efficient Black-Box Optimization Algorithm

This paper introduces a method to simultaneously optimize design and control parameters for legged robots to improve the performance of locomotion based tasks. The morphology of a quadrupedal robot was optimized for a trotting and bounding gait to achieve a certain speed while tuning the control parameters of a robust locomotion controller at the same time. The results of the optimization show that a change of the structure of the robot can help increase its admissable top speed while using the same actuation units.

Dynamic locomotion and whole-body control for quadrupedal robots

Robotic learning on real hardware requires an efficient algorithm which minimizes the number of trials needed to learn an optimal policy. Prolonged use of hardware causes wear and tear on the system and demands more attention from an operator. To this end, we present a novel black-box optimization algorithm, Reward Optimization with Compact Kernels and fast natural gradient regression (ROCK*). Our algorithm immediately updates knowledge after a single trial and is able to extrapolate in a controlled manner. These features make fast and safe learning on real hardware possible. We have evaluated our algorithm on two simulated reaching tasks of a 50 degree-of-freedom robot arm and on a hopping task of a real articulated legged system. ROCK* outperformed current state-of-the-art algorithms in all tasks by a factor of three or more.

ANYmal - a highly mobile and dynamic quadrupedal robot

Robotic learning on real hardware requires an efficient algorithm which minimizes the number of trials needed to learn an optimal policy. Prolonged use of hardware causes wear and tear on the system and demands more attention from an operator. To this end, we present a novel black-box optimization algorithm, Reward Optimization with Compact Kernels and fast natural gradient regression (ROCK⋆). Our algorithm immediately updates knowledge after a single trial and is able to extrapolate in a controlled manner. These features make fast and safe learning on real hardware possible. The performance of our method is evaluated with standard benchmark functions that are commonly used to test optimization algorithms. We also present three different robotic optimization examples using ROCK⋆. The first robotic example is on a simulated robot arm, the second is on a real articulated legged system, and the third is on a simulated quadruped robot with 12 actuated joints. ROCK⋆ outperforms the current state-of-the-art algorithms in all tasks sometimes even by an order of magnitude.

Fusion of Optical Flow and Inertial Measurements for Robust Egomotion Estimation

This paper presents a framework which allows a quadrupedal robot to execute dynamic gaits including trot, pace and dynamic lateral walk, as well as a smooth transition between them. Our method relies on an online ZMP based motion planner which continuously updates the reference motion trajectory as a function of the contact schedule and the state of the robot. The planner is coupled with a hierarchical whole-body controller which optimizes the whole-body motion and contact forces by solving a cascade of prioritized tasks. We tested our framework on ANYmal, a fully torque controllable quadrupedal robot which is actuated by series-elastic actuators.