Seung-Joon Yi

Online learning of a full body push recovery controller for omnidirectional walking

Bipedal humanoid robots are inherently unstable to external perturbations, especially when they are walking on uneven terrain in the presence of unforeseen collisions. In this paper, we present a push recovery controller for position-controlled humanoid robots which is tightly integrated with an omnidirectional walk controller. The high level push recovery controller learns to integrate three biomechanically motivated push recovery strategies with a zero moment point based omnidirectional walk controller. Reinforcement learning is used to map the robot walking state, consisting of foot configuration and onboard sensory information, to the best combination of the three biomechanical responses needed to reject external perturbations. Experimental results show how this online method can stabilize an inexpensive, commercially-available DARwin-OP small humanoid robot.

Team THOR's Entry in the DARPA Robotics Challenge Trials 2013

This paper describes the technical approach, hardware design, and software algorithms that have been used by Team THOR in the DARPA Robotics Challenge DRC Trials 2013 competition. To overcome big hurdles such as a short development time and limited budget, we focused on forming modular components-in both hardware and software-to allow for efficient and cost-effective parallel development. The hardware of THOR-OP Tactical Hazardous Operations Robot-Open Platform consists of standardized, advanced actuators and structural components. These aspects allowed for efficient maintenance, quick reconfiguration, and most importantly, a relatively low build cost. We also pursued modularity in the software, which consisted of a hybrid locomotion engine, a hierarchical arm controller, and a platform-independent remote operator interface. These modules yielded multiple control options with different levels of autonomy to suit various situations. The flexible software architecture allowed rapid development, quick migration to hardware changes, and multiple parallel control options. These systems were validated at the DRC Trials, where THOR-OP performed well against other robots and successfully acquired finalist status.

Practical bipedal walking control on uneven terrain using surface learning and push recovery

Bipedal walking in human environments is made difficult by the unevenness of the terrain and by external disturbances. Most approaches to bipedal walking in such environments either rely upon a precise model of the surface or special hardware designed for uneven terrain. In this paper, we present an alternative approach to stabilize the walking of an inexpensive, commercially-available, position-controlled humanoid robot in difficult environments. We use electrically compliant swing foot dynamics and onboard sensors to estimate the inclination of the local surface, and use a online learning algorithm to learn an adaptive surface model. Perturbations due to external disturbances or model errors are rejected by a hierarchical push recovery controller, which modulates three biomechanically motivated push recovery controllers according to the current estimated state. We use a physically realistic simulation with an articulated robot model and reinforcement learning algorithm to train the push recovery controller, and implement the learned controller on a commercial DARwIn-OP small humanoid robot. Experimental results show that this combined approach enables the robot to walk over unknown, uneven surfaces without falling down.

Learning full body push recovery control for small humanoid robots

Dynamic bipedal walking is susceptible to external disturbances and surface irregularities, requiring robust feedback control to remain stable. In this work, we present a practical hierarchical push recovery strategy that can be readily implemented on a wide range of humanoid robots. Our method consists of low level controllers that perform simple, biomechanically motivated push recovery actions and a high level controller that combines the low level controllers according to proprioceptive and inertial sensory signals and the current robot state. Reinforcement learning is used to optimize the parameters of the controllers in order to maximize the stability of the robot over a broad range of external disturbances. The controllers are learned on a physical simulation and implemented on the Darwin-HP humanoid robot platform, and the resulting experiments demonstrate effective full body push recovery behaviors during dynamic walking.

Team THOR's adaptive autonomy for disaster response humanoids

This paper describes Team THORs approach to sliding autonomy in manipulation and full body control of a disaster response robot for the 2015 DARPA Robotics Challenge (DRC) Finals. Under the duress of unpredictable bandwidth constraints, autonomous behaviors become critical for reducing response time and dealing with dynamic disturbances. However, the nature of disaster response presents situations where fine grained and intricate teleoperation remain the only safe method of operation. We present sets of algorithms that gracefully switch among high level autonomous behaviors and low levels of teleoperated control. Manipulation algorithms interact in a hierarchical fashion within a state machine, transitioning between states autonomously or through human intervention. Similarly, the balancing controller scales from low degree of freedom walking to full body motions. To validate our methods, we show results from attempts at the DRC Finals and in our preparation for it.

Active stabilization of a humanoid robot for impact motions with unknown reaction forces

During heavy work, humans utilize whole body motions in order to generate large forces. In extreme cases, exaggerated weight shifts are used to impart large impact forces. There have been approaches to design stable whole body impact motions based on precise dynamic models of the robot and the target object, but they have practical limitations as the uncertainty in the ensuing reaction forces can lead to instability. In the current work, we describe a motion controller for a humanoid robot that generates impacts at an end effector while keeping the robot body balanced before and after the impact. Instead of relying on the accuracy of the impact dynamics model, we use a simplified model of the robot and biomechanically motivated push recovery controllers to reactively stabilize the robot against unknown perturbations from the impact. We demonstrate our approach in physically realistic simulations, as well as experimentally on a small humanoid robot platform.

A hybrid walk controller for resource-constrained humanoid robots

Zero moment point (ZMP) preview controller is a widely adopted method for bipedal locomotion. However, for robots which are resource constrained or working in dynamic environments, simple reactive walk controllers are still favored as ZMP preview controllers have more control latency and are computationally more demanding. In this work, we present a hybrid walk controller that dynamically switches between a reactive walk controller based on the analytic solution of the linear inverted pendulum model and a ZMP preview controller that uses future foothold positions for more demanding tasks. The boundary conditions of center of mass (COM) state are considered in the optimization process of the ZMP preview controller to ensure a seamless transition between two controllers. We demonstrate the controller in a physically realistic simulations, as well as experimentally on a small humanoid robot platform.

Heel and toe lifting walk controller for traversing uneven terrain

When a humanoid robot traverses uneven terrain, such as stairs, possible footstep positions are constrained and the robot must take large strides. For robots with relatively short leg lengths, making such big strides is kinematically challenging. Possible solutions include lowering the torso height, relying on fast and dynamic stepping, and reducing foot size. However, all of these methods negatively affect performance by either reducing the stability or requiring higher joint torques. In this paper, we present a new locomotion controller that utilizes toe and heel lift to overcome this kinematic constraint for uneven terrain traversal. Given the ground inclination and projected ankle position, desirable toe and heel lift angles are calculated so that the robot can remain in double support while satisfying kinematic and joint range of motion constraints. We demonstrate the controller in physically realistic simulations and with the THOR-RD full-sized humanoid robot in the DARPA Robotics Challenge Finals competition.

Modular low-cost humanoid platform for disaster response

Developing a reliable humanoid robot that operates in uncharted real-world environments is a huge challenge for both hardware and software. Commensurate with the technology hurdles, the amount of time and money required can also be prohibitive barriers. This paper describes Team THORs approach to overcoming such barriers for the 2013 DARPA Robotics Challenge (DRC) Trials. We focused on forming modular components - in both hardware and software - to allow for efficient and cost effective parallel development. The robotic hardware consists of standardized and general purpose actuators and structural components. These allowed us to successfully build the robot from scratch in a very short development period, modify configurations easily and perform quick field repair. Our modular software framework consists of a hybrid locomotion controller, a hierarchical arm controller and a platform-independent operator interface. These modules helped us to keep up with hardware changes easily and to have multiple control options to suit various situations. We validated our approach at the DRC Trials where we fared very well against robots many times more expensive.

Active stabilization of a humanoid robot for real-time imitation of a human operator

Imitating the motion of a human operator is an intuitive and efficient way to make humanoid robots perform complex, human-like behaviors. With the help of recently introduced affordable and real-time depth sensors, the real time imitation of human behavior has become more feasible. However, due to their small footprint and high center of mass, humanoid robots are not inherently stable. The momentum generated by dynamic upper body movements can induce instabilities that are often large enough to make the robot fall down. In this work, we describe a motion controller for a humanoid robot where the upper body is controlled in real time to imitate a human teacher, and the lower body is reactively stabilized based on the current measured state of the robot. Instead of relying on the accuracy of robot dynamics, we use biomechanically motivated push recovery controllers to stabilize the robot against unknown perturbations that include possible impacts. We demonstrate our approach experimentally on a small humanoid robot platform.