Cenk Oguz Saglam

Robust Policies via Meshing for Metastable Rough Terrain Walking

In this paper, we present and verify methods for developing robust, high-level policies for metastable (i.e., rarely falling) rough-terrain robot walking. We focus on simultaneously addressing the important, real-world challenges of (1) use of a tractable mesh, to avoid the curse of dimensionality and (2) maintaining near-optimal performance that is robust to uncertainties. Toward our first goal, we present an improved meshing technique, which captures the step-to-step dynamics of robot walking as a discrete-time Markov chain with a small number of points. We keep our methods and analysis generic, and illustrate robustness by quantifying the stability of resulting control policies derived through our methods. To demonstrate our approach, we focus on the challenge of optimally switching among a finite set of low-level controllers for underactuated, rough-terrain walking. Via appropriate meshing techniques, we see that even terrain-blind switching between multiple controllers increases the stability of the robot, while lookahead (terrain information) makes this improvement dramatic. We deal with both noise on the lookahead information and on the state of the robot. These two robustness requirements are essential for our methods to be applicable to real high-DOF robots, which is the primary motivation of the authors.

Switching policies for metastable walking

In this paper, we study an underactuated five-link biped walking on stochastically rough terrain. We propose a simple and powerful Sliding Mode Control scheme. By taking Poincaré sections just before the impact, we accurately represent ten dimensional system dynamics of metastable walking as a Markov process. By switching between two qualitatively different controllers, we show that the number of steps before failure can be increased by more than 10 million times compared to using either one of the controllers only. To achieve this, only the current state and approximate terrain slope for a one step lookahead on geometrically rough terrain is needed. The analysis techniques in this paper are also designed for future application to a range of other simulated or experimental walkers.

Stability and gait transition of the five-link biped on stochastically rough terrain using a discrete set of sliding mode controllers

The five-link biped is a simple, planar model of human-like walking in which scuffing can be avoided. In this paper, we focus on controller design and stability analysis for the important case of non-steady walking, toward such goals as avoiding obstacles on terrain or meeting specific requirements in speed or energetics. To achieve such tasks as new sensor information about upcoming terrain becomes available, control must be adjusted on-the-fly, preferably using a continuous family of controllers. Here, we present an illustrative case using only two, discrete sets of controllers and investigate the effect of switching between them on a stochastically rough terrain. Of note, we find that the tenth-order system dynamics of unsteady walking can be accurately represented as a Markov process, using only a sparse, quasi-2D mesh of discrete states. This transition matrix approach is then used to determine bounded limits on terrain noise for which guarantees of stability (i.e., never falling) may be given for a particular controller and for arbitrary switching between the controllers, as well as to estimate fall rates for cases where these bounds are exceeded. Our results also allow us to quantify the increase in stability gained by a simple policy of switching based on a noisy, single-step lookahead on terrain. This illustrative example, using two controllers that behave differently and allow for arbitrary switching, provides a framework for future work where tasks or requirements for biped walking are clearly defined and can only be achieved by a wider set of, or ideally a continuous range of, controllers.

Quantifying the trade-offs between stability versus energy use for underactuated biped walking.

In this paper, we address the problem of incorporating both energy consumption and stability into a cost function for bipedal walking. To solve the problem, we also propose a basic framework and demonstrate its effectiveness in simulation. This framework allows one to use a scalar coefficient to adjust the trade-off between stability and energy use. The optimal scalar value depends on the robot, terrain, task and priorities. In order to implement the methods in this paper, multiple low-level walking controllers and meshing of a ten-dimensional state space are needed. This latter requirement would normally be impractical for a 10D system; however, we exploit the observation that our low-level controllers cause the step-to-step dynamics to fill only a small, quasi-2D region, thus enabling meshing and, correspondingly, dynamic programming based on the resulting Markov Decision Process (MDP). Both the introduction of the energy/stability trade-off problem and our proposed framework for its solution have potential for significant utility in the future, as robot locomotion is developed to operate in increasingly less structured (stochastic) environments.

Metastable Markov chains

In this paper, we discuss the dynamics of metastable systems. Such systems exhibit interesting long-living behaviors from which they are guaranteed to inevitably escape (e.g., eventually arriving at a distinct failure or success state). At the heart of this work, we emphasize (1) that for our goals, hybrid systems can be approximated as Markov Decision Processes, (2) that although corresponding Markov chains may include a very large number of discrete states, much of their dynamic behavior is well-characterized simply by the second-largest eigenvalue, which is directly analogous to a dominant pole for a discrete-time system and describes both the mean and higher-order modes of the escape statistics, and (3) that for many systems, one can accurately describe initial conditions as being rapidly forgotten, due to a significant separation in slow and fast decay rates. We present both theory and intuitive toy examples that illustrate our approach in analyzing such systems, toward enabling and encouraging other researchers to adopt similar methods.

Lyapunov-based versus Poincaré map analysis of the rimless wheel

Hybrids systems are combinations of continuous and discrete systems. The bouncing ball is an extensively studied hybrid system, for which many solid Lyapunov-based tools are now available. Toward applying these tools to walking robots, where a hybrid dynamical system framework is also a natural fit, the rimless wheel provides a salient dynamic model because it shares commonalities with both bouncing balls and two-legged robots. While much of existing locomotion research is based on Poincaré analysis, in this paper we also study the rimless wheel using Lyapunov-based tools. Our results motivate future use of Poincaré maps for certain hybrid systems and Lyapunov-based tools for more complicated walkers.

Meshing hybrid zero dynamics for rough terrain walking

For an underactuated biped on a constant-slope terrain, the hybrid zero dynamics (HZD) controller framework provides exponentially stable walking motions. In this paper, we quantify the stability of such a control system on rough terrain by estimating the expected number of steps before failure. In addition, we show how to switch between multiple HZD controllers (optionally using terrain look-ahead) to increase the stability dramatically, e.g., 10 thousand steps compared to 10. To do this robustly, we make use of the new meshing method proposed in this paper.

Quantifying and Optimizing Robustness of Bipedal Walking Gaits on Rough Terrain

Legged robots need “good” disturbance rejection to operate reliably in real-world environments, and achieving this goal arguably requires quantifying robustness. In this work, we consider a point-foot biped on variable-height terrain and measure robustness by the expected number of steps before failure. Unlike our previous work, in which we always assumed a fixed set of low-level gait controllers exist and focused on high-level control design, in this work we finally use quantification of robustness to benchmark and optimize a given (low-level) controller itself. Specifically, we study two particular control strategies as case demonstrations. One scheme is the now-familiar hybrid zero dynamics approach and the other is a method using piece-wise reference trajectories with a sliding mode control. This work provides a methodology for optimization of a broad variety of parameterizable gait control strategies and illustrates dramatic increases in robustness due to both gait optimization and choice of control strategy.

Metastable legged locomotion: methods to quantify and optimize reliability

Measuring the stability of highly-dynamic bipedal locomotion is a challenging but essential task for more capable human-like walking. By discretizing the walking dynamics, we treat the system as a Markov chain, which lends itself to an easy quantification of failure rates by the expected number of steps before falling. This meaningful and intuitive metric is then used for optimizing and benchmarking given controllers. While this method is applicable to any controller scheme, we illustrate the results with two case demonstrations. One scheme is the now-familiar hybrid zero dynamics approach and the other is a method using piece-wise reference trajectories with a sliding mode control. We optimize low-level controllers, to minimize failure rates for any one gait, and we adopt a hierarchical control structure to switch among low-level gaits, providing even more dramatic improvements on the system performance.

Passive frontal plane coupling in 3D walking

This paper explores the use of a single passive design to stabilize frontal plane dynamics for 3D biped walking across a range of forward velocities and/or step lengths. Particular goals are to determine if design of sagittal plane control can be done independently from design of frontal plane stabilization mechanisms, and to explore how dynamic coupling between the two planned motions affects energetic efficiency of walking. Passive dynamic walkers have long utilized curved feet for low energy frontal plane stabilization in 3D walking, with the current design practice of matching the linearized resonance of the curvature to match a particular, steady-state walking gait to achieve stable coupled limit cycle in 3D dynamics. However, practical legged walking systems should operate across a range of velocities and step widths. We examine aspects of the nonlinear dynamics that contribute to the energy efficiency and stability of the system through simulations. Specifically, we focus on the tight coupling between the frontal plane dynamics and stepping speed. We find that roll velocity is strongly coupled to the stepping speed and energy consumption. Our decoupled analysis explains some aspects of the 3D motions; however, the actual effects on cost of transport demonstrate interesting phenomena we had not anticipated. Specifically, while a general trend of increasing cost of transport for 3D vs 2D gaits with stride time does hold in our simulations, the 3D gaits sometimes require less energy than their constrained 2D counterparts, which was a surprising and encouraging result. This work provides a promising direction for the development of practical methods to utilize control designed for planar 2D walking models on more sophisticated 3D dynamic models using little or no additional active control.