Enric Celaya

Control of a six-legged robot walking on abrupt terrain

Legged robots are well suited to walk on difficult terrains at the expense of requiring complex control systems to walk even on flat surfaces. But simply walking on a flat surface is not worth using a legged robot. It should be assumed that walking on abrupt terrain is the typical situation for a legged robot. With this premise in mind, we have developed a robust controller for a six-legged robot that allows it to walk over difficult terrains in an autonomous way, with a limited use of sensory information (no vision is involved). This walk controller can be driven by an upper level which need not be concerned about the details of foot placement or leg movements, taking care only of high level aspects such as global speed and direction.

Reactive free-gait generation to follow arbitrary trajectories with a hexapod robot

We present a reactive controller that is able to displace a legged robot along an arbitrary trajectory with a high degree of accuracy. We designed the dieren t modules of our controller so that they can deal with arbitrary leg congurations. In this way, any leg movement necessary to overcome unexpected terrain irregularities can be correctly compensated by the controller, while still following the trajectory commanded by the user. Since we move the robot as a reaction to leg movements while stepping, the speed of the robot is automatically adjusted to the terrain prole: the more obstacles in the terrain, the more leg movements necessary to overcome them, and the slower the movement of the robot. We prove that, as the terrain becomes simpler, so does the gait generated by our controller, automatically converging to the tripod gait when the terrain becomes at. This is achieved without requiring a map of the terrain and, thus, our controller can be used by robots with minimum computational and sensing capabilities. The results we report using dieren t legged robots and in dieren t environments prove the adequacy of our approach.

Reinforcement learning for agents with many sensors and actuators acting in categorizable environments

In this paper, we confront the problem of applying reinforcement learning to agents that perceive the environment through many sensors and that can perform parallel actions using many actuators as is the case in complex autonomous robots. We argue that reinforcement learning can only be successfully applied to this case if strong assumptions are made on the characteristics of the environment in which the learning is performed, so that the relevant sensor readings and motor commands can be readily identified. The introduction of such assumptions leads to strongly-biased learning systems that can eventually lose the generality of traditional reinforcement-learning algorithms. In this line, we observe that, in realistic situations, the reward received by the robot depends only on a reduced subset of all the executed actions and that only a reduced subset of the sensor inputs (possibly different in each situation and for each action) are relevant to predict the reward. We formalize this property in the so called categorizability assumption and we present an algorithm that takes advantage of the categorizability of the environment, allowing a decrease in the learning time with respect to existing reinforcement-learning algorithms. Results of the application of the algorithm to a couple of simulated realistic-] robotic problems (landmark-based navigation and the six-legged robot gait generation) are reported to validate our approach and to compare it to existing flat and generalization-based reinforcement-learning approaches.

Interval propagation for solving parallel spherical mechanisms

An interval propagation method for spherical kinematic loops is used in a branch and prune algorithm to solve the direct kinematics of parallel spherical mechanisms. The algorithm finds all solutions with a desired resolution. The use of specific properties of the rotation equations involved allows the method to be more efficient than more general algorithms for this problem.

Reinforcement Learning with a Gaussian mixture model

Recent approaches to Reinforcement Learning (RL) with function approximation include Neural Fitted Q Iteration and the use of Gaussian Processes. They belong to the class of fitted value iteration algorithms, which use a set of support points to fit the value-function in a batch iterative process. These techniques make efficient use of a reduced number of samples by reusing them as needed, and are appropriate for applications where the cost of experiencing a new sample is higher than storing and reusing it, but this is at the expense of increasing the computational effort, since these algorithms are not incremental. On the other hand, non-parametric models for function approximation, like Gaussian Processes, are preferred against parametric ones, due to their greater flexibility. A further advantage of using Gaussian Processes for function approximation is that they allow to quantify the uncertainty of the estimation at each point. In this paper, we propose a new approach for RL in continuous domains based on Probability Density Estimations. Our method combines the best features of the previous methods: it is non-parametric and provides an estimation of the variance of the approximated function at any point of the domain. In addition, our method is simple, incremental, and computationally efficient. All these features make this approach more appealing than Gaussian Processes and fitted value iteration algorithms in general.

Visually-guided robot navigation: From artificial to natural landmarks

Landmark-based navigation in unknown unstructured environments is far from solved. The bottleneck nowadays seems to be the fast detection of reliable visual references in the image stream as the robot moves. In our research, we have decoupled the navigation issues from this visual bottleneck, by first using artificial landmarks that could be easily detected and identified. Once we had a navigation system working, we developed a strategy to detect and track salient regions along image streams by just performing on-line pixel sampling. This strategy continuously updates the mean and covariances of the salient regions, as well as creates, deletes and merges regions according to the sample flow. Regions detected as salient can be considered as potential landmarks to be used in the navigation task.

Solving multiloop linkages with limited-range joints

Abstract An algorithm to obtain the solution of underconstrained rotation equations in the form of sets of values that the variables can take is given. Furthermore, an interval propagation algorithm is presented which permits obtaining the subsets of the solution that are compatible with a given interval for one of the variables. The propagation technique provides a way to take into account non-intersection constraints in the solution. In addition, propagation can be used to solve systems of rotation equations. Finally, it is shown how the algorithms described can be used in the solution of certain spatial problems, including 6-bar mechanisms.

Online em with weight-based forgetting

In the online version of the EM algorithm introduced by Sato and Ishii (2000), a time-dependent discount factor is introduced for forgetting the effect of the old estimated values obtained with an earlier, inaccurate estimator. In their approach, forgetting is uniformly applied to the estimators of each mixture component depending exclusively on time, irrespective of the weight attributed to each unit for the observed sample. This causes an excessive forgetting in the less frequently sampled regions. To address this problem, we propose a modification of the algorithm that involves a weight-dependent forgetting, different for each mixture component, in which old observations are forgotten according to the actual weight of the new samples used to replace older values. A comparison of the time-dependent versus the weight-dependent approach shows that the latter improves the accuracy of the approximation and exhibits much greater stability.

Natural Landmark Detection for Visually-Guided Robot Navigation

The main difficulty to attain fully autonomous robot navigation outdoors is the fast detection of reliable visual references, and their subsequent characterization as landmarks for immediate and unambiguous recognition. Aimed at speed, our strategy has been to track salient regions along image streams by just performing on-line pixel sampling. Persistent regions are considered good candidates for landmarks, which are then characterized by a set of subregions with given color and normalized shape. They are stored in a database for posterior recognition during the navigation process. Some experimental results showing landmark-based navigation of the legged robot Lauron III in an outdoor setting are provided.

On Finding the Set of Inverse Kinematic Solutions for Redundant Manipulators

The inverse kinematic solution of a manipulator with ρ redundant d.o.f.’s can be seen as the configuration space of a closed kinematic loop with mobility M = ρ. This set can be described by means of the fasible ranges of values that each variable can take. It is possible, for all planar and spherical loops, obtaining such ranges without explicitly finding the algebraic expression of the solution. The form presented by such ranges permits inferring topological properties of the solution space as a whole.