Bernhard Hengst

Omnidirectional Locomotion for Quadruped Robots

Competing at the RoboCup 2000 Sony legged robot league, the UNSW team won both the challenge competition and all their soccer matches, emerging the outright winners for this league against eleven other international teams. The main advantage that the UNSW team had was speed. A major contributor to the speed was a novel omnidirectional locomotion method developed for the quadruped Sony ERS-110 robot used in the competition. It is believed to be the fastest walk style known for this type of robot. In this paper we describe the parameterised omnidirectional walk in detail. The walk also made a positive contribution to other robot tasks such as ball tracking and localisation while playing soccer. The authors believe that this omnidirectional locomotion could be applied more generally in other legged robots.

Extracting Terrain Features from Range Images for Autonomous Random Stepfield Traversal

One of the challenges of rescue robotics is to create robots that can autonomously traverse rough, unstructured terrain. Although mechanical engineering can produce very capable robots, mechanical engineering alone will not drive them. In this paper, we present a terrain feature extractor that can be taught to find significant features in range images of terrain around a robot from a human expert. This novel approach has the advantage that it potentially allows the human experts knowledge to be captured rapidly. A terrain model is generated from the many points in the range sensor data. Techniques from the field of knowledge acquisition are then used to find patterns in the terrain model. A knowledge acquisition system can then be taught to drive a robot in unstructured terrain based on these features. We evaluate the performance of the initial stages of the feature extractor on a real robot, traversing NIST specification red stepfields.

The UNSW RoboCup 2000 Sony Legged League Team

We describe our technical approach in competing at the RoboCup 2000 Sony legged robot league. The UNSW team won both the challenge competition and all their soccer matches, emerging the outright winners for this league against eleven other international teams. The main advantage that the UNSW team had was speed. The robots not only moved quickly, due to a novel locomotion method, but they also were able to localise and decide on an appropriate action quickly and reliably. This report describes the individual software sub-systems and software architecture employed by the team.

Structural abstraction experiments in reinforcement learning

A challenge in applying reinforcement learning to large problems is how to manage the explosive increase in storage and time complexity. This is especially problematic in multi-agent systems, where the state space grows exponentially in the number of agents. Function approximation based on simple supervised learning is unlikely to scale to complex domains on its own, but structural abstraction that exploits system properties and problem representations shows more promise. In this paper, we investigate several classes of known abstractions: 1) symmetry, 2) decomposition into multiple agents, 3) hierarchical decomposition, and 4) sequential execution. We compare memory requirements, learning time, and solution quality empirically in two problem variations. Our results indicate that the most effective solutions come from combinations of structural abstractions, and encourage development of methods for automatic discovery in novel problem formulations.

Robot Localisation Using Natural Landmarks

This paper introduces an optimised method for extracting natural landmarks to improve localisation during RoboCup soccer matches. The method uses modified 1D SURF features extracted from pixels on the robot’s horizon. Consistent with the original SURF algorithm, the extracted features are robust to lighting changes, scale changes, and small changes in viewing angle or to the scene itself. Furthermore, we show that on a typical laptop 1D SURF runs more than one thousand times faster than SURF, achieving sub-millisecond performance. This makes the method suitable for visual navigation of resource constrained mobile robots. We demonstrate that by using just two stored images, it is possible to largely resolve the RoboCup SPL field end ambiguity.

Learning ankle-tilt and foot-placement control for flat-footed bipedal balancing and walking

We learn a controller for a flat-footed bipedal robot to optimally respond to both (1) external disturbances caused by, for example, stepping on objects or being pushed, and (2) rapid acceleration, such as reversal of demanded walk direction. The reinforcement learning method employed learns an optimal policy by actuating the ankle joints to assert pressure at different points along the support foot, and to determine the next swing foot placement. The controller is learnt in simulation using an inverted pendulum model and the control policy transferred and tested on two small physical humanoid robots.

The UNSW RoboCup 2001 Sony Legged Robot League Team

In 2001, the UNSW United team in the Sony legged robot league successfully defended its title. While the main effort in last years competition was to develop sound low-level skills, this years team focussed primarily on experimenting with new behaviours. An important part of the teams preparation was playing practice matches in which the behaviour of the robots could be studied under actual game-play conditions. In this paper, we describe the evolution of the software from previous years and the new skills displayed by the robots.

Safe state abstraction and reusable continuing subtasks in hierarchical reinforcement learning

Hierarchical reinforcement learning methods have not been able to simultaneously abstract and reuse subtasks with discounted value functions. The contribution of this paper is to introduce two completion functions that jointly decompose the value function hierarchically to solve this problem. The significance of this result is that the benefits of hierarchical reinforcement learning can be extended to discounted value functions and to continuing (infinite horizon) reinforcement learning problems. This paper demonstrates the method with the an algorithm that discovers subtasks automatically. An example is given where the optimum policy requires a subtask never to terminate.

Behavioural cloning for driving robots over rough terrain

Controllers for autonomous mobile robots that operate in rough terrain must consider the shape of the surrounding terrain and its impact on the robots movements. For complex terrain, these interactions are extremely difficult to model in a way that allows traditional controllers to be built. We have used Behavioural Cloning, a type of learning by imitation that produces rules that clone the skills of an expert human operator. We have also developed an autonomous instructor in simulation and used it to generate training data from which we have cloned controllers. The resulting controllers perform at a level comparable to that of a human expert. The controllers behave similarly both in simulation, where they were developed, and on the physical robot without the need for further modification or training.

On the performance of adaptive traffic signal control

In this paper, we present a study in understanding sensing errors impact on traffic signal control performance. Adaptive traffic signal control systems depend on information from traffic sensors to interpret the state of traffic. Signal timings are adjusted at real time according to the state of traffic. Queue length is an important element of the state of traffic, and errors in estimating queue length influences control decision and hence the performance. This paper presents the first attempt to quantify the effects of sensing error on control performance in the field of traffic control. A novel technique to estimate queue length using data from single loop detector is presented, and estimations are compared with parallel observations. The results show that moderate overestimation of queue length may significantly improve control performance. The benefit from overestimation suggests including arriving traffic in system state, and using look-ahead algorithms to calculate signal timings.