Antony Waldock

Multiple objective optimisation applied to route planning

This paper presents an evaluation of the benefits of multi-objective optimisation algorithms, compared to single objective optimisation algorithms, when applied to the problem of planning a route over an unstructured environment, where a route has a number of objectives defined using real-world data sources. The paper firstly introduces the problem of planning a route over an unstructured environment (one where no pre-determined set of possible routes exists) and identifies the data sources, Digital Terrain Elevation Data (DTED) and NASA Landsat Hyperspectral data, used to calculate the route objectives (time taken, exposure and fuel consumed). A number of different route planning problems are then used to compare the performance of two single-objective optimisation algorithms and a range of multi-objective optimisation algorithms selected from the literature. The experimental results show that the multi-objective optimisation algorithms result in significantly better routes than the single-objective optimisation algorithms and have the advantage of returning a set of routes that represent the trade-off between objectives. The MOEA/D and SMPSO algorithms are shown, in these experiments, to outperform the other multi-objective optimisation algorithms for this type of problem. Future work will focus on how these algorithms can be integrated into a route planning tool and especially on reducing the time taken to produce routes.

Fuzzy Q-Learning with an adaptive representation

Reinforcement learning (RL) is learning how to map states to actions so as to maximise a numeric reward signal. Fuzzy Q-learning (FQL) extends the RL technique Q-learning to large or continuous problems and has been applied to a wide range of applications from data mining to robot control. Typically, FQL uses a uniform or pre-defined internal representation provided by the human designer. A uniform representation usually provides poor generalisation for control applications, and a pre-defined representation requires the designer to have an in-depth knowledge of the desired control policy. In this paper, the approach taken is to reduce the reliance on a human designer by adapting the internal representation, to improve the generalisation over the control policy, during the learning process. A hierarchical fuzzy rule based system (HFRBS) is used to improve the generalisation of the control policy through iterative refinement of an initial coarse representation on a classical RL problem called the mountain car problem. The process of adapting the representation is shown to significantly reduce the time taken to learn a suitable control policy.

Multi-objective probability collectives

We describe and evaluate a multi-objective optimisation (MOO) algorithm that works within the Probability Collectives (PC) optimisation framework. PC is an alternative approach to optimization where the optimization process focusses on finding an ideal distribution over the solution space rather than an ideal solution. We describe one way in which MOO can be done in the PC framework, via using a Pareto-based ranking strategy as a single objective. We partially evaluate this via testing on a number of problems, and compare the results with state of the art alternatives. We find that this first multi-objective probability collectives (MOPC) approach performs competitively, indicating both clear promise, and clear room for improvement.

A Framework for Cooperative Control Applied to a Distributed Sensor Network

This paper proposes and evaluates a framework for cooperative control of a multi-agent system. The framework is evaluated on a target-tracking application where a distributed sensor network is tasked to autonomously observe targets within the environment. The problem of cooperative control is defined using two distinct levels of cooperation: implicit and explicit. Implicit cooperation is defined as cooperation through only the exchange of environmental data to compile a common picture over which to reason locally. For example, in this paper, decentralized data fusion algorithms are used to build and update a common picture of the target positions and velocities. Explicit cooperation, which is the main focus of this paper, negotiates the agents explicitly on a joint set of actions to perform. In this paper, the problem of explicit cooperation is formulated as a distributed optimization, and a framework to find the joint set of actions is proposed. The framework utilizes two algorithms, the Max-Sum algorithm, to globally solve a factorizable utility function, and Probability Collectives (PC), to solve the individual factors of the utility function. The paper presents experimental results of the two algorithms using a simulated distributed sensor network when the tracking problem is and is not factorizable. The results show that the proposed framework can efficiently and effectively enable cooperation in a distributed sensor network. The Max-Sum algorithm provides a distributed and flexible approach to solve a factorizable utility function, where the PC algorithm was shown to efficiently solve the individual factors when more than four sensors are required to cooperate.

Learning a robot controller using an adaptive hierarchical fuzzy rule-based system

The majority of machine learning techniques applied to learning a robot controller generalise over either a uniform or pre-defined representation that is selected by a human designer. The approach taken in this paper is to reduce the reliance on the human designer by adapting the representation to improve the generalisation during the learning process. An extension of a Hierarchical Fuzzy Rule-Based System (HFRBS) is proposed that identifies and refines inaccurate regions of a fuzzy controller, while interacting with the environment, for both supervised and reinforcement learning problems. The paper shows that a controller using an adaptive HFRBS can learn a suitable control policy using a fewer number of fuzzy rules for both a supervised and reinforcement learning problem and is not sensitive to the layout as with a uniform representation. In supervised learning problems, a small number of extra trials are required to find an effective representation but for reinforcement learning problems, the process of adapting the representation is shown to significantly reduce the time taken to learn a suitable control policy and hence open the door to high-dimensional problems.

Efficient systems analysis by combining SysML and coevolution

The engineering of a complex and large scale system with hundreds of competing requirements is a time consuming and costly process. In recent years, Model Based Systems Engineering has been adopted as a means of moving from a document-centric approach to a model based approach where reusable models can be used to analyse the proposed system. The application of multi-objective optimisation algorithms to generate a set of designs that represent the trade-off between competing system requirements would be highly desirable. In this paper, the authors apply different optimisation strategies, inspired by coevolution, to efficiently generate a set of solutions by identifying and exploiting the structure within the design. The preliminary results show that using the structure inherent in a SysML design has significant benefits in terms of the number of evaluations needed to generate the solutions.

The Next Generation of Unmanned Ground Vehicles

Over the last decade, the development of Unmanned Ground Vehicles (UGVs) has received significant attention through technology competitions, such as the DARPA Grand Challenges, where an unmanned vehicle autonomously navigated across a desert or through semiurban roads. Although this marked a significant step forward in autonomous navigation, the current generation of UGVs are only capable of operating in controlled environments where the dynamics of a scene are well-understood. For example in both of the DARPA Grand Challenges, the participants were given detailed maps and the environment was carefully controlled during the competition. The next generation of UGVs will need to operate in uncontrolled environments where the terrain and infrastructure is uncertain and humans could be present. This paper discusses the challenges and current developments in the areas of sensing, localisation and planning to realise the next generation of UGVs.

Hierarchical fuzzy rule based systems using an information theoretic approach

This paper proposes a novel anytime algorithm for the construction of a Hierarchical Fuzzy Rule Based System using an information theoretic approach to specialise rules that do not effectively model the decision space. The amount of uncertainty tolerated within the decision provides a single tuneable parameter to control the trade off between accuracy and interpretability. The algorithm is empirically compared with existing methods of function approximation and is demonstrated on a mobile robot application in simulation.

Optical flow localisation and appearance mapping (OFLAAM) for long-term navigation

This paper presents a novel method to use optical flow navigation for long term navigation. Unlike standard SLAM approaches for augmented reality, OFLAAM is designed for Micro Air Vehicles (MAV). It uses a optical flow camera pointing downwards, a IMU and a monocular camera pointing frontwards. That configuration avoids the computational expensive mapping and tracking of the 3D features. It only maps these features in a vocabulary list by a localization module to tackle the optical flow drift and the lose of the navigation estimation. That module, based on the well established algorithm DBoW2, will be also used to close the loop and allow long-term navigation in previously visited areas. The combination of high speed optical flow navigation with a low rate localization algorithm allows fully autonomous navigation for MAV, at the same time it reduces the overall computational load. This framework is implemented in ROS (Robot Operating System) and tested attached to a laptop. A representative scenario is used to validate and analyze the performance of the system.

MOPC/D: A new probability collectives algorithm for multiobjective optimisation

Decomposition strategies in Multiobjective optimisation (MOO) are known to be superior to other approaches on a wide variety of problems. Probability Collectives (PC) is a recent distribution-centric optimisation framework that has origins in game-theory and statistical physics. Here, we present a new Probability Collectives MOO algorithm, MOPC/D, based on a decomposition strategy that exploits the search operators which arise naturally from the use of a probabilistic Gaussian mixture model formulation. Evaluation of this approach, using the 2-and 3- objective unconstrained problems from the CEC2009 benchmark suite, found MOPC/D to perform competitively with the state of the art (across these problems it has the best mean rank and rank standard deviation of 14 algorithms in the CEC2009 competition, e.g. above MOEA/D), and significantly outperform the (only) previously published MOO algorithm in the PC framework. We conclude that the performance of MOPC/D shows considerable promise, and suggest a number of lines for further research.