Karl Mason

A meta optimisation analysis of particle swarm optimisation velocity update equations for watershed management learning

Abstract Particle swarm optimisation (PSO) is a general purpose optimisation algorithm used to address hard optimisation problems. The algorithm operates as a result of a number of particles converging on what is hoped to be the best solution. How the particles move through the problem space is therefore critical to the success of the algorithm. This study utilises meta optimisation to compare a number of velocity update equations to determine which features of each are of benefit to the algorithm. A number of hybrid velocity update equations are proposed based on other high performing velocity update equations. This research also presents a novel application of PSO to train a neural network function approximator to address the watershed management problem. It is found that the standard PSO with a linearly changing inertia, the proposed hybrid Attractive Repulsive PSO with avoidance of worst locations (AR PSOAWL) and Adaptive Velocity PSO (AV PSO) provide the best performance overall. The results presented in this paper also reveal that commonly used PSO parameters do not provide the best performance. Increasing and negative inertia values were found to perform better.

Multi-objective dynamic economic emission dispatch using particle swarm optimisation variants

Abstract Particle swarm optimisation (PSO) is a bio-inspired swarm based approach to solving optimisation problems. The algorithm functions as a result of particles traversing and evaluating the problem space, eventually converging on the optimum solution. This paper applies a number of PSO variants to the dynamic economic emission dispatch (DEED) problem. The DEED problem is a multi-objective optimisation problem in which the goal is to optimise two conflicting objectives: cost and emissions. The PSO variants tested include: the standard PSO (SPSO), the PSO with avoidance of worst locations (PSO AWL), and also a selection of different topologies including the PSO with a gradually increasing directed neighbourhood (PSO GIDN). The aim of the paper is to test the performance of different variants of the PSO AWL against variants of the SPSO on the DEED problem. The results show that the PSO AWL outperforms the SPSO for every topology implemented. The results are also compared to state of the art genetic algorithm (NSGA-II) and multi-agent eeinforcement learning (MARL). This paper then examines the performance of each PSO algorithm when the power demand is modified to form a triangle wave. The purpose of this experiment was to analyse the performance of different PSO variants on an increasingly constrained problem.

Neural network topology and weight optimization through neuro differential evolution

This research proposes a novel algorithm, Neuro Differential Evolution (NDE), to optimize the topology and weights of neural networks. NDE makes a clear distinction between neural network topology optimization and weight optimization. A genetic algorithm (GA) is implemented to optimize the network topology as this is a discrete problem while differential evolution (DE) is applied to the network weights, which are continuous variables. The results presented in this paper demonstrate that this combined approach can successfully grow neural networks, from just a single neuron, that can produce feasible solutions when other methods fail. NDE outperforms the current state of the art neuroevolution algorithms on a range of increasingly complex reinforcement learning problems.

Evolving multi-objective neural networks using differential evolution for dynamic economic emission dispatch

This research presents a novel framework for evolving Multi-Objective Neural Networks using Differential Evolution (MONNDE). In recent years, the Differential Evolution algorithm has shown to be an effective and robust global optimisation algorithm. The algorithm uses evolutionary operators to optimise complex and continuous problem spaces and has been applied to a range of problems, recently including neural networks. This research continues this trend by utilizing differential evolution to evolve neural networks capable of addressing dynamic problems with multiple objectives. The proposed MONNDE framework is applied to the Dynamic Economic Emission Dispatch (DEED) problem. This problem consists of scheduling a group of power generators in a manner that minimises both cost and emissions produced by the generators. The power generators must also meet a series of constraints relating to their power output, power demand and network loss. The proposed MONNDE is performs very competitively when compared to algorithms such as NSGA-II, PSO, PSOAWL and MARL.

Policy invariance under reward transformations for multi-objective reinforcement learning

Reinforcement Learning (RL) is a powerful and well-studied Machine Learning paradigm, where an agent learns to improve its performance in an environment by maximising a reward signal. In multi-objective Reinforcement Learning (MORL) the reward signal is a vector, where each component represents the performance on a different objective. Reward shaping is a well-established family of techniques that have been successfully used to improve the performance and learning speed of RL agents in single-objective problems. The basic premise of reward shaping is to add an additional shaping reward to the reward naturally received from the environment, to incorporate domain knowledge and guide an agent’s exploration. Potential-Based Reward Shaping (PBRS) is a specific form of reward shaping that offers additional guarantees. In this paper, we extend the theoretical guarantees of PBRS to MORL problems. Specifically, we provide theoretical proof that PBRS does not alter the true Pareto front in both single- and multi-agent MORL. We also contribute the first published empirical studies of the effect of PBRS in single- and multi-agent MORL problems.

Exploring avoidance strategies and neighbourhood topologies in particle swarm optimisation

Particle swarm optimisation (PSO) is a stochastic optimisation algorithm in which particles evaluate solutions in a problem space and converge on the best known solution. This paper presents a PSO variant with avoidance of worst locations (AWL). The particles in PSO AWL remember the worst previous positions as well as the best. This new information changes the motion of the particles and results in spending less time exploring areas which are known to have the worst fitness. A small influence from the worst locations leads to the best performance. The performance of PSO AWL is promising compared to the standard PSO. The PSO AWL also performs significantly better compared to previous implementations of worst location memory. This paper also explores the effect of static vs. dynamic topology on the PSO AWL. It is found that the dynamic topology, gradually increasing directed neighbourhoods (GIDN), greatly improves the performance of PSO AWL.

Avoidance Strategies in Particle Swarm Optimisation

Particle swarm optimisation (PSO) is an optimisation algorithm in which particles traverse a problem space moving towards promising locations which either they or their neighbours have previously visited. This paper presents a new PSO variant with the Avoidance of Worst Locations (AWL). This variation was inspired by animal behaviour. In the wild, an animal will react to negative stimuli as well as positive, e.g. an animal looking for food will also be conscious of danger. PSO AWL enables particles to remember previous poor solutions as well as good. As a result, the particles change the way they move and avoid known bad areas. Balancing the influence of these poor locations is vital. The research in this paper found that a small influence from bad locations on the particles leads to a significant improvement on overall performance when compared to the standard PSO. When compared to previous implementations of worst location memory, PSO AWL demonstrates vast improvements.

Predicting host CPU utilization in the cloud using evolutionary neural networks

Abstract The Infrastructure as a Service (IaaS) platform in cloud computing provides resources as a service from a pool of compute, network, and storage resources. One of the major challenges facing cloud computing is to predict the usage of these resources in real time. By knowing future demands, cloud data centres can dynamically scale resources to decrease energy consumption while maintaining a high quality of service. However cloud resource consumption is ever changing, making it difficult for accurate predictions to be produced. This motivates the research presented in this paper which aims to predict in advance the level of CPU consumption of a host. This research implements evolutionary Neural Networks (NN), a powerful machine learning method, to make these predictions. A number of state of the art swarm and evolutionary optimization algorithms are implemented to train the neural networks to predict host utilization: Particle Swarm Optimization (PSO), Differential Evolution (DE) and Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES). The results of this research demonstrate that CMA-ES converges faster to a better solution on the training data. However when evaluated on the test data, DE performs statistically equal to CMA-ES. The results also demonstrate that the trained networks are still accurate when applied to CPU utilization data from different hosts with no further training needed. When evaluated to predict multiple steps into the future, the accuracy of the network understandably decreases but still performs well on average.

Watershed management using neuroevolution

Neuroevolution refers to evolving neural networks using evolutionary methods. These algorithms have been applied to many problem domains, from game playing to robotics, which motivates this research. The problem of watershed management is addressed here in this research using the most prominent neuroevolution algorithms, i.e. NeuroEvolution of Augmenting Topologies (NEAT), neuro differential evolution and Enforced SubPopulations . The results indicate that neuroevolution is a suitable approach at addressing the watershed management problem, outperforming the other methods of neural network training.