Boye Annfelt Høverstad

Short-Term Load Forecasting With Seasonal Decomposition Using Evolution for Parameter Tuning

This paper studies data-driven short-term load forecasting, where historic data are used to predict the expected load for the next 24 h. Our focus is to simplify and automate the estimation and analysis of various forecasting models. We propose a three-stage approach to load forecasting, consisting of preprocessing, forecasting, and postprocessing, where the forecasting stage uses evolution to automatically set the parameters for each model. In our implementation, the preprocessing stage includes removal of daily and weekly seasonality by a nonparametric method. This seasonal pattern is added in the postprocessing stage. The system allows for easy exploration of several forecasting models, without the need to have in-depth knowledge of how to obtain the best performance for each model. We apply the method to several forecasting algorithms and on three datasets: (1) distribution substation; (2) GEFCom 2012; and (3) a transmission level dataset. We find that the forecasting algorithms considered produce significantly more accurate forecasts when combined with our proposed preprocessing stage compared with applying the same algorithms directly on the raw data. We also find that the parameter values chosen by evolution often provide insights into the interplay between the different datasets and forecast models. Software is available online.

Noise and the evolution of neural network modularity

We study the selective advantage of modularity in artificially evolved networks. Modularity abounds in complex systems in the real world. However, experimental evidence for the selective advantage of network modularity has been elusive unless it has been supported or mandated by the genetic representation. The evolutionary origin of modularity is thus still debated: whether networks are modular because of the process that created them, or the process has evolved to produce modular networks. It is commonly argued that network modularity is beneficial under noisy conditions, but experimental support for this is still very limited. In this article, we evolve nonlinear artificial neural network classifiers for a binary classification task with a modular structure. When noise is added to the edge weights of the networks, modular network topologies evolve, even without representational support.

Effects of data cleansing on load prediction algorithms

The rollout of advanced metering infrastructure that is planned in many countries worldwide will lead to a massive inflow of data from moderately reliable sensory equipment. In principle, this will make intelligent and automated planning and operation possible at an increasingly finer scale in the electric grid. However, errors can creep into the meter data, either from faulty sensors or during transmission from the meters to the database. This work studies the role of data cleansing as a preprocessing step for short-term (24-hour) power load prediction. We focus on cleansing and prediction at several levels of granularity, from the transmission level via distribution substations down to single households. We believe that preprocessing filters such as cleansing should lead to more robustness and/or precision in the subsequent processing step. However, load cleansing frameworks tend to make the popular assumption of normally and independently distributed noise in the time series. We show that this is incorrect at the diurnal level, due to the characteristic pattern of power consumption, with two peak loads during daytime and a nighttime trough. Moreover, we present empirical evidence that a preprocessing step based on this assumption fails to contribute positively to the performance of the subsequent prediction step. To rectify this problem, we suggest to subtract the average power load consumption in a given period before cleansing. We present empirical evidence that this improves the robustness and efficiency of load cleansing as a preprocessing step. Data cleansing and load prediction is performed by a system that searches out parameters using an evolutionary approach.

Simdist: a distribution system for easy parallelization of evolutionary computation

This article introduces Simdist, a software tool for parallel execution of evolutionary algorithms (EAs) in a master-slave configuration on cluster architectures. Clusters have become a cost-effective parallel solution, and the potential computational capabilities are phenomenal. However, the transition from traditional R&D on a personal computer to parallel development and deployment can be a major step. Simdist simplifies this transition considerably, by separating the task of distributing data across the cluster network from the actual EA-related processing performed on the master and slave nodes. Simdist is constructed in the vein of traditional Unix command line tools; it runs in a separate process and communicates with EA child processes via standard input and output. As a result, Simdist is oblivious to the programming language(s) used in the EA, and the EA is similarly oblivious to the internals of Simdist.

Revisiting the personal satellite assistant: neuroevolution with a modified enforced sub-populations algorithm

This paper revisits the evolution of a neural controller for asimulated Personal Satellite Assistant (PSA) using the En-forced Sub-Populations (ESP) neuroevolutionary algorithm, as described by Sit et al. in 2005 [8].ESP has previously been shown to be a very efficient algo-rithm for neuroevolution. As opposed to the original paper,we are not primarily concerned with the solutions discov-ered by the system, but rather with how ESP performs itsevolutionary search; using the unstable PSA control task asa vehicle for fitness evaluation. We propose several changes to the original ESP algorithm. Our experiments suggest that these improve both the inter-nal consistency, and the success rate of the algorithm.We further analyze the ability of ESP to go beyond classicweight evolution. We compare our evolutionary results with those of a simple hill-climb algorithm, and propose that im-proved heuristics for the modifications of network topologyin ESP may be necessary to evolve increasingly complex androbust controllers.

On the effect of network modularity on evolutionary search

Modularity is an omnipresent feature of biological neural networks. It is also a cornerstone of indirect genetic encodings and developmental evolutionary algorithms for neural networks. Modularity may give evolution the ability to reflect regularities in the environment in its solutions, thus making good solutions easier to find. Furthermore, it has been proposed that the density of highly fit solutions is higher in modular networks than in non-modular networks. In this paper we investigate how the degree of modularity in neural networks affects the search landscape for neuroevolution. We use multi-objective evolution to explicitly guide evolution towards modular and non-modular areas of network search space. We find that the fitness landscape is radically different in these different areas, but that network modularity is not accompanied by increased efficiency on a modular classification task. We therefore cannot find support for the popular assumption that modular networks are “better” than non-modular networks.

PEEC: Evolving efficient connections using Pareto optimality

Pareto optimality is a criteria of individual evaluation originally introduced in multi-objective evolutionary algorithms. In the last decade, a growing interest in the integration of Pareto optimality and other evolutionary techniques can be observed. In this work, we integrate EEC, a neuroevolutionary (NE) algorithm, with Pareto optimality. The proposed algorithm is called PEEC. We demonstrate the algorithm on a classic board game, Tic-Tac-Toe, and compare its performance with EEC using three other evaluation models. Our experimental results show that PEEC outperforms all of these and Pareto optimality indeed provides more accurate evaluation to guide NE toward optimal solutions.

Entropy and mutual information can improve fitness evaluation in coevolution of neural networks

Accurate fitness estimates are notoriously difficult to attain in cooperative coevolution, as it is often unclear how to reward the individual parts given an evaluation of the evolved system as a whole. This is particularly true for cooperative approaches to neuroevolution, where neurons or neuronal groups are highly interdependent. In this paper we investigate this problem in the context of evolving neural networks for unstable control problems. We use measures from information theory and neuroscience to reward neurons in a neural network based on their degree of participation in the behavior of the network as a whole. In particular, we actively seek networks with high complexity and little redundancy, and argue that this can lead to efficient evolution of robust controllers. Preliminary results support this claim, and indicate that measures from information theory may provide meaningful information about the role of each neuron in a network.