Khulood Alyahya

Artificial Bee Colony Training of Neural Networks

The Artificial Bee Colony (ABC) is a recently introduced swarm intelligence algorithm for optimization, that has previously been applied successfully to the training of neural networks. This paper explores more carefully the performance of the ABC algorithm for optimizing the connection weights of feed-forward neural networks for classification tasks, and presents a more rigorous comparison with the traditional Back-Propagation (BP) training algorithm. The empirical results show that using the standard “stopping early” approach with optimized learning parameters leads to improved BP performance over the previous comparative study, and that a simple variation of the ABC approach provides improved ABC performance too. With both improvements applied, we conclude that the ABC approach does perform very well on small problems, but the generalization performances achieved are only significantly better than standard BP on one out of six datasets, and the training times increase rapidly as the size of the problem grows.

Phase Transition and Landscape Properties of the Number Partitioning Problem

This paper empirically studies basic properties of the fitness landscape of random instances of number partitioning problem, with a focus on how these properties change with the phase transition. The properties include number of local and global optima, number of plateaus, basin size and its correlation with fitness. The only two properties that were found to change when the problem crosses the phase transition are the number of global optima and the number of plateaus, the rest of the properties remained oblivious to the phase transition. This paper, also, studies the effect of different distributions of the weights and different neighbourhood operators on the problem landscape.

On the exploitation of search history and accumulative sampling in robust optimisation

Efficient robust optimisation methods exploit the search history when evaluating a new solution by using information from previously visited solutions that fall in the new solutions uncertainty neighbourhood. We propose a full exploitation of the search history by updating the robust fitness approximations across the entire search history rather than a fixed population. Our proposed method shows promising results on a range of test problems compared with other approaches from the literature.

Simple Random Sampling Estimation of the Number of Local Optima

We evaluate the performance of estimating the number of local optima by estimating their proportion in the search space using simple random sampling (SRS). The performance of this method is compared against that of the jackknife method. The methods are used to estimate the number of optima in two landscapes of random instances of some combinatorial optimisation problems. SRS provides a cheap, unbiased and accurate estimate when the proportion is not exceedingly small. We discuss choices of confidence interval in the case of extremely small proportion. In such cases, the method more likely provides an upper bound to the number of optima and can be combined with other methods to obtain a better lower bound. We suggest that SRS should be the first choice for estimating the number of optima when no prior information is available about the landscape under study.

Local Optima and Weight Distribution in the Number Partitioning Problem

This paper investigates the relation between the distribution of the weights and the number of local optima in the Number Partitioning Problem (NPP). The number of local optima in the 1-bit flip landscape was found to be strongly and negatively correlated with the coefficient of variation (CV) of the weights. The average local search cost using the 1-bit flip operator was also found to be strongly and negatively correlated with the CV of the weights. A formula based on the CV of the weights for estimating the average number of local optima in the 1-bit flip landscape is proposed in the paper. The paper also shows that the CV of the weights has a potentially useful application in guiding the choice of heuristic search algorithm.

Voronoi-based archive sampling for robust optimisation

We propose a framework for estimating the quality of solutions in a robust optimisation setting by utilising samples from the search history and using MC sampling to approximate a Voronoi tessellation. This is used to determine a new point in the disturbance neighbourhood of a given solution such that - along with the relevant archived points - they form a well-spread distribution, and is also used to weight the archive points to mitigate any selection bias in the neighbourhood history. Our method performs comparably well with existing frameworks when implemented inside a CMA-ES on 9 test problems collected from the literature in 2 and 10 dimensions.

Landscape Analysis of a Class of NP-Hard Binary Packing Problems

This article presents an exploratory landscape analysis of three NP-hard combinatorial optimisation problems: the number partitioning problem, the binary knapsack problem, and the quadratic binary knapsack problem. In the article, we examine empirically a number of fitness landscape properties of randomly generated instances of these problems. We believe that the studied properties give insight into the structure of the problem landscape and can be representative of the problem difficulty, in particular with respect to local search algorithms. Our work focuses on studying how these properties vary with different values of problem parameters. We also compare these properties across various landscapes that were induced by different penalty functions and different neighbourhood operators. Unlike existing studies of these problems, we study instances generated at random from various distributions. We found a general trend where some of the landscape features in all of the three problems were found to vary between the different distributions. We captured this variation by a single, easy to calculate parameter and we showed that it has a potentially useful application in guiding the choice of the neighbourhood operator of some local search heuristics.

Optimisation and landscape analysis of computational biology models: a case study

The parameter explosion problem is a crucial bottleneck in modelling gene regulatory networks (GRNs), limiting the size of models that can be optimised to experimental data. By discretising state, but not time, Boolean delay equations (BDEs) provide a significant reduction in parameter numbers, whilst still providing dynamical complexity comparable to more biochemically detailed models, such as those based on differential equations. Here, we explore several approaches to optimising BDEs to timeseries data, using a simple circadian clock model as a case study. We compare the effectiveness of two optimisers on our problem: a genetic algorithm (GA) and an elite accumulative sampling (EAS) algorithm that provides robustness to data discretisation. Our results show that both methods are able to distinguish effectively between alternative architectures, yielding excellent fits to data. We also perform a landscape analysis, providing insights into the properties that determine optimiser performance (e.g. number of local optima and basin sizes). Our results provide a promising platform for the analysis of more complex GRNs, and suggest the possibility of leveraging cost landscapes to devise more efficient optimisation schemes.

Landscape Properties of the 0-1 Knapsack Problem

This paper studies two landscapes of different instances of the 0-1 knapsack problem. The instances are generated randomly from varied weight distributions. We show that the variation of the weights can be used to guide the selection of the most suitable local search operator for a given instance.