Yossi Borenstein

Information landscapes and problem hardness

In [20] we introduced a new concept of a landscape: the information landscape. We showed that for problems of very small size (e.g. a 3-bit problem), it can be used to generally and accurately predict the performance of a GA. Based on this framework, in this paper we develop a method to predict GA hardness on realistic landscapes. We give empirical results which support our approach.

Theory and Principled Methods for the Design of Metaheuristics

Metaheuristics, and evolutionary algorithms in particular, are known to provide efficient, adaptable solutions for many real-world problems, but the often informal way in which they are defined and applied has led to misconceptions, and even successful applications are sometimes the outcome of trial and error. Ideally, theoretical studies should explain when and why metaheuristics work, but the challenge is huge: mathematical analysis requires significant effort even for simple scenarios and real-life problems are usually quite complex. In this book the editors establish a bridge between theory and practice, presenting principled methods that incorporate problem knowledge in evolutionary algorithms and other metaheuristics. The book consists of 11 chapters dealing with the following topics: theoretical results that show what is not possible, an assessment of unsuccessful lines of empirical research; methods for rigorously defining the appropriate scope of problems while acknowledging the compromise between the class of problems to which a search algorithm is applied and its overall expected performance; the top-down principled design of search algorithms, in particular showing that it is possible to design algorithms that are provably good for some rigorously defined classes; and, finally, principled practice, that is reasoned and systematic approaches to setting up experiments, metaheuristic adaptation to specific problems, and setting parameters. With contributions by some of the leading researchers in this domain, this book will be of significant value to scientists, practitioners, and graduate students in the areas of evolutionary computing, metaheuristics, and computational intelligence.

Kolmogorov complexity, Optimization and Hardness

The Kolmogorov complexity (KC) of a string is defined as the length of the shortest program that can print that string and halts. This measure of complexity is often used in optimization to indicate expected function difficulty. While it is often used, there are known counterexamples. This paper investigates the applicability of KC as an estimator of problem difficulty for optimization in the black box scenario. In particular we address the known counterexamples (e.g., pseudorandom functions, the NIAH) and explore the connection of KC to the NFLTs. We conclude that high KC implies hardness however, while easy fitness functions have low KC the reverse is not necessarily true.

Fitness Distributions and GA Hardness

Considerable research effort has been spent in trying to formulate a good definition of GA-Hardness. Given an instance of a problem, the objective is to estimate the performance of a GA. Despite partial successes current definitions are still unsatisfactory. In this paper we make some steps towards a new, more powerful way of assessing problem difficulty based on the properties of a problem’s fitness distribution. We present experimental results that strongly support this idea

On the partitioning of dynamic workforce scheduling problems

This problem is based on the British Telecom workforce scheduling problem, in which technicians (with different skills) are assigned to tasks (which require different skills) which arrive (partially) dynamically during the day. In order to manage their workforce, British Telecom divides the different regions into several areas. At the beginning of each day all the technicians in a region are assigned to one of these areas. During the day, each technician is limited to tasks within the assigned area.This effectively decomposes a large dynamic scheduling problem into smaller problems. On one hand, it makes the problem more manageable. On the other hand, it gives rise to, potentially, a mismatch between technicians and tasks within an area. Furthermore, it prevents technicians from being assigned a job which is just outside their area but happens to be close to where they are currently working.This paper studies the effect of the number of partitions on the expected objective (number of completed tasks) that a rule-based system (responsible for the dynamic assignment and reassignment of tasks to resources following dynamic events) can reach.

Decomposition of fitness functions in random heuristic search

We show that a fitness function, when taken together with an algorithm, can be reformulated as a set of probability distributions. This set can, in some cases, be equivalently viewed as an information vector which gives ordering information about pairs of search points in the domain. Certain performance criteria definable over such an information vector can be learned by linear regression in such a way that extrapolations can sometimes be made: the regression can make performance predictions about functions it has not seen. In addition, the vector can be taken as a model of the fitness function and used to compute features of it like difficultly via vector calculations.

Structure and metaheuristics

Metaheuristics have often been shown to be effective for difficult combinatorial optimization problems. The reason for that, however, remains unclear. A framework for a theory of metaheuristics crucially depends on a formal representative model of such algorithms. This paper unifies/reconciles in a single framework the model of a black box algorithm coming from the no-free-lunch research (e.g. Wolpert et al. [25], Wegener [23]) with the study of fitness landscape. Both are important to the understanding of meta-heuristics, but they have so far been studied separately. The new model is a natural environment to study meta-heuristics.

Information perspective of optimization

In this paper we relate information theory and Kolmogorov Complexity (KC) to optimization in the black box scenario. We define the set of all possible decisions an algorithm might make during a run, we associate a function with a probability distribution over this set and define accordingly its entropy. We show that the expected KC of the set (rather than the function) is a better measure of problem difficulty. We analyze the effect of the entropy on the expected KC. Finally, we show, for a restricted scenario, that any permutation closure of a single function, the finest level of granularity for which a No Free Lunch Theorem can hold [7], can be associated with a particular value of entropy. This implies bounds on the expected performance of an algorithm on members of that closure.

A gaussian random field model of smooth fitness landscapes

The smoothness of a fitness landscape, to date still an elusive notion, is considered to be a fundamental empirical requirement to obtain good performance for many existing meta-heuristics. In this paper, we suggest that a theory of smooth fitness landscapes is central to bridge the gap between theory and practice in EC. As a first step towards this theory, we formalize the notion of smooth fitness landscapes in a general setting using a Gaussian random field model on metric spaces. Then, for the specific case of the Hamming space, we show experimentally that traditional search algorithms with search operators based on this space reach better performance on smoother fitness landscapes. This shows that the formalized notion of smoothness captures the important heuristic property of its informal counterpart.

On the partitioning of dynamic scheduling problems -: assigning technicians to areas

BT workforce scheduling problem considers technicians (with different skills) which are assigned to tasks which arrive (partially) dynamically during the day. In order to manage their workforce, BT divides the different regions into several areas. In the beginning of each day all the technicians in a region are assigned to one of these areas. During the day, tasks can only be allocated to technicians from the same area. In this paper we use a (1+1) EA in order to decide, once the area have been defined, which technicians to assign to which areas.