Ender Özcan

Hyper-heuristics: a survey of the state of the art

Hyper-heuristics comprise a set of approaches that are motivated (at least in part) by the goal of automating the design of heuristic methods to solve hard computational search problems. An underlying strategic research challenge is to develop more generally applicable search methodologies. The term hyper-heuristic is relatively new; it was first used in 2000 to describe heuristics to choose heuristics in the context of combinatorial optimisation. However, the idea of automating the design of heuristics is not new; it can be traced back to the 1960s. The definition of hyper-heuristics has been recently extended to refer to a search method or learning mechanism for selecting or generating heuristics to solve computational search problems. Two main hyper-heuristic categories can be considered: heuristic selection and heuristic generation. The distinguishing feature of hyper-heuristics is that they operate on a search space of heuristics (or heuristic components) rather than directly on the search space of solutions to the underlying problem that is being addressed. This paper presents a critical discussion of the scientific literature on hyper-heuristics including their origin and intellectual roots, a detailed account of the main types of approaches, and an overview of some related areas. Current research trends and directions for future research are also discussed.

Particle swarm optimization: surfing the waves

A new optimization method has been proposed by J. Kennedy and R.C. Eberhart (1997; 1995), called Particle Swarm Optimization (PSO). This approach combines social psychology principles and evolutionary computation. It has been applied successfully to nonlinear function optimization and neural network training. Preliminary formal analyses showed that a particle in a simple one-dimensional PSO system follows a path defined by a sinusoidal wave, randomly deciding on both its amplitude and frequency (Y. Shi and R. Eberhart, 1998). The paper takes the next step, generalizing to obtain closed form equations for trajectories of particles in a multi-dimensional search space.

A Classification of Hyper-heuristic Approaches

The current state of the art in hyper-heuristic research comprises a set of approaches that share the common goal of automating the design and adaptation of heuristic methods to solve hard computational search problems. The main goal is to produce more generally applicable search methodologies. In this chapter we present an overview of previous categorisations of hyper-heuristics and provide a unified classification and definition, which capture the work that is being undertaken in this field. We distinguish between two main hyper-heuristic categories: heuristic selection and heuristic generation. Some representative examples of each category are discussed in detail. Our goals are to clarify the mainfeatures of existing techniques and to suggest new directions for hyper-heuristic research.

Exploring Hyper-heuristic Methodologies with Genetic Programming

Hyper-heuristics represent a novel search methodology that is motivated by the goal of automating the process of selecting or combining simpler heuristics in order to solve hard computational search problems. An extension of the original hyper-heuristic idea is to generate new heuristics which are not currently known. These approaches operate on a search space of heuristics rather than directly on a search space of solutions to the underlying problem which is the case with most meta-heuristics implementations. In the majority of hyper-heuristic studies so far, a framework is provided with a set of human designed heuristics, taken from the literature, and with good measures of performance in practice. A less well studied approach aims to generate new heuristics from a set of potential heuristic components. The purpose of this chapter is to discuss this class of hyper-heuristics, in which Genetic Programming is the most widely used methodology. A detailed discussion is presented including the steps needed to apply this technique, some representative case studies, a literature review of related work, and a discussion of relevant issues. Our aim is to convey the exciting potential of this innovative approach for automating the heuristic design process.

A Reinforcement Learning-Great-Deluge Hyper-Heuristic for Examination Timetabling

Hyper-heuristics can be identified as methodologies that search the space generated by a finite set of low level heuristics for solving search problems. An iterative hyper-heuristic framework can be thought of as requiring a single candidate solution and multiple perturbation low level heuristics. An initially generated complete solution goes through two successive processes heuristic selection and move acceptance until a set of termination criteria is satisfied. A motivating goal of hyper-heuristic research is to create automated techniques that are applicable to a wide range of problems with different characteristics. Some previous studies show that different combinations of heuristic selection and move acceptance as hyper-heuristic components might yield different performances. This study investigates whether learning heuristic selection can improve the performance of a great deluge based hyper-heuristic using an examination timetabling problem as a case study.

An experimental study on hyper-heuristics and exam timetabling

Hyper-heuristics are proposed as a higher level of abstraction as compared to the metaheuristics. Hyper-heuristic methods deploy a set of simple heuristics and use only non-problem-specific data, such as fitness change or heuristic execution time. A typical iteration of a hyper-heuristic algorithm consists of two phases: the heuristic selection method and move acceptance. In this paper, heuristic selection mechanisms and move acceptance criteria in hyper-heuristics are analyzed in depth. Seven heuristic selection methods and five acceptance criteria are implemented. The performance of each selection and acceptance mechanism pair is evaluated on 14 well-known benchmark functions and 21 exam timetabling problem instances.

Memetic algorithms for timetabling

Course timetabling problems are real world constraint optimization problems that are often coped with educational institutions, such as universities or high schools. In this paper, we present a variety of new operators that can be also applied in evolutionary algorithms for other timetabling problems, such as, exam timetabling. Operators include violation directed mutations, crossovers, and a successful violation directed hierarchical hill climbing method. Tests are performed on a small portion of a real data and results are promising.

Examination timetabling using late acceptance hyper-heuristics

A hyperheuristic is a high level problem solving methodology that performs a search over the space generated by a set of low level heuristics. One of the hyperheuristic frameworks is based on a single point search containing two main stages: heuristic selection and move acceptance. Most of the existing move acceptance methods compare a new solution, generated after applying a heuristic, against a current solution in order to decide whether to reject it or replace the current one. Late Acceptance Strategy is presented as a promising local search methodology based on a novel move acceptance mechanism. This method performs a comparison between the new candidate solution and a previous solution that is generated L steps earlier. In this study, the performance of a set of hyper-heuristics utilising different heuristic selection methods combined with the Late Acceptance Strategy are investigated over an examination timetabling problem. The results illustrate the potential of this approach as a hyperheuristic component. The hyper-heuristic formed by combining a random heuristic selection with Late Acceptance Strategy improves on the best results obtained in a previous study.

Memetic algorithms for nurse rostering

Nurse rostering problems represent a subclass of scheduling problems that are hard to solve. The goal is finding high quality shift and resource assignments, satisfying the needs and requirements of employees as well as the employers in healthcare institutions. In this paper, a real case of a nurse rostering problem is introduced. Memetic Algorithms utilizing different type of promising genetic operators and a self adaptive violation directed hierarchical hill climbing method are presented based on a previously proposed framework.

Hill climbers and mutational heuristics in hyperheuristics

Hyperheuristics are single candidate solution based and simple to maintain mechanisms used in optimization. At each iteration, as a higher level of abstraction, a hyperheuristic chooses and applies one of the heuristics to a candidate solution. In this study, the performance contribution of hill climbing operators along with the mutational heuristics are analyzed in depth in four different hyperheuristic frameworks. Four different hill climbing operators and three mutational operators are used during the experiments. Various subsets of the heuristics are evaluated on fourteen well-known benchmark functions.