Enda Ridge

Tuning the performance of the MMAS heuristic

This paper presents an in-depth Design of Experiments (DOE) methodology for the performance analysis of a stochastic heuristic. The heuristic under investigation is Max-Min Ant System (MMAS). for the Travelling Salesperson Problem (TSP). Specifically, the Response Surface Methodology is used to model and tune MMAS performance with regard to 10 tuning parameters, 2 problem characteristics and 2 performance metrics--solution quality and solution time. The accuracy of these predictions is methodically verified in a separate series of confirmation experiments. The two conflicting responses are simultaneously optimised using desirability functions. Recommendations on optimal parameter settings are made. The optimal parameters are methodically verified. The large number of degrees-of-freedom in the MMAS design are overcome with a Minimum Run Resolution V design. Publicly available algorithm and problem generator implementations are used throughout. The paper should therefore serve as an illustrative case study of the principled engineering of a stochastic heuristic.

Tuning an Algorithm Using Design of Experiments

This chapter is a tutorial on using a design of experiments approach for tuning the parameters that affect algorithm performance. A case study illustrates the application of the method and interpretation of its results.

Analyzing heuristic performance with response surface models: prediction, optimization and robustness

This research uses a Design of Experiments (DOE) approach to build a predictive model of the performance of a combinatorial optimization heuristic over a range of heuristic tuning parameter settings and problem instance characteristics. The heuristic is Ant Colony System (ACS) for the Travelling Salesperson Problem. 10 heurstic tuning parameters and 2 problem characteristics are considered. Response Surface Models (RSM) of the solution quality and solution time predicted ACS performance on both new instances from a publicly available problem generator and new real-world instances from the TSPLIB benchmark library. A numerical optimisation of the RSMs is used to find the tuning parameter settings that yield optimal performance in terms of solution quality and solution time. This paper is the first use of desirability functions, a well-established technique in DOE, to simultaneously optimise these conflicting goals. Finally, overlay plots are used to examine the robustness of the performance of the optimised heuristic across a range of problem instance characteristics. These plots give predictions on the range of problem instances for which a given solutionquality can be expected within a given solution time.

Screening the parameters affecting heuristic performance

This research screens the tuning parameters of a combinatorial optimization heuristic. Specifically, it presents a Design of Experiments (DOE) approach that uses a Fractional Factorial Design to screen the tuning parameters of Ant Colony System (ACS) for the Travelling Sales person problem. Screening is a preliminary step towards building a full Response Surface Model (RSM) [2]. It identifies parametersthat have little influence on performance and can be omittedfrom the RSM design. This reduces the complexity andexpense of the RSM design. 10 algorithm parameters and 2 problem characteristics are considered. Open questionson the effect of 3 parameters on performance are answered.A further parameter, sometimes assumed important, was shown to have no effect on performance. A new problem characteristic that effects performance was identified. A full version of this paper is available [3].

Determining Whether a Problem Characteristic Affects Heuristic Performance

This chapter presents a rigorous Design of Experiments (DOE) approach for determining whether a problem characteristic affects the performance of a heuristic. Specifically, it reports a study on the effect of the cost matrix standard deviation of symmetric Travelling Salesman Problem (TSP) instances on the performance of Ant Colony Optimisation (ACO) heuristics. Results demonstrate that for a given instance size, an increase in the standard deviation of the cost matrix of instances results in an increase in the difficulty of the instances. This implies that for ACO, it is insufficient to report results on problems classified only by problem size, as has been commonly done in most ACO research to date. Some description of the cost matrix distribution is also required when attempting to explain and predict the performance of these heuristics on the TSP. The study should serve as a template for similar investigations with other problems and other heuristics.

A roadmap of nature-inspired systems research and development

Nature-inspired algorithms such as genetic algorithms, particle swarm optimisation and ant colony algorithms have successfully solved computer science problems of search and optimisation. The initial implementations of these techniques focused on static problems solved on single machines. These have been extended by adding parallelisation capabilities in the vein of distributed computing with a centralised master/slave approach. However, the natural systems on which nature-inspired algorithms are based possess many additional characteristics that are of potential benefit within computing environments. In this paper, we discuss the benefits of nature-inspired techniques within modern and emerging computing environments. Software entities within these environments execute and interact in a fashion that is parallel, asynchronous, and decentralised. Given that the natural environment is in itself parallel, asynchronous and decentralised, nature-inspired techniques are an excellent fit for computing environments that exhibit these characteristics. Future research challenges for nature-inspired techniques within emerging computing environments are also discussed.

An analysis of problem difficulty for a class of optimisation heuristics

This paper investigates the effect of the cost matrix standard deviation of Travelling Salesman Problem (TSP) instances on the performance of a class of combinatorial optimisation heuristics. Ant Colony Optimisation (ACO) is the class of heuristic investigated. Results demonstrate that for a given instance size, an increase in the standard deviation of the cost matrix of instances results in an increase in the difficulty of the instances. This implies that for ACO, it is insufficient to report results on problems classified only by problem size, as has been commonly done in most ACO research to date. Some description of the cost matrix distribution is also required when attempting to explain and predict the performance of these algorithms on the TSP.

A Study of Concurrency in the Ant Colony System Algorithm

This paper reports the results of a study of a specific type of concurrency in the ant colony system (ACS) algorithm. Studies of cellular automata (CA) have shown that the update mechanism used can have a dramatic influence on the dynamics of the CA. ACS is usually implemented with a sequential update mechanism. A new method for controlling the concurrency in a nature-inspired algorithm is introduced. Comprehensive tests on a wide range of problem instances are reported. The study found that concurrency levels had no statistically significant effect on ACS performance. This result is interesting because it contradicts what has been observed in another form of nature-inspired algorithm, namely CAs.

The collective: a common information service for self-managed middleware

As the deployment of self-managed reflective middleware platforms increases, the process of collecting and examining information used within the reflective process becomes ever more complex. The quality of such information is vital to ensure the successful outcome of the self-management process. However, the cost associated with the collection of this information plays a major role in influencing the success of a self-managed system.Within typical deployment environments it is not uncommon for multiple self-managed systems to be deployed, each collecting information for use within their respective reflective computations. In many cases, these systems will collect the same information, replicating the effort required to retrieve the information. Such replication could be avoided by sharing information between systems to reduce the overall cost of collection within the deployment environments.Current self-managed systems lack adequate support for information collection and sharing. This work proposes the use of an independent information service to assist in the collection and management of information within self-managed middleware systems.

Special issue on Nature-inspired systems for parallel, asynchronous and decentralised environments

Nature-inspired algorithms such as genetic algorithms, particle swarm optimisation and ant colony algorithms are the state-of-the-art solution technique for some problems. Furthermore, their populationbased stochastic search approach promises desirable algorithm features such as anytime decentralised solution and robustness to problem change. However, the efficient pursuit of more accurate solutions leads researchers to appeal to centralised, highly tuned and sequential implementations that are only loosely related to their successful natural counterparts. This renders them brittle in the face of the dynamism of changing problem specifications and operating conditions and limits their usefulness to industry’s direction of increasing distribution, decentralisation and adaptability. Emerging computing environments such as autonomic computing, ubiquitous computing, Peer-to-Peer systems, the Grid and the Semantic Web demand the interaction of large numbers of decentralised, parallel, asynchronous, and distributed software entities in a standardised fashion. If nature-inspired algorithms are to make an impact on these emerging computing environments, disciplined scientific and engineering investigations must be undertaken into the successful transfer of these algorithms, techniques and infrastructures into such environments. 2. Contributions