Jonathan E. Fieldsend

A MOPSO algorithm based exclusively on pareto dominance concepts

In extending the Particle Swarm Optimisation methodology to multi-objective problems it is unclear how global guides for particles should be selected. Previous work has relied on metric information in objective space, although this is at variance with the notion of dominance which is used to assess the quality of solutions. Here we propose methods based exclusively on dominance for selecting guides from a non-dominated archive. The methods are evaluated on standard test problems and we find that probabilistic selection favouring archival particles that dominate few particles provides good convergence towards and coverage of the Pareto front. We demonstrate that the scheme is robust to changes in objective scaling. We propose and evaluate methods for confining particles to the feasible region, and find that allowing particles to explore regions close to the constraint boundaries is important to ensure convergence to the Pareto front.

Using unconstrained elite archives for multiobjective optimization

Multiobjective evolutionary algorithms (MOEAs) have been the subject of numerous studies over the past 20 years. Recent work has highlighted the use of an active archive of elite, nondominated solutions to improve the optimization speed of these algorithms. However, preserving all elite individuals is costly in time (due to the linear comparison with all archived solutions needed before a new solution can be inserted into the archive). Maintaining an elite population of a fixed maximum size (by clustering or other means) alleviates this problem, but can cause retreating (or oscillitory) and shrinking estimated Pareto fronts - which can affect the efficiency of the search process. New data structures are introduced to facilitate the use of an unconstrained elite archive, without the need for a linear comparison to the elite set for every new individual inserted. The general applicability of these data structures is shown by their use in an evolution-strategy-based MOEA and a genetic-algorithm-based MOEA. It is demonstrated that MOEAs using the new data structures run significantly faster than standard, unconstrained archive MOEAs, and result in estimated Pareto fronts significantly ahead of MOEAs using a constrained archive. It is also shown that the use of an unconstrained elite archive permits robust criteria for algorithm termination to be used, and that the use of the data structure can also be used to increase the speed of algorithms using /spl epsi/-dominance methods.

Dominance-Based Multiobjective Simulated Annealing

Simulated annealing is a provably convergent optimizer for single-objective problems. Previously proposed multiobjective extensions have mostly taken the form of a single-objective simulated annealer optimizing a composite function of the objectives. We propose a multiobjective simulated annealer utilizing the relative dominance of a solution as the system energy for optimization, eliminating problems associated with composite objective functions. We also propose a method for choosing perturbation scalings promoting search both towards and across the Pareto front. We illustrate the simulated annealers performance on a suite of standard test problems and provide comparisons with another multiobjective simulated annealer and the NSGA-II genetic algorithm. The new simulated annealer is shown to promote rapid convergence to the true Pareto front with a good coverage of solutions across it comparing favorably with the other algorithms. An application of the simulated annealer to an industrial problem, the optimization of a code-division-multiple access (CDMA) mobile telecommunications networks air interface, is presented and the simulated annealer is shown to generate nondominated solutions with an even and dense coverage that outperforms single objective genetic algorithm optimizers.

Multi-class ROC analysis from a multi-objective optimisation perspective

The receiver operating characteristic (ROC) has become a standard tool for the analysis and comparison of classifiers when the costs of misclassification are unknown. There has been relatively little work, however, examining ROC for more than two classes. Here we discuss and present an extension to the standard two-class ROC for multi-class problems. We define the ROC surface for the Q-class problem in terms of a multi-objective optimisation problem in which the goal is to simultaneously minimise the Q(Q-1) misclassification rates, when the misclassification costs and parameters governing the classifiers behaviour are unknown. We present an evolutionary algorithm to locate the Pareto front-the optimal trade-off surface between misclassifications of different types. The use of the Pareto optimal surface to compare classifiers is discussed and we present a straightforward multi-class analogue of the Gini coefficient. The performance of the evolutionary algorithm is illustrated on a synthetic three class problem, for both k-nearest neighbour and multi-layer perceptron classifiers.

Pareto evolutionary neural networks

For the purposes of forecasting (or classification) tasks neural networks (NNs) are typically trained with respect to Euclidean distance minimization. This is commonly the case irrespective of any other end user preferences. In a number of situations, most notably time series forecasting, users may have other objectives in addition to Euclidean distance minimization. Recent studies in the NN domain have confronted this problem by propagating a linear sum of errors. However this approach implicitly assumes a priori knowledge of the error surface defined by the problem, which, typically, is not the case. This study constructs a novel methodology for implementing multiobjective optimization within the evolutionary neural network (ENN) domain. This methodology enables the parallel evolution of a population of ENN models which exhibit estimated Pareto optimality with respect to multiple error measures. A new method is derived from this framework, the Pareto evolutionary neural network (Pareto-ENN). The Pareto-ENN evolves a population of models that may be heterogeneous in their topologies inputs and degree of connectivity, and maintains a set of the Pareto optimal ENNs that it discovers. New generalization methods to deal with the unique properties of multiobjective error minimization that are not apparent in the uni-objective case are presented and compared on synthetic data, with a novel method based on bootstrapping of the training data shown to significantly improve generalization ability. Finally experimental evidence is presented in this study demonstrating the general application potential of the framework by generating populations of ENNs for forecasting 37 different international stock indexes.

Multi-objective optimisation in the presence of uncertainty

There has been only limited discussion on the effect of uncertainty and noise in multi-objective optimization problems and how to deal with it. We address this problem by assessing the probability of dominance and maintaining an archive of solutions which are, with some known probability, mutually non-dominating. We examine methods for estimating the probability of dominance. These depend crucially on estimating the effective noise variance and we introduce a novel method of learning the variance during optimization. Probabilistic domination contours are presented as a method for conveying the confidence that may be placed in objectives that are optimized in the presence of uncertainty

Dominance measures for multi-objective simulated annealing

Simulated annealing (SA) is a provably convergent optimiser for single-objective (SO) problems. Previously proposed MO extensions have mostly taken the form of an SO SA optimising a composite function of the objectives. We propose an MO SA utilising the relative dominance of a solution as the system energy for optimisation, eliminating problems associated with composite objective functions. We also propose a method for choosing perturbation scalings promoting search both towards and across the Pareto front. We illustrate the SAs performance on standard test problems. The new SA is shown to promote rapid convergence to the true Pareto front with a good coverage of points across it.

Visualizing Mutually Nondominating Solution Sets in Many-Objective Optimization

As many-objective optimization algorithms mature, the problem owner is faced with visualizing and understanding a set of mutually nondominating solutions in a high dimensional space. We review existing methods and present new techniques to address this problem. We address a common problem with the well-known heatmap visualization, since the often arbitrary ordering of rows and columns renders the heatmap unclear, by using spectral seriation to rearrange the solutions and objectives and thus enhance the clarity of the heatmap. A multiobjective evolutionary optimizer is used to further enhance the simultaneous visualization of solutions in objective and parameter space. Two methods for visualizing multiobjective solutions in the plane are introduced. First, we use RadViz and exploit interpretations of barycentric coordinates for convex polygons and simplices to map a mutually nondominating set to the interior of a regular convex polygon in the plane, providing an intuitive representation of the solutions and objectives. Second, we introduce a new measure of the similarity of solutions - the dominance distance - which captures the order relations between solutions. This metric provides an embedding in Euclidean space, which is shown to yield coherent visualizations in two dimensions. The methods are illustrated on standard test problems and data from a benchmark many-objective problem.

Identification of masses in digital mammograms with MLP and RBF nets

We study the identification of masses in digital mammograms using texture analysis. A number of texture measures are calculated for bilateral difference images showing regions of interest. The measurements are made on co-occurrence matrices in four different direction giving a total of seventy features. These features include the ones proposed by Haralick et al. (1973) and Chan et al. (1997). We study a total of 144 breast images from the MIAS database. The dimensionality of the dataset is reduced using principal components analysis (PCA), PCA components are classified using both multilayer perceptron networks using backpropagation (MLP) and radial basis functions based on Gaussian kernels (RBF). The two methods are compared on the same data across a ten fold cross-validation. The results are generated on the average recognition rate over these folds on correctly recognising masses and normal regions. Further analysis is based on the receiver operating characteristic (ROC) plots. The best results show recognition rates of 77% correct recognition and an area under the ROC curve value Az of 0.74.

Cardinality constrained portfolio optimisation

The traditional quadratic programming approach to portfolio optimisation is difficult to implement when there are cardinality constraints. Recent approaches to resolving this have used heuristic algorithms to search for points on the cardinality constrained frontier. However, these can be computationally expensive when the practitioner does not know a priori exactly how many assets they may desire in a portfolio, or what level of return/risk they wish to be exposed to without recourse to analysing the actual trade-off frontier.