Martin Kreutz

Operator adaptation in evolutionary computation and its application to structure optimization of neural networks

Abstract The problem of finding a suitable neural network topology for a given task is often solved by evolutionary computation. In this paper, we show empirically that online adaptation of the search strategy can increase the performance of evolutionary structure optimization. After a brief overview of strategy adaptation in evolutionary computation, we present a general method for adjusting the probabilities of applying variation operators. In example problems the adaptation method leads to faster optimization and better solutions when used in structure optimization of neural networks. We observe that during the evolutionary process the operator probabilities change in an intuitive way depending on the task.

Scale-invariant image recognition based on higher-order autocorrelation features

We propose a framework and a complete implementation of a translation and scale-invariant image recognition system for natural indoor scenes. The system employs higher-order autocorrelation features of scale space data which permit linear classification. An optimal linear classification method is presented, which is able to cope with a large number of classes represented by many, as well as very few samples. In the course of the analysis of our system, we examine which numerical methods for feature transformation and classification show sufficient stability to fulfill these demands. The implementation has been extensively tested. We present the results of our own application and several classification benchmarks.

A Model for the Dynamic Interaction Between Evolution and Learning

The interaction between learning and evolution has elicited much interest particularly among researchers who use evolutionary algorithms for the optimization of neural structures. In this article, we will propose an extension of the existing models by including a developmental phase – a growth process – of the neural network. In this way, we are able to examine the dynamical interaction between genetic information and information learned during development. Several measures are proposed to quantitatively examine the benefits and the effects of such an overlap between learning and evolution. The proposed model, which is based on the recursive encoding method for structure optimization of neural networks, is applied to the problem domain of time series prediction. Furthermore, comments are made on problem domains which associate growing networks (size) during development with problems of increasing complexity.

Using fitness distributions to improve the evolution of learning structures

The absolute benefit, a measure of improvement in the fitness space, is derived from the viewpoint of fitness distribution and fitness trajectory analysis. It is used for online operator adaptation, where the optimization of density estimation models serves as an example. A new information theory based measure is proposed to judge the accuracy of the evolved models. Further, the absolute benefit is applied to offline analysis of new gradient based operators used for coefficient adaptation in genetic programming. An efficient method to calculate the gradient information is presented.

Automated Diagnosis of Skin Cancer Using Digital Image Processing and Mixture-of-Experts

The incidence of malignant melanoma, the most lethal form of skin cancers, has risen rapidly during the last decades. Fortunately, if detected early, even malignant melanoma can be treated successfully. Thus, in recent years, methods for automated detection and diagnosis of skin cancer, particulary malignant melanoma, have elicited much interest. In this paper we present an artificial neural network approach for the classification of skin lesions. Sophisticated image processing, feature extraction, pattern recognition and methods from the field of statistics and artificial neural networks are combined in order to achieve a fast and reliable diagnosis. With this approach, for reasonably balanced training and test sets, we are able to obtain above 90% correct classification of malignant and benign skin lesions coming from the DANAOS data collection.

Solving Extended Hybrid-Flow-Shop Problems Using Active Schedule Generation and Genetic Algorithms

We propose a hybrid approach for solving hybrid-flow-shop problems based on the combination of genetic algorithms and a modified Giffler & Thompson (G&T) algorithm. Several extensions of the hybrid-flow-shop are considered and discussed in the context of a real-world example. The genome in the GA encodes a choice of rules to be used to generate production schedules via the G&T algorithm. All constraints to the scheduling task are observed by the G&T algorithm. Therefore, it provides a well suited representation for the GA and leads to a decoupling of domain specific details and genetic optimization. The proposed method is apphed to the optimization of a batch annealing plant.

Optimisation of Density Estimation Models with Evolutionary Algorithms

We propose a new optimisation method for estimating both the parameters and the structure, i. e. the number of components, of a finite mixture model for density estimation. We employ a hybrid method consisting of an evolutionary algorithm for structure optimisation in conjunction with a gradient-based method for evaluating each candidate model architecture. For structure modification we propose specific, problem dependent evolutionary operators. The introduction of a regularisation term prevents the models from over-fitting the data. Experiments show good generalisation abilities of the optimised structures.

Analysis of Possible Genome-Dependence of Mutation Rates in Genetic Algorithms

The performance of the evolutionary algorithms depends strongly upon the combined effect of the operators (e.g. mutation) and the mappings from genotype to phenotype space and phenotype to fitness space. We demonstrate, with the example of the canonical Genetic Algorithm (cGA) for parameter optimization, that the right choice of the mutation operator should depend on the genom position and we show that generally point-mutation alone might not be sufficient for the particular binary mapping in the cGA. We take up the idea from Evolution Strategy to mutate via addition of normally distributed random numbers and construct the point-mutation operator in a way to resemble this on average. This concrete approach for genetic algorithms is accompanied by more general remarks on the analysis of evolutionary algorithms in the first and the last section of this paper.

Analyse von dermatologischen Farben in der computergestützten Hautkrebsdiagnose

Unter den malignen Neoplasien der Haut stellt insbesondere das maligne Melanom, einerseits aufgrund seiner weltweit steigenden Inzidenz und seiner hohen Mortalitat, andererseits aufgrund der guten Heilungsaussichten bei dessen fruhzeitiger operativer Entfernung, eine Herausforderung an die Medizin hinsichtlich einer optimalen Fruherkennung dar. Um die Diagnosegute computergestutzter Systeme zu verbessern, werden klassische Kohonenkarten zur Klassifikation von Farbverteilungen untersucht.

STRUKTURANALYSE DER HAUT MIT HILFE VON GERICHTETER BELEUCHTUNG ZUR ERKENNUNG VON MALIGNEN HAUTLÄSIONEN

In den letzten Jahren ist eine Zunahme von Hautkrebs insbesondere des malignen Mclanoms zu beobachten (3|. Dies hat zahlreiche Bemühungen motiviert, automatisierte Verfahren zur Hautkrebserkennung zu entwickeln. Eine Eigenschaft von malignen Melanomen ist, daß sie die Topologie der Haut insofern verändern, daß die ursprüngliche Oberflächenstruktur der Haut zerstört wird und sich im Melanom glatte Bereiche bilden. Unterscheiden sich nun die Strukturen innerhalb und außerhalb der Läsion dahingehend, daß eine Faltigkeit der Haut außerhalb der Läsion vorhanden ist, innerhalb der Läsion diese Falten aber durch planare Bereiche abgelöst werden, so ist das ein Indiz für die Malignität der betrachteten Stelle [4j. Bis dato wurde die Oberfläche von Hautläsionen durch aufwendige Verfahren, wie zum Beispiel durch Laserprofilometrie, abgebildet. Diese Verfahren sind sehr exakt, aber auf Grund ihrer Komplexität wenig praktikabel für das Screening von Hautmalen.