Vinicius Veloso de Melo

Investigating Multi-View Differential Evolution for solving constrained engineering design problems

Several constrained and unconstrained optimization problems have been adequately solved over the years thanks to advances in the metaheuristics area. In the last decades, different metaheuristics have been proposed employing new ideas, and hybrid algorithms that improve the original metaheuristics have been developed. One of the most successfully employed metaheuristics is the Differential Evolution. In this paper it is proposed a Multi-View Differential Evolution algorithm (MVDE) in which several mutation strategies are applied to the current population to generate different views at each iteration. The views are then merged according to the winner-takes-all paradigm, resulting in automatic exploration/exploitation balance. MVDE was tested to solve a set of well-known constrained engineering design problems and the obtained results were compared to those from many state-of-the-art metaheuristics. Results show that MVDE was very competitive in the considered problems, largely outperforming several of the compared algorithms.

Kaizen programming

This paper presents Kaizen Programming, an evolutionary tool based on the concepts of Continuous Improvement from Kaizen Japanese methodology. One may see Kaizen Programming as a new paradigm since, as opposed to classical evolutionary algorithms where individuals are complete solutions, in Kaizen Programming each expert proposes an idea to solve part of the problem, thus a solution is composed of all ideas together. Consequently, evolution becomes a collaborative approach instead of an egocentric one. An ideas quality (analog to an individuals fitness) is not how good it fits the data, but a measurement of its contribution to the solution, which improves the knowledge about the problem. Differently from evolutionary algorithms that simply perform trial-and-error search, one can determine, exactly, parts of the solution that should be removed or improved. That property results in the reduction in bloat, number of function evaluations, and computing time. Even more important, the Kaizen Programming tool, proposed to solve symbolic regression problems, builds the solutions as linear regression models - not linear in the variables, but linear in the parameters, thus all properties and characteristics of such statistical tool are valid. Experiments on benchmark functions proposed in the literature show that Kaizen Programming easily outperforms Genetic Programming and other methods, providing high quality solutions for both training and testing sets while requiring a small number of function evaluations.

Investigating Smart Sampling as a population initialization method for Differential Evolution in continuous problems

Recently, researches have shown that the performance of metaheuristics can be affected by population initialization. Opposition-based Differential Evolution (ODE), Quasi-Oppositional Differential Evolution (QODE), and Uniform-Quasi-Opposition Differential Evolution (UQODE) are three state-of-the-art methods that improve the performance of the Differential Evolution algorithm based on population initialization and different search strategies. In a different approach to achieve similar results, this paper presents a technique to discover promising regions in a continuous search-space of an optimization problem. Using machine-learning techniques, the algorithm named Smart Sampling (SS) finds regions with high possibility of containing a global optimum. Next, a metaheuristic can be initialized inside each region to find that optimum. SS and DE were combined (originating the SSDE algorithm) to evaluate our approach, and experiments were conducted in the same set of benchmark functions used by ODE, QODE and UQODE authors. Results have shown that the total number of function evaluations required by DE to reach the global optimum can be significantly reduced and that the success rate improves if SS is employed first. Such results are also in consonance with results from the literature, stating the importance of an adequate starting population. Moreover, SS presents better efficacy to find initial populations of superior quality when compared to the other three algorithms that employ oppositional learning. Finally and most important, the SS performance in finding promising regions is independent of the employed metaheuristic with which SS is combined, making SS suitable to improve the performance of a large variety of optimization techniques.

Evaluating differential evolution with penalty function to solve constrained engineering problems

Over the years, several metaheuristics have been developed to solve hard constrained and unconstrained optimization problems. In general, a metaheuristic is proposed and following researches are made to improve the original algorithm. In this paper, we evaluate a not so new metaheuristic called differential evolution (DE) to solve constrained engineering design problems and compare the results with some recent metaheuristics. Results show that the classical DE with a very simple penalty function to handle constraints is still very competitive in the tested problems.

A modified Covariance Matrix Adaptation Evolution Strategy with adaptive penalty function and restart for constrained optimization

In the last decades, a number of novel meta-heuristics and hybrid algorithms have been proposed to solve a great variety of optimization problems. Among these, constrained optimization problems are considered of particular interest in applications from many different domains. The presence of multiple constraints can make optimization problems particularly hard to solve, thus imposing the use of specific techniques to handle fitness landscapes which generally show complex properties. In this paper, we introduce a modified Covariance Matrix Adaptation Evolution Strategy (CMA-ES) specifically designed for solving constrained optimization problems. The proposed method makes use of the restart mechanism typical of most modern variants of CMA-ES, and handles constraints by means of an adaptive penalty function. This novel CMA-ES scheme presents competitive results on a broad set of benchmark functions and engineering problems, outperforming most state-of-the-art algorithms as for both efficiency and constraint handling.

Defining and simulating open-ended novelty: requirements, guidelines, and challenges

The open-endedness of a system is often defined as a continual production of novelty. Here we pin down this concept more fully by defining several types of novelty that a system may exhibit, classified as variation, innovation, and emergence. We then provide a meta-model for including levels of structure in a system’s model. From there, we define an architecture suitable for building simulations of open-ended novelty-generating systems and discuss how previously proposed systems fit into this framework. We discuss the design principles applicable to those systems and close with some challenges for the community.

Predicting High-Performance Concrete Compressive Strength Using Features Constructed by Kaizen Programming

The compressive strength of high-performance concrete (HPC) can be predicted by a nonlinear function of the proportions of its components. However, HPC is a complex material, and finding that nonlinear function is not trivial. Many distinct techniques such as traditional statistical regression methods and machine learning methods have been used to solve this task, reaching considerably different levels of accuracy. In this paper, we employ the recently proposed Kaizen Programming coupled with classical Ordinary Least Squares (OLS) regression to find high-quality nonlinear combinations of the original features, resulting in new sets of features. Those new features are then tested with various regression techniques to perform prediction. Experimental results show that the features constructed by our technique provide significantly better results than the original ones. Moreover, when compared to similar evolutionary approaches, Kaizen Programming builds only a small fraction of the number of prediction models, but reaches similar or better results.

Investigation of Linear Genetic Programming Techniques for Symbolic Regression

In this paper, we investigate some variants of a basic linear genetic programming (LGP) algorithm in the problem of symbolic regression. We explore the effects of using techniques to control bloat and to privilege a greater percentage of effective code in the population, individually, and examine its possibility of producing better solutions. We also test the effects and performance of an operator that considers two successful individuals as sub functions and join them into a new individual. We conduct experiments and discuss what effects each variant introduces to the evolution and its chance of producing better solutions.

Improving global numerical optimization using a search-space reduction algorithm

We have developed an algorithm for reduction of search-space, called Domain Optimization Algorithm (DOA), applied to global optimization. This approach can efficiently eliminate search-space regions with low probability of containing a global optimum. DOA basically worksusing simple models for search-space regions to identify and eliminate non-promising regions. The proposed approach has shown relevant results for tests using hard benchmark functions.

Improving the prediction of material properties of concrete using Kaizen Programming with Simulated Annealing

Abstract Predicting the properties of materials like concrete has been proven a difficult task given the complex interactions among its components. Over the years, researchers have used Statistics, Machine Learning, and Evolutionary Computation to build models in an attempt to accurately predict such properties. High-quality models are often non-linear, justifying the study of nonlinear regression tools. In this paper, we employ a traditional multiple linear regression method by ordinary least squares to solve the task. However, the model is built upon nonlinear features automatically engineered by Kaizen Programming, a recently proposed hybrid method. Experimental results show that Kaizen Programming can find low-correlated features in an acceptable computational time. Such features build high-quality models with better predictive quality than results reported in the literature.