Giovani Guizzo

A Hyper-Heuristic for the Multi-Objective Integration and Test Order Problem

Multi-objective evolutionary algorithms (MOEAs) have been efficiently applied to Search-Based Software Engineering (SBSE) problems. However, skilled software engineers waste significant effort designing such algorithms for a particular problem, adapting them, selecting operators and configuring parameters. Hyper-heuristics can help in these tasks by dynamically selecting or creating heuristics. Despite of such advantages, we observe a lack of works regarding this subject in the SBSE field. Considering this fact, this work introduces HITO, a Hyper-heuristic for the Integration and Test Order Problem. It includes a set of well-defined steps and is based on two selection functions (Choice Function and Multi-armed Bandit) to select the best low-level heuristic (combination of mutation and crossover operators) in each mating. To perform the selection, a quality measure is proposed to assess the performance of low-level heuristics throughout the evolutionary process. HITO was implemented using NSGA-II and evaluated to solve the integration and test order problem in seven systems. The introduced hyper-heuristic obtained the best results for all systems, when compared to a traditional algorithm.

Grammatical Evolution for the Multi-Objective Integration and Test Order Problem

Search techniques have been successfully applied for solving different software testing problems. However, choosing, implementing and configuring a search technique can be hard tasks. To reduce efforts spent in such tasks, this paper presents an offline hyper-heuristic named GEMOITO, based on Grammatical Evolution (GE). The goal is to automatically generate a Multi-Objective Evolutionary Algorithm (MOEA) to solve the Integration and Test Order (ITO) problem. The MOEAs are distinguished by components and parameters values, described by a grammar. The proposed hyper-heuristic is compared to conventional MOEAs and to a selection hyper-heuristic used in related work. Results show that GEMOITO can generate MOEAs that are statistically better or equivalent to the compared algorithms.

A Pattern-Driven Mutation Operator for Search-Based Product Line Architecture Design

The application of design patterns through mutation operators in search-based design may improve the quality of the architectures produced in the evolution process. However, we did not find, in the literature, works applying such patterns in the optimization of Product Line Architecture (PLA). Existing works offer manual approaches, which are not search-based, and only apply specific patterns in particular domains. Considering this fact, this paper introduces a meta-model and a mutation operator to allow the design patterns application in the search-based PLA design. The model represents suitable scopes, that is, set of architectural elements that are suitable to receive a pattern. The mutation operator is used with a multi-objective and evolutionary approach to obtain PLA alternatives. Quantitative and qualitative analysis of empirical results show an improvement in the quality of the obtained solutions.

Evaluating a Multi-objective Hyper-Heuristic for the Integration and Test Order Problem

Multi-objective evolutionary algorithms (MOEAs) have been successfully applied for solving different software engineering problems. However, adapting and configuring these algorithms for a specific problem can demand significant effort from software engineers. Therefore, to help in this task, a hyper-heuristic, named HITO (Hyper-heuristic for the Integration and Test Order problem) was proposed to adaptively select search operators during the optimization process. HITO was successfully applied using NSGA-II for solving the integration and test order problem. HITO can use two hyper-heuristic selection methods: Choice Function and Multi-armed Bandit. However, a hypotheses behind this study is that HITO does not depend of NSGA-II and can be used with other MOEAs. To this aim, this paper presents results from evaluation experiments comparing the performance of HITO using two different MOEAs: NSGA-II and SPEA2. The results show that HITO is able to outperform both MOEAs.

Evaluating Different Strategies for Reduction of Mutation Testing Costs

Mutation testing presents high efficacy in terms of revealed faults, but with high computational costs, because test cases must be executed against a great number of mutants. To reduce such costs several strategies exist. In general, they select a reduced number of mutants that maintain a great overall mutation score. Recently, Higher-Order Mutation Testing (HOM) was proposed. HOM introduces more than one fault at a time in the mutants such that they are harder to kill. This kind of testing can also be used as a cost reduction strategy. However, few works in literature evaluate the cost reduction potential of HOM or compare it to other existing strategies. In this work, we evaluate HOM as a mutation testing cost reduction strategy. We conducted an experiment to compare four HOM-based strategies (First to Last, Random Mix, Different Operators, and Each-Choice) and three conventional ones (Random Mutant Selection, Selective Mutation, and Search-Based Mutation by using Genetic Algorithm). The analysis considers the number of selected mutants, number of test cases, mutation score and efficiency measures. Selective Mutation presented the best results overall. Among the HOM-based strategies, Each-Choice was the best.

A multi-objective and evolutionary hyper-heuristic applied to the Integration and Test Order Problem

Display Omitted This paper presents HITO, a hyper-heuristic to solve the Integration and Test Order Problem.HITO adaptively selects search operators used by Multi-objective and Evolutionary Algorithms.HITO includes a novel performance assessment metric based on the Pareto dominance concept.HITO implements two selection methods: Choice Function and Multi-Armed Bandit.The experimental results show that HITO outperformed its random version, the traditional NSGA-II, and the state of the art MOEA/DD. The field of Search-Based Software Engineering (SBSE) has widely utilized Multi-Objective Evolutionary Algorithms (MOEAs) to solve complex software engineering problems. However, the use of such algorithms can be a hard task for the software engineer, mainly due to the significant range of parameter and algorithm choices. To help in this task, the use of Hyper-heuristics is recommended. Hyper-heuristics can select or generate low-level heuristics while optimization algorithms are executed, and thus can be generically applied. Despite their benefits, we find only a few works using hyper-heuristics in the SBSE field. Considering this fact, we describe HITO, a Hyper-heuristic for the Integration and Test Order Problem, to adaptively select search operators while MOEAs are executed using one of the selection methods: Choice Function and Multi-Armed Bandit. The experimental results show that HITO can outperform the traditional MOEAs NSGA-II and MOEA/DD. HITO is also a generic algorithm, since the user does not need to select crossover and mutation operators, nor adjust their parameters.

Metaheuristic Design Pattern: Visitor for Genetic Operators

Metaheuristics, such as Genetic Algorithms (GAs), and hyper-heuristics have been widely studied and applied in the literature. This led to the development of several frameworks to aid the execution and development of such algorithms. Consequently, the reusability, scalability and maintainability became fundamental points to be attacked by developers. Such points can be improved using Design Patterns, but despite their advantages, few works have explored their usage with metaheuristics and hyper-heuristics. In order to contribute to this research topic, we present a solution based on the Visitor pattern used to design genetic operators. A case study is presented with the Hyper-heuristic for the Integration and Test Order problem (HITO). This case study shows that the proposed solution can increase the reusability of the implemented operators, and also enable easy addition of new genetic operators and representations.

A pattern-driven solution for designing multi-objective evolutionary algorithms

Multi-objective evolutionary algorithms (MOEAs) have been widely studied in the literature, which led to the development of several frameworks and techniques to implement them. Consequently, the reusability, scalability and maintainability became fundamental concerns in the development of such algorithms. To this end, the use of design patterns (DPs) can benefit, ease and improve the design of MOEAs. DPs are reusable solutions for common design problems, which can be applied to almost any context. Despite their advantages to decrease coupling, increase flexibility, and allow an easier design extension, DPs have been underexplored for MOEA design. In order to contribute to this research topic, we propose a pattern-driven solution for the design of MOEAs. The MOEA designed with our solution is compared to another MOEA designed without it. The comparison considered: the Integration and Test Order (ITO) problem and the Traveling Salesman problem (TSP). Obtained results show that the use of this DP-driven solution allows the reuse of MOEA components, without decreasing the quality, in terms of hypervolume. This means that the developer can extend the algorithms to include other components using only object-oriented mechanisms in an easier way, while maintaining the expected results.

A Hyper-heuristic for Multi-objective Integration and Test Ordering in Google Guava

Integration testing seeks to find communication problems between different units of a software system. As the order in which units are considered can impact the overall effort required to perform integration testing, deciding an appropriate sequence to integrate and test units is vital. Here we apply a multi-objective hyper-heuristic set within an NSGA-II framework to the Integration and Test Order Problem (ITO) for Google Guava, a set of open-source common libraries for Java. Our results show that an NSGA-II based hyper-heuristic employing a simplified version of Choice Function heuristic selection, outperforms standard NSGA-II for this problem.

Applying design patterns in the search-based optimization of software product line architectures

The design of the product line architecture (PLA) is a difficult activity that can benefit from the application of design patterns and from the use of a search-based optimization approach, which is generally guided by different objectives related, for instance, to cohesion, coupling and PLA extensibility. The use of design patterns for PLAs is a recent research field, not completely explored yet. Some works apply the patterns manually and for a specific domain. Approaches to search-based PLA design do not consider the usage of these patterns. To allow such use, this paper introduces a mutation operator named “Pattern-Driven Mutation Operator” that includes methods to automatically identify suitable scopes and apply the patterns Strategy, Bridge and Mediator with the search-based approach multi-objective optimization approach for PLA. A metamodel is proposed to represent and identify suitable scopes to receive each one of the patterns, avoiding the introduction of architectural anomalies. Empirical results are also presented, showing evidences that the use of the proposed operator produces a greater diversity of solutions and improves the quality of the PLAs obtained in the search-based optimization process, regarding the values of software metrics.