Francisco Chicano

Observations in using parallel and sequential evolutionary algorithms for automatic software testing

In this paper we analyze the application of parallel and sequential evolutionary algorithms (EAs) to the automatic test data generation problem. The problem consists of automatically creating a set of input data to test a program. This is a fundamental step in software development and a time consuming task in existing software companies. Canonical sequential EAs have been used in the past for this task. We explore here the use of parallel EAs. Evidence of greater efficiency, larger diversity maintenance, additional availability of memory/CPU, and multi-solution capabilities of the parallel approach, reinforce the importance of the advances in research with these algorithms. We describe in this work how canonical genetic algorithms (GAs) and evolutionary strategies (ESs) can help in software testing, and what the advantages are (if any) of using decentralized populations in these techniques. In addition, we study the influence of some parameters of the proposed test data generator in the results. For the experiments we use a large benchmark composed of twelve programs that includes fundamental algorithms in computer science.

Finding safety errors with ACO

Model Checking is a well-known and fully automatic technique forchecking software properties, usually given as temporal logicformulae on the program variables. Most model checkers found inthe literature use exact deterministic algorithms to check theproperties. These algorithms usually require huge amounts ofcomputational resources if the checked model is large. We proposehere the use of a new kind of Ant Colony Optimization (ACO) model, ACOhg, to refute safety properties in concurrent systems. ACO algorithms are stochastic techniques belonging to the class of metaheuristic algorithms and inspired by the foraging behaviour of real ants. The traditional ACO algorithms cannot deal with the model checking problem and thus we use ACOhg to tackle it. The results state that ACOhg algorithms find optimal or near optimal error trails in faulty concurrent systems with a reduced amount of resources, outperforming algorithms that are the state-of-the-art in model checking. This fact makes them suitable for checking safety properties in large concurrent systems, in which traditional techniques fail to find errors because of the model size.

Evolutionary algorithms for the multi-objective test data generation problem

Automatic test data generation is a very popular domain in the field of search‐based software engineering. Traditionally, the main goal has been to maximize coverage. However, other objectives can be defined, such as the oracle cost, which is the cost of executing the entire test suite and the cost of checking the system behavior. Indeed, in very large software systems, the cost spent to test the system can be an issue, and then it makes sense by considering two conflicting objectives: maximizing the coverage and minimizing the oracle cost. This is what we did in this paper. We mainly compared two approaches to deal with the multi‐objective test data generation problem: a direct multi‐objective approach and a combination of a mono‐objective algorithm together with multi‐objective test case selection optimization. Concretely, in this work, we used four state‐of‐the‐art multi‐objective algorithms and two mono‐objective evolutionary algorithms followed by a multi‐objective test case selection based on Pareto efficiency. The experimental analysis compares these techniques on two different benchmarks. The first one is composed of 800 Java programs created through a program generator. The second benchmark is composed of 13 real programs extracted from the literature. In the direct multi‐objective approach, the results indicate that the oracle cost can be properly optimized; however, the full branch coverage of the system poses a great challenge. Regarding the mono‐objective algorithms, although they need a second phase of test case selection for reducing the oracle cost, they are very effective in maximizing the branch coverage. Copyright

Using multi-objective metaheuristics to solve the software project scheduling problem

The Software Project Scheduling (SPS) problem relates to the decision of who does what during a software project lifetime. This problem has a capital importance for software companies. In the SPS problem, the total budget and human resources involved in software development must be optimally managed in order to end up with a successful project. Companies are mainly concerned with reducing both the duration and the cost of the projects, and these two goals are in conflict with each other. A multi-objective approach is therefore the natural way of facing the SPS problem. In this paper, a number of multi-objective metaheuristics have been used to address this problem. They have been thoroughly compared over a set of 36 publicly available instances that cover a wide range of different scenarios. The resulting project schedulings of the algorithms have been analyzed in order to show their relevant features. The algorithms used in this paper and the analysis performed may assist project managers in the difficult task of deciding who does what in a software project.

The software project scheduling problem: A scalability analysis of multi-objective metaheuristics

Computer aided techniques for scheduling software projects are a crucial step in the software development process within the highly competitive software industry. The Software Project Scheduling (SPS) problem relates to the decision of who does what during a software project lifetime, thus involving mainly both people-intensive activities and human resources. Two major conflicting goals arise when scheduling a software project: reducing both its cost and duration. A multi-objective approach is therefore the natural way of facing the SPS problem. As companies are getting involved in larger and larger software projects, there is an actual need of algorithms that are able to deal with the tremendous search spaces imposed. In this paper, we analyze the scalability of eight multi-objective algorithms when they are applied to the SPS problem using instances of increasing size. The algorithms are classical algorithms from the literature (NSGA-II, PAES, and SPEA2) and recent proposals (DEPT, MOCell, MOABC, MO-FA, and GDE3). From the experimentation conducted, the results suggest that PAES is the algorithm with the best scalability features.

Multi-objective Optimal Test Suite Computation for Software Product Line Pairwise Testing

Software Product Lines (SPLs) are families of related software products, which usually provide a large number of feature combinations, a fact that poses a unique set of challenges for software testing. Recently, many SPL testing approaches have been proposed, among them pair wise combinatorial techniques that aim at selecting products to test based on the pairs of feature combinations such products provide. These approaches regard SPL testing as an optimization problem where either coverage (maximize) or test suite size (minimize) are considered as the main optimization objective. Instead, we take a multi-objective view where the two objectives are equally important. In this exploratory paper we propose a zero-one mathematical linear program for solving the multi-objective problem and present an algorithm to compute the true Pareto front, hence an optimal solution, from the feature model of a SPL. The evaluation with 118 feature models revealed an interesting trade-off between reducing the number of constraints in the linear program and the runtime which opens up several venues for future research.

Finding deadlocks in large concurrent java programs using genetic algorithms

Model checking is a fully automatic technique for checking concurrent software properties in which the states of a concurrent system are explored in an explicit or implicit way. However, the state explosion problem limits the size of the models that are possible to check. Genetic Algorithms (GAs) are metaheuristic techniques that have obtained good results in problems in which exhaustive techniques fail due to the size of the search space. Unlike exact techniques, metaheuristic techniques cannot be used to verify that a program satisfies a given property, but they can find errors on the software using a lower amount of resources than exact techniques. In this paper, we compare a GA against classical exact techniques and we propose a new operator for this problem, called memory operator, which allows the GA to explore even larger search spaces. We implemented our ideas in the Java PathFinder (JPF) model checker to validate them and present our results. To the best of our knowledge, this is the first implementation of a Genetic Algorithm in this model checker.

Optimal antenna placement using a new multi-objective chc algorithm

Radio network design (RND) is a fundamental problem in cellular networks for telecommunications. In these networks, the terrain must be covered by a set of base stations (or antennae), each of which defines a covered area called cell. The problem may be reduced to figure out the optimal placement of antennae out of a list of candidate sites trying to satisfy two objectives: to maximize the area covered by the radio signal and to reduce the number of used antennae. Consequently, RND is a bi-objective optimization problem. Previous works have solved the problem by using single-objective techniques which combine the values of both objectives. The used techniques have allowed to find optimal solutions according to the defined objective, thus yielding a unique solution instead of the set of Pareto optimal solutions. In this paper, we solve the RND problem using a multi-objective version of the algorithm CHC, which is the metaheuristic having reported the best results when solving the single-objective formulation of RND. This new algorithm, called MOCHC, is compared against a binary-coded NSGA-II algorithm and also against the provided results in the literature. Our experiments indicate that MOCHC outperfoms NSGA-II and, more importantly, it is more efficient finding the optimal solutions than single-objectives techniques.

ACOhg: dealing with huge graphs

Ant Colony Optimization (ACO) has been successfully applied to those combinatorial optimization problems which can be translated into a graph exploration. Artificial ants build solutions step by step adding solution components that are represented by graph nodes. The existing ACO algorithms are suitable when the graph is not very large (thousands of nodes) but is not useful when the graph size can be a challenge for the computer memory and cannot be completely generated or stored in it. In this paper we study a new ACO model that overcomes the difficulties found when working with a huge construction graph. In addition to the description of the model, we analyze in the experimental section one technique used for dealing with this huge graph exploration. The results of the analysis can help to understand the meaning of the new parameters introduced and to decide which parameterization is more suitable for a given problem. For the experiments we use one real problem with capital importance in Software Engineering: refutation of safety properties in concurrent systems. This way, we foster an innovative research line related to the application of ACO to formal methods in Software Engineering.

Evolutionary algorithm for prioritized pairwise test data generation

Combinatorial Interaction Testing (CIT) is a technique used to discover faults caused by parameter interactions in highly configurable systems. These systems tend to be large and exhaustive testing is generally impractical. Indeed, when the resources are limited, prioritization of test cases is a must. Important test cases are assigned a high priority and should be executed earlier. On the one hand, the prioritization of test cases may reveal faults in early stages of the testing phase. But, on the other hand the generation of minimal test suites that fulfill the demanded coverage criteria is an NP-hard problem. Therefore, search based approaches are required to find the (near) optimal test suites. In this work we present a novel evolutionary algorithm to deal with this problem. The experimental analysis compares five techniques on a set of benchmarks. It reveals that the evolutionary approach is clearly the best in our comparison. The presented algorithm can be integrated into a professional tool for CIT.