Qing Gu

Variable Strength Interaction Testing with an Ant Colony System Approach

Interaction testing (also called combinatorial testing) is an cost-effective test generation technique in software testing. Most research work focuses on finding effective approaches to build optimal t-way interaction test suites. However, the strength of different factor sets may not be consistent due to the practical test requirements. To solve this problem, a variable strength combinatorial object and several approaches based on it have been proposed. These approaches include simulated annealing (SA) and greedy algorithms. SA starts with a large randomly generated test suite and then uses a binary search process to find the optimal solution. Although this approach often generates the minimal test suites, it is time consuming. Greedy algorithms avoid this shortcoming but the size of generated test suites is usually not as small as SA. In this paper, we propose a novel approach to generate variable strength interaction test suites (VSITs). In our approach, we adopt a one-test-at-a-time strategy to build final test suites. To generate a single test, we adopt ant colony system (ACS) strategy, an effective variant of ant colony optimization (ACO). In order to successfully adopt ACS, we formulize the solution space, the cost function and several heuristic settings in this framework. We also apply our approach to some typical inputs. Experimental results show the effectiveness of our approach especially compared to greedy algorithms and several existing tools.

Applying Particle Swarm Optimization to Pairwise Testing

Combinatorial testing (also called interaction testing) is an effective specification-based test input generation technique. By now most of research work in combinatorial testing aims to propose novel approaches trying to generate test suites with minimum size that still cover all the pairwise, triple, or n-way combinations of factors. Since the difficulty of solving this problem is demonstrated to be NP-hard, existing approaches have been designed to generate optimal or near optimal combinatorial test suites in polynomial time. In this paper, we try to apply particle swarm optimization (PSO), a kind of meta-heuristic search technique, to pairwise testing (i.e. a special case of combinatorial testing aiming to cover all the pairwise combinations). To systematically build pairwise test suites, we propose two different PSO based algorithms. One algorithm is based on one-test-at-a-time strategy and the other is based on IPO-like strategy. In these two different algorithms, we use PSO to complete the construction of a single test. To successfully apply PSO to cover more uncovered pairwise combinations in this construction process, we provide a detailed description on how to formulate the search space, define the fitness function and set some heuristic settings. To verify the effectiveness of our approach, we implement these algorithms and choose some typical inputs. In our empirical study, we analyze the impact factors of our approach and compare our approach to other well-known approaches. Final empirical results show the effectiveness and efficiency of our approach.

Building Prioritized Pairwise Interaction Test Suites with Ant Colony Optimization

Interaction testing offers a stable cost-benefit ratio in identifying faults. But in many testing scenarios, the entire test suite cannot be fully executed due to limited time or cost. In these situations, it is essential to take the importance of interactions into account and prioritize these tests. To tackle this issue, the biased covering array is proposed and the Weighted Density Algorithm (WDA) is developed. To find a better solution, in this paper we adopt ant colony optimization (ACO) to build this prioritized pairwise interaction test suite (PITS). In our research, we propose four concrete test generation algorithms based on Ant System, Ant System with Elitist, Ant Colony System and Max-Min Ant System respectively. We also implement these algorithms and apply them to two typical inputs and report experimental results. The results show the effectiveness of these algorithms.

A study of relative redundancy in test-suite reduction while retaining or improving fault-localization effectiveness

Test-suite reduction technique aims to find a subset of the test suite while still satisfying the original test requirements; therefore, it can save the cost of software testing. Because many test cases have been removed, the testing information is also lost. Fault localization is a technique using testing information to locate the fault and widely used by programmers to debug programs, so it suffers from the side effects of test-suite reduction. How to reduce the test-suite size in software testing with the premise of retaining or improving fault-localization effectiveness has become the hot spot in the area of software debugging recently. In this paper, we propose a new approach that selectively keeps the limited redundant test cases in the reduced set; it makes the new reduced set relatively redundant compared to the original one, and we expect that it retains or even improves fault-localization effectiveness. We also describe a framework that implements our approach and conduct a set of empirical studies for evaluation. The results show that our approach can retain or even improve fault-localization effectiveness as expected.

A test suite reduction approach based on pairwise interaction of requirements

Test suite reduction is one of the effective techniques to reduce the cost of regression testing. In particular, it tries to identify and remove redundant test cases according to a specific test coverage criterion. However, the excessive reduction in test cases may also significantly weaken the fault detection ability of the original test suite. In this paper, we conjecture that covering interaction of test requirements can improve the fault detection ability and propose a new test suite reduction approach. As a preliminary study, we firstly propose a pairwise interaction based coverage criterion (PWIC). Then we propose a pairwise interaction of requirements based test suite reduction approach (PWIR). To assess the feasibility and usefulness of our proposed approach, we implement PWIR approach and conduct an empirical study on seven real C programs. After analyzing the results of the empirical studies, we conclude that our approach can improve the fault detection ability without severely increasing the reduced test suite size.

SNN - A Neural Network Based Combination of Software Reliability Growth Models

Applying SRGMs (Software Reliability Growth Models) to real projects is a major concern in software reliability. Sometimes, it is hard to decide the best model for a specific project. Researchers have made a first step on solving this problem by combination, but the effect was limited in accuracy and adaptability. Aiming to improve the usability of the SRGMs, we propose a neural network based combination method to build accurate and adaptive SNN (Selective Neural Network) model. It avoids relying on a single model, thus reduces the risk to produce inaccurate predictions and improves the average performance in accuracy. Neural network and multi-criteria model selection strategy enable the SNN model to be adapted to various projects, producing accurate predictions. Experiment results show that the SNN model makes a notable improvement in accuracy compared with its component models and other combinational models do.

Fault Localization Based on Multi-level Similarity of Execution Traces

Since automated fault localization can improve the efficiency of both the testing and debugging process, it is an important technique for the development of reliable software. This paper proposes a novel fault localization approach based on multi-level similarity of execution traces, which is suitable for object-oriented software. It selects useful test cases at class level and computes code suspiciousness at block level. We develop a tool that implements the approach, and conduct empirical studies to evaluate its effectiveness. The experimental results show that our approach has the potential to be effective in localizing faults for object-oriented software.

A Hybrid Approach to Build Prioritized Pairwise Interaction Test Suites

Traditional interaction testing aims to build test suites that cover all t-way interactions of inputs. But in many test scenarios, the entire test suites cannot be fully run due to the limited budget. Therefore it is necessary to take the importance of interactions into account and prioritize these tests of the test suite. In the paper, we use the hybrid approach to build prioritized pairwise interaction test suites (PITS). It adopts a one-test-at-a-time strategy to construct final test suites. But to generate a single test it firstly generates a candidate test and then applies a specific metaheuristic search strategy to enhance this test. Here we experiment four different metaheuristic search strategies. In the experiments, we compare our approach to weighted density algorithm (WDA). Meanwhile, we also analyze the effectiveness of four different search strategies and the effectiveness of the increasing iterations. Empirical results demonstrate the effectiveness of our proposed approach. Keywords-prioritized interaction test suites; greedy algorithm; metaheuristic search