Kamal Zuhairi Zamli

A variable strength interaction test suites generation strategy using Particle Swarm Optimization

This paper highlights a novel strategy for generating variable-strength (VS) interaction test suites, called VS Particle Swarm Test Generator (VS-PSTG). As the name suggests, VS-PSTG adopts Particle Swarm Optimization to ensure optimal test size reduction. To determine its efficiency in terms of the size of the generated test suite, VS-PSTG was subjected to well-known benchmark configurations. Comparative results indicate that VS-PSTG gives competitive results as compared to existing strategies. An empirical case study was conducted on a non-trivial software system to show the applicability of the strategy and to determine the effectiveness of the generated test suites to detect faults.

Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support

Context: Although useful, AI-based variable strength t-way strategies are lacking in terms of the support for high interaction strength. Additionally, most AI-based strategies generally do not address the support for constraints. Addressing the aforementioned issues, this paper elaborates the design, implementation, and evaluation of a novel variable-strength-based on harmony search algorithm, called Harmony Search Strategy (HSS). Objective: The objective of this work is to investigate the adoption of harmony search algorithm for constructing variable-strength t-way strategy. Method: Implemented in Java, HSS integrates the harmony search algorithm as parts of its search engine. Result: Benchmarking results demonstrate that HSS gives competitive results against most existing AI-based (and pure computational) counterparts. However, unlike other AI-based counterparts, HSS addresses the support for high interaction strength and permits the support for constraints. Conclusion: AI-based t-way strategies tend to outperform the pure computational-based strategies in terms of test size.

G2Way A Backtracking Strategy for Pairwise Test Data Generation

Our continuous dependencies on software (i.e. to assist as well as facilitate our daily chores) often raise dependability issue particularly when software is being employed harsh and life threatening or (safety) critical applications. Here, rigorous software testing becomes immensely important. Many combinations of possible input parameters, hardware/software environments, and system conditions need to be tested and verified against for conformance. Due to resource constraints as well as time and costing factors, considering all exhaustive test possibilities would be impossible (i.e. due to combinatorial explosion problem). Earlier work suggests that pairwise sampling strategy (i.e. based on two-way parameter interaction) can be effective. Building and complementing earlier work, this paper discusses an efficient pairwise test data generation strategy, called G2Way. In doing so, this paper demonstrates the correctness of G2Way as well as compares its effectiveness against existing strategies including AETG and its variations, IPO, SA, GA, ACA, and All Pairs. Empirical evidences demonstrate that G2Way, in some cases, outperformed other strategies in terms of the number of generated test data within reasonable execution time.

Application of Particle Swarm Optimization to uniform and variable strength covering array construction

Recently, researchers have started to explore the use of Artificial Intelligence (AI)-based algorithms as t-way (where t indicates the interaction strength) testing strategies. Many AI-based strategies have been developed, such as Ant Colony, Simulated Annealing, Genetic Algorithm, and Tabu Search. Although useful, most existing AI-based t-way testing strategies adopt complex search processes and require heavy computations. For this reason, existing AI-based t-way testing strategies have been confined to small interaction strengths (i.e., t@?3) and small test configurations. Recent studies demonstrate the need to go up to t=6 in order to capture most faults. In this paper, we demonstrate the effectiveness of our proposed Particle Swarm-based t-way Test Generator (PSTG) for generating uniform and variable strength covering arrays. Unlike other existing AI-based t-way testing strategies, the lightweight computation of the particle swarm search process enables PSTG to support high interaction strengths of up to t=6. The performance of our proposed PSTG is evaluated using several sets of benchmark experiments. Comparatively, PSTG consistently outperforms its AI counterparts and other existing testing strategies as far as the size of the array is concerned. Furthermore, our case study demonstrates the usefulness of PSTG for facilitating fault detection owing to interactions of the input components.

IRPS --- An Efficient Test Data Generation Strategy for Pairwise Testing

Software testing is an integral part of software engineering. Lack of testing often leads to disastrous consequences including loss of data, fortunes, and even lives. In order to ensure software reliability, many combinations of possible input parameters, hardware/software environments, and system configurations need to be tested and verified against for conformance. Due to costing factors as well as time to market constraints, considering all exhaustive test possibilities would be infeasible (i.e. due to combinatorial explosion problem). Earlier work suggests that pairwise sampling strategy (i.e. based on two-way parameter interaction) can be effective. Building and complementing earlier work, this paper discusses an efficient pairwise test data generation strategy, called IRPS. In doing so, IRPS is compared against existing strategies including AETG and its variations, IPO, SA, GA, ACA, and All Pairs. Empirical results demonstrate that IRPS strategy, in most cases, outperformed other strategies as far as the number of test data generated within reasonable time.

Modified Recursive Least Squares algorithm to train the Hybrid Multilayered Perceptron (HMLP) network

In this paper, a new learning algorithm, called the Modified Recursive Least Square (MRLS), is introduced for the Hybrid Multilayered Perceptron (HMLP) network. Adopting the Recursive Least Square (RLS) algorithm as its basis, the MRLS algorithm differs from RLS in the way that the weight of the linear connections for the HMLP network is estimated. The convergence rate of the MRLS algorithm is further improved by varying the forgetting factor, optimizing the way the momentum and learning rate are assigned. To investigate its applicability, the MRLS algorithm is demonstrated on the HMLP network using six benchmark data sets obtained from the UCI repository. The classification performance of the HMLP network trained with the MRLS algorithm is compared with those of the HMLP network trained with the Modified Recursive Prediction Error (MRPE) algorithm and the MLP trained with the standard RLS algorithm as well as with other commonly adopted machine learning classifiers. The comparison results indicated that the proposed MRLS trained HMLP network provides significant improvement over RLS trained MLP network, MRPE trained HMLP network, and other machine learning classifiers in terms of accuracy, convergence rate and mean square error (MSE).

Automatic programming assessment and test data generation a review on its approaches

Automatic programming assessment has recently become an important method in assisting lecturers and instructors of programming courses to automatically mark and grade students programming exercises as well as to provide useful feedbacks on students programming solutions. As part of the method, test data generation process plays as an integral part to perform a dynamic testing on students programs. To date, various automated methods for test data generation particularly in software testing field are available. Unfortunately, they are seldom used in the context of automatic programming assessment research area. Nevertheless, there have been limited studies taking a stab to integrate both of them due to more useful features and to include a better quality program testing coverage. Thus, this paper provides a review on approaches that have been implemented in various studies with regard to automatic programming assessment, test data generation and integration of both of them. This review is aimed at gathering different techniques that have been employed to forward an interested reader the starting points for finding further information regarding its trends. In addition, the result of the review reveals the main gap that exists within the context of the considered areas which contributes to our main research topic of interest.

A strategy for Grid based t-way test data generation

Although desirable as an important activity for ensuring quality assurances and enhancing reliability, complete and exhaustive software testing is next to impossible due to resources as well as timing constraints. While earlier work has indicated that pairwise testing (i.e. based on 2-way interaction of variables) can be effective to detect most faults in a typical software system, a counter argument suggests such conclusion cannot be generalized to all software system faults. In some system, faults may also be caused by more than two parameters. As the number of parameter interaction coverage (i.e. the strength) increases, the number of t-way test set also increases exponentially. As such, for large system with many parameters, considering higher order t-way test set can lead toward combinatorial explosion problem (i.e. too many data set to consider). We consider this problem for t-way generation of test set using the Grid strategy. Building and complementing from earlier work in In-Parameter-Order-General (or IPOG) and its modification (or MIPOG), we present the Grid MIPOG strategy (G_MIPOG). Experimental results demonstrate that G_MIPOG scales well against the sequential strategies IPOG and MIPOG with the increase of the computers as computational nodes.

Algebraic strategy to generate pairwise test set for prime number parameters and variables

Generating pairwise test set when the total number of variables is prime numbers has a remarkable property in that the test case generation process can be simplified by applying straightforward strategy that does not require any storage. This paper discusses the said algebraic strategy and compares the results with the well-known orthogonal array strategy. Additionally, this paper also demonstrates the applicability and simplicity of the strategy as compared to orthogonal array to obtain optimal test set for pairwise testing.

PSTG: A T-Way Strategy Adopting Particle Swarm Optimization

As an activity to ensure quality and conformance, testing is one of the most important activities in any software or hardware product development cycle. Often, the challenge in testing is that the system may support a wide range of configurations. Ideally, it is desirable to test all of these configurations exhaustively. However, exhaustive testing is practically impossible due to time and resource limitations. To address this issue, there is a need for a sampling strategy that can select a subset of inputs as test data from an inherently large search space. Recent findings demonstrate that t-way interaction testing strategies based on artificial intelligence (i.e. where t indicates interaction strength) have been successful to obtain a near optimal solution resulting into smaller test set to be considered. Motivated by such findings, we have developed a new test generation strategy, called Particle Swarm Test Generator (PSTG). In this paper, we discuss the design of PSTG and demonstrate our preliminary test size reduction results against other competing t-way strategies including IPOG, WHITCH, Jenny, TConfig, and TVG.