Code Coverage Based Test Case Selection and Prioritization
IInternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013DOI : 10.5121/ijsea.2013.4604 39 C ODE C OVERAGE B ASED T EST C ASE S ELECTION A ND P RIORITIZATION
R.Beena , Dr.S.Sarala Research Scholar, Dept. of Information Technology, Bharathiar University, Coimbatore. Assistant Professor, Dept. of Information Technology, Bharathiar University,Coimbatore A BSTRACT
Regression Testing is exclusively executed to guarantee the desirable functionality of existing softwareafter pursuing quite a few amendments or variations in it. Perhaps, it testifies the quality of the modifiedsoftware by concealing the regressions or software bugs in both functional and non-functional applicationsof the system. In fact, the maintenance of test suite is enormous as it necessitates a big investment of timeand money on test cases on a large scale. So, minimizing the test suite becomes the indispensable requisiteto lessen the budget on regression testing. Precisely, this research paper aspires to present an innovativeapproach for the effective selection and prioritization of test cases which in return may procure a maximumcode average. K EYWORDS
Test Case Selection, Test Case Prioritization, Code Coverage
1. I
NTRODUCTION
Regression testing is an authentication method pursued in all levels of system and softwaretesting. Despite ensuring the functioning capacity of the software or system after makingamendments, Regression Testing, exhibits a predominant function with the previously deployedtest codes of the enhanced software. The prime aspiration of running a Regression Test is toassure that modified or amended component of software does not give way for bugs in theunaltered portion of the software. The re-execution of test cases are performed to verify that theprevious functionality clubbed with the present changes is desirably functioning.The various regression testing techniques are test case minimization, test case selection and testcase prioritization .The aim of test case minimization technique is to eliminate the redundant testcases, while test case selection techniques are performed to reduce the size of a test suite. Testcase prioritization techniques are concerned with ordering of test cases for detection of faults atthe earliest. This paper presents a customized technique for Test case selection and Test caseprioritization. nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 40
Test case selection implies identifying a smaller subset of test suite from the existing large testsuite [1]. According to [2], test case selection problem is stated subsequently.
Given : The original program, P, the revised version of P, P' and a test suite, T.
Aim : To identify T' ∈ T, for the modified version P'Test Case Prioritization is the process of arranging test cases in an order according to somecriteria. Test case prioritization problem defined by Rothermel et al. [3] is follows:
Given : A test suite, T, the set of permutations of T, PT, a function from PT to the realnumbers, f.
Aim:
To identify T ′ ∈ PT such that ( ∀ T ″) (T″ ∈ PT) (T ″≠T′) f (T ′) ≥ f (T ″)Here, ‘PT’ represents the set of all possible prioritizations of ‘T’ and ‘f’. The function that is applied to any such ordering actually yields an award value for the respective ordering.
2. RELATED WORK
Fischer et al. formulated a test case selection problem with the application of IntegerProgramming [4]. The variations of the control flow were not discussed in this approach.Agrawal et al. outlined an exclusive strategy on test case selection with a special perspective tothe discrepancies found in the program slicing techniques [5].Rothermel and Harrold elucidated regression test case selection techniques based on graphwalking of Control, Program Dependence Graphs [6], and System Dependence Graphs [7]besides, Control Flow Graphs [8].Benedusi et al. executed path analysis for test case selection [9]. A testing structure calledTestTube was introduced by Chen et al. [10] which make use of a modification-based method forselection of test cases.Leung and White highlighted a firewall technique for regression testing of system integration[11]. Laski and Szemer offered a technique for test case selection which is based on clusteridentification technique [12].In [13], [14], Rothermel et al. were the premiers to study test case prioritization predicaments thatpaved a way to them to present six different strategies based on the coverage of statement orbranches.In [15], Li et al. gives empirical study results of two metaheuristic search techniques and threegreedy search techniques applied to six programs for regression test case prioritization.In [16], Praveen et al. initiated a novel test case prioritization algorithm that calculates averagefaults observed per minute. nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 41
A Regression Testing Technique for Test Case Prioritization based on Code Coverage criteria isrecommended by K.K.Aggarwal in [17].
3. T
EST C ASE S ELECTION
The test cases those are available for the existing version of the program is grouped into threeclusters. Those clusters are named as out-dated, required and surplus. The out-dated clustercontains the test cases that are not required by both the original program and the modifiedprogram. The required test case group consists of the test cases that are required to be executedfor the modified version of the software. The surplus group comprises of test cases that may beessential for the later versions of P but are not required for the modified version of P i.e. P'. Thealgorithm for Test Case Selection (TCS) which is contributed in the previous work [18] is givenin Figure 1.
Algorithm TCS
Input:- Matrix TCC ij representing the test cases and their statements covered- Vector SDEL i representing the statements deleted in P’ - Vector SMOD i representing the statements modified in P’ Output :- Modified Matrix TCC ij , Cluster of Test Cases out_dated i , surplus i , required i begin1. for each statement that belongs to SDEL i Remove the corresponding statements from TCC ij .2. Find the sum of each row.3. if sum of the row is 0 thenAdd the corresponding test case in the vector out_dated i. andRemove it from TCC ij .4. Find the test cases that do not cover the statements in the vector SMOD i ,Add the corresponding test case in the vector surplus i andRemove it from TCC ij .5. Add the left over test cases in the vector required i. End
Figure 1. Algorithm TCS
4. T
EST C ASE P RIORITIZATION
The output obtained from algorithm TCS is supplied as input to the algorithm Test CasePrioritization (TCP) which is described in Figure 2. An example for the steps of the algorithmTCS and TCP is elucidated in section 5. nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 42
Algorithm TCP
Input:- Modified Matrix TCC ij representing selected test cases and their statements coveredOutput :- Vector TCP i which contains the test cases to achieve 100% code coverage.begin1. Find the sum of each row of the matrix TCC ij .2. Select the test case with highest sum and add that test case into the vector TCP i .3. Remove all the statements covered by that test case.4. Repeat step1 until all the statements are deleted.End Figure 2. Algorithm TCP
5. EXPERIMENTS AND RESULTS
The original version of the program contains 15 statements and 15 test cases. The test cases andthe coverage of the statements by the test cases are represented as a binary matrix. The binarymatrix represented as (TCC ij ) is given in Table 1. Table1. Test cases and statement coverage TCC ij S S S S S S S S S S S S S S S T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
T11
T12
T13
T14
T15
Let us consider, in the modified version of the program the statements S3, S4, S6, S8, S10, S13have been deleted and the statements S2, S7, S15 have been modified. So the two vectors SDEL i and SMOD i are represented as nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 43 SDEL i = {S3, S4, S6, S8, S10, S13} SMOD i = {S2, S7, S15} Table2. Modified TCC ij The matrix TCC ij is personalized by removing the statements that are available in the vectorgroup SDEL i at the end of the execution of step 1.The modified TCC ij is given in Table 2. Thenumber of statements covered by each test case is calculated according to step 2. For example T1covers four statements namely S1, S5, S7 and S11.Table 3 represents the total number ofstatements covered by each test case. Table3. Number of statements covered by test cases
Test Cases StatementsCovered
T1 4T2 2T3 0T4 4T5 4T6 5T7 5T8 4T9 3T10 6T11 0T12 5T13 4T14 1T15 5
S1 S2 S5 S7 S9 S11 S12 S14 S15T1 T2 T3 T4 T5 T6 T7 T8 T9 T10
T11
T12
T13
T14
T15
As given in step 3, the test cases with the sum as zero are removed from the matrix TCC ij . Nowthe new matrix TCC ij is given in Table 4. A new vector out_dated i is created which contains theremoved test cases from TCC ij . The vector out-dated i = {T3, T11} Table4.TCC ij without out-dated i S1 S2 S5 S7 S9 S11 S12 S14 S15T1 T2 T4 T5 T6 T7 T8 T9 T10
T12
T13
T14
T15
The vector SMOD i contains the statements that are modified in the new version of the programand the test cases that do not cover those statements are removed from TCC ij and inserted into thecluster surplus i . The new TCC ij is given in Table 5. The vector surplus i = {T2, T7, T15} Table5.TCC ij without surplus i S1 S2 S5 S7 S9 S11 S12 S14 S15T1 T4 T5 T6 T8 T9 T10
T12
T13
T14
All the remaining test cases that are available in TCC ij are inserted into a new cluster grouprequired i as mentioned in step 5.The vectorrequired i = {T1, T4, T5, T6, T8, T9, T10, T12, T13, T14}The comparison between the original size of the test suite and the reduced size of the test suite isspecified in Figure 3. The result shows that there is a notable reduction in the size between thetwo test suites. nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 45Figure3. Test Suite Size after Selection Input matrix TCC ij for Test Case Prioritization is given in Table 5.Iteration 1:As given in step 1, the number of statements covered by each test case is counted from the newmatrix TCC ij . It is given in Table 6. Table 6. Number of statements covered by test cases
Test Cases StatementsCovered
T1 4T4 4T5 4T6 5T8 4T9 3T10 6T12 5T13 4T14 1
As given in step 2, the test case with highest sum is removed from TCC ij and that test case isadded into the Test Case Prioritized vector TCP i . The vector TCP i = {T10}.All the statementsthat are covered by the test case T10 is removed from TCC ij . The modified TCC ij is given inTable 7. Test Suite Size nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 46Table7.Updated TCC ij Iteration 2:As given in step 1, the sum of each row of the updated matrix TCC ij given in Table 7 is computedand the sum is specified in Table 8. Table8. Number of statements covered by test cases
Test Cases StatementsCovered
T1 2T4 0T5 1T6 2T8 2T9 1T10 0T12 1T13 1T14 0
As given in step 2, the test case with highest sum is removed from TCC ij and that test case isadded into the vector TCP i . Here in this example, there are three test cases {T1, T6, T8} withhighest sum. The test case T1 is selected here. The issue of equal priority is to be considered infuture. Now the vector TCP i = {T10, T1}. All the statements that are covered by the test case T1is removed from TCC ij . The modified TCC ij is given in Table 9. nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 47Table 9. Updated TCC ij Iteration 3:As given in step 1, the sum of each row of the updated matrix TCC ij given in Table 9 is computedand the sum is specified in Table 10. Table 10. Number of statements covered by test cases
Test Cases StatementsCovered
T1 0T4 0T5 0T6 1T8 1T9 0T10 0T12 0T13 1T14 0
As given in step 2, the test case with highest sum is removed from TCC ij and that test case isadded into the vector TCP i . Here in this example, there are three test cases {T6, T8, T13} withhighest sum. The test case T6 is selected here. The final prioritized vectorTCP i = {T10, T1, T6}Figure4 gives the size of the test suite after test case prioritization. The size of the test suite isvery much reduced and hence the cost of regression testing and time for execution of test casescan be minimized to a greater extent. nternational Journal of Software Engineering & Applications (IJSEA), Vol.4, No.6, November 2013 48Figure4. Test Suite Size after Prioritization
6. C
ONCLUSION
Regression testing is carried out in the maintenance phase of the software development to retestthe software for the revisions it has endured and to confirm the accurate functionalities of therevised version. A new technique for test case selection and test case prioritization process forregression testing is proposed in this paper. The proposed technique is very effective in terms ofcost and time involved in regression testing. In future, the regression testing techniques may becombined with optimization algorithms to contribute more enhanced results. R EFERENCES [1] Rothermel G, Harrold MJ. A safe, efficient algorithm for regression test selection. Proceedings ofInternational Conference on Software Maintenance (ICSM 2003), IEEE Computer Society Press,1993; 358 – – G. Rothermel, R. Untch, C. Chu, and M.J.Harrold, “Prioritizing Test Cases for Regression Testing,”