Kuo-Chung Tai

Test generation for Boolean expressions

We propose a new strategy for generating test cases for Boolean expressions. In the past, we reported the BOR (Boolean Operator) strategy for generating test cases for predicates which are singular: which contain only one occurrence of each constituent Boolean variable. We also reported results of the empirical studies that were carried out to study the effectiveness of the strategy, but the BOR algorithm did not work well with non-singularities: multiple occurrences of constituent Boolean variables. The solution we propose for the problem is a combination of the original BOR strategy and the MI (Meaning Impact) strategy reported elsewhere. Our approach is to divide a Boolean expression into components that do not have common variables, apply the MI strategy to non-singular components, and the BOR strategy to singular components, and then apply the BOR strategy to combine the test sets generated for all component. Our empirical results indicate that our hybrid approach produces fewer tests for a Boolean expression than the MI strategy. The fault detection capability of our proposed approach has also been found to be comparable to that of the MI strategy. Our test generation strategy can be used to improve the reliability and safety of a program.

Automatic test generation for predicates

We propose a new technique for automatic generation of test cases for predicates. Earlier we proposed an efficient and effective test generation strategy for Boolean expressions. We now extend this strategy to predicates. Our new strategy addresses a number of issues, including: analysis of dependencies between relational expressions in a predicate P; generation of test constraints for P based on the detection of Boolean and relational operator faults in P; and generation of actual tests according to the generated test constraints for P. We propose the use of constraint logic programming (CLP) to automate test data generation for a predicate. Furthermore, we propose an incremental approach to apply CLP techniques to solve a constraint system. Since our technique is specification-based, it can facilitate generation of expected outputs for actual tests.

Empirical studies of predicate-based software testing

We report the results of three empirical studies of fault detection and stability performance of the predicate-based BOR (Boolean Operator) testing strategy. BOR testing is used to develop test cases based on formal software specification, or based on the implementation code. We evaluated the BOR strategy with respect to some other strategies by using Boolean expressions and actual software. We applied it to software specification cause-effect graphs of a safety-related real-time control system, and to a set of N-version programs. We found that BOR testing is very effective at detecting faults in predicates, and that BOR-based approach has consistently better fault detection performance than branch testing, thorough (but informal) functional testing, simple state-based testing, and random testing. Our results indicate that BOR test selection strategy is practical and effective for detection of faulty predicates and is suitable for generation of safety-sensitive test-cases.<<ETX>>

Evaluation of a predicate-based software testing strategy

In this paper, we report the results of four empirical studies for evaluating a predicate-based software testing strategy, called BOR (Boolean operator) testing. The BOR testing strategy focuses on the detection of Boolean operator faults in a predicate, including incorrect AND/OR operators and missing or extra NOT operators. Our empirical studies involved comparisons of BOR testing with several other predicate-based testing strategies, using Boolean expressions, a real-time control system, and a set of N-version programs. For program-based test generation, BOR testing was applied to predicates in a program. For specification-based test generation, BOR testing was applied to cause-effect graphs representing software specification. The results of our studies indicate that BOR testing is practical and effective for both specification- and program-based test generation.

Specification-based testing using cause-effect graphs

In this paper we discuss the advantages and limitations of a specification‐based software testing technique we call CEG‐BOR. There are two phases in this approach. First, informal software specifications are converted into cause‐effect graphs (CEG). Then, the Boolean OperatoR (BOR) strategy is applied to design and select test cases. The conversion of an informal specification into a CEG helps detect ambiguities and inconsistencies in the specification and sets the stage for design of test cases. The number of test cases needed to satisfy the BOR strategy grows linearly with the number of Boolean operators in CEG, and BOR testing guarantees detection of certain classes of Boolean operator faults. But, what makes the approach especially attractive is that the BOR based test suites appear to be very effective in detecting other fault types. We have empirically evaluated this broader aspect of the CEG‐BOR strategy on a simplified safety‐related real‐time control system, a set of N‐version programs, and on elements of a commercial data‐base system. In all cases, CEG‐BOR testing required fewer test cases than those generated for the applications without the use of CEG‐BOR. Furthermore, in all cases CEG‐BOR testing detected all faults that the original, and independently generated, application test‐suites did. In two instances CEG‐BOR testing uncovered additional faults. Our results indicate that the CEG‐BOR strategy is practical, scalable, and effective across diverse applications. We believe that it is a cost‐effective methodology for the development of systematic specification‐based software test‐suites.

Condition-based software testing strategies

The author defines two condition testing strategies, BRO (Boolean and relational operator) and BRE (Boolean and relational expression) testing. These two testing strategies are different from existing condition testing strategies in that they are based on the detection of both Boolean and relational expression errors in a condition. For a condition with n operands, the number of tests required by BRO or BRE testing is at most 2(n+1). Based on empirical studies of the algorithms SBEMIN and SBEMINSEN and the theoretical properties of BRO and BRE testing, it is believed that BRO and BRE testing is practical and effective for testing programs containing complicated conditions.<<ETX>>

Model Checking Based on Simultaneous Reachability Analysis

Simultaneous reachability analysis (SRA) is a recently proposed approach to alleviating the state space explosion problem in reachability analysis of concurrent systems. The concept of SRA is to allow a global transition in a reachability graph to contain a set of transitions of different processes such that the state reached by the global transition is independent of the execution order of the associated process transitions. In this paper, we describe how to apply the SRA approach to concurrent systems for model checking. We first describe an SRA-based framework for producing a reduced state graph that provides sufficient information for model checking. Following this framework, we present an algorithm that generates a reduced state graph for the extended finite state machine (EFSM) model with multiple ports. Empirical results indicate that, our SRA reduction algorithm performs as good as or better than the partial order reduction algorithm in SPIN.

Automatic test-generation for predicates [software testing]

The authors propose a new technique for the automatic generation of test cases for predicates. Earlier, they proposed an efficient effective test generation strategy for Boolean expressions. They now extend this strategy to predicates. Their new strategy addresses several issues, including: analysis of dependencies between relational expressions in a predicate /spl Pscr/; generation of test constraints for /spl Pscr/ based on the detection of Boolean and relational operator faults in /spl Pscr/; and generation of actual tests according to the generated test constraints for /spl Pscr/. They propose: the use of constraint logic programming (CLP) to automate test-data generation for a predicate; and an incremental approach to apply CLP techniques to solve a constraint system. Since their technique is specification-based, it can facilitate generation of anticipated outputs for actual tests.

Deadlock detection of EFSMs using simultaneous reachability analysis

Simultaneous reachability analysis (SRA) is a recently proposed technique to alleviate the state space explosion problem in reachability analysis of concurrent systems. Its goal is to reduce the number of generated states while guaranteeing the detection of certain types of faults in the system such as deadlock and unexecutable transitions. The main idea of SRA is to allow a global transition in a reachability graph to contain a set of local transitions (i.e. transitions of individual processes) such that the state reached by the global transition is independent of the execution order of the associated local transitions. In this paper, we show how to apply the SRA approach to systems modeled as extended finite state machines (EFSM) with multiple ports. Empirical results from applying our SRA algorithm to the dining philosophers problem indicate that our algorithm reduces the number of generated states and the computation time by about 90%.

On Godefroid's state-less search technique for testing concurrent programs

P. Godefroid (1997) developed a state-space exploration technique that does not store states in memory. This stateless search technique is effective for testing concurrent programs. It performs deterministic executions of a concurrent program by automatically controlling the execution of synchronization operations. This technique also uses partial order reduction methods to reduce the chance of executing two or more different totally-ordered paths with the same partial order. In this paper, we propose extensions to Godefroids technique and compare it with other stateless search techniques for testing concurrent programs. Our empirical results indicate that a combination of stateless searching and a simple cycle prediction method is very cost-effective for detecting deadlocks and violations of assertions.