Sau-Fun Tang

On the identification of categories and choices for specification-based test case generation

The category-partition method and the classification-tree method help construct test cases from specifications. In both methods, an early step is to identify a set of categories (or classifications) and choices (or classes). This is often performed in an ad hoc manner due to the absence of systematic techniques. In this paper, we report and discuss three empirical studies to investigate the common mistakes made by software testers in such an ad hoc approach. The empirical studies serve three purposes: (a) to make the knowledge of common mistakes known to other testers so that they can avoid repeating the same mistakes, (b) to facilitate researchers and practitioners develop systematic identification techniques, and (c) to provide a means of measuring the effectiveness of newly developed identification techniques. Based on the results of our studies, we also formulate a checklist to help testers detect such mistakes.

Contributions of tester experience and a checklist guideline to the identification of categories and choices for software testing

An early step for most black-box testing methods is to identify a set of categories and choices (or their equivalents) from the specification. The identification is often performed in an ad hoc manner, thus the quality of categories and choices is in doubt. Poorly identified categories and choices will affect the comprehensiveness of test cases. In this paper, we describe several comparative studies using three commercial specifications and discuss the major results. The objectives of our studies are (a) to investigate the differences in the types and amounts of mistakes made between inexperienced and experienced software testers in an ad hoc identification approach and (b) to determine the extent of mistake reduction after discussing the mistakes with the software testers and providing them with an identification checklist.

Towards a problem-driven approach to perspective-based reading

The quality of a requirements specification has a great impact on the quality of the software developed. Because of this, a requirements specification should be complete, correct, consistent, and unambiguous. Otherwise, defects may remain undetected, resulting in the delivery of a faulty software product to the users. Motivated by this, Basili et al. have developed the perspective-based reading (PBR) technique to help identify defects in requirements specifications. In this paper we propose a problem-driven approach for supporting the PBR technique. We also discuss the experience of applying our proposal to a real-life requirements specification.

CHOC'LATE: a framework for specification-based testing

In spite of its importance in software reliability, testing is labor intensive and expensive. It has been found that software testing without a good strategy may not be more effective than testing the system with random data. Obviously, the effectiveness of testing relies heavily on how well the test suite --- the set of test cases actually used --- is generated. This is because the comprehensiveness of the test suite will affect the scope of testing and, hence, the chance of revealing software faults. There are two main approaches to generating test suites: specification-based and code-based. The former generates a test suite from information derived from the specification, without requiring the knowledge of the internal structure of the program. The latter approach, on the other hand, generates a test suite based on the source code of the program. Neither of these approaches is sufficient; they are complementary to one another. In software development, the requirements have to be established before implementation, and the specification should exist prior to coding. In this respect, the specification-based approach to test suite generation is particularly useful because test cases can be generated before coding has been completed. This facilitates software development phases to be performed in parallel, thus allowing time for preparing more thorough test plans and yet shortening the length of the whole process.

An Automatic Test Data Generation System Based on the Integrated Classification-Tree Methodology

Grochtmann and Grimm have developed the classification-tree method (CTM) to facilitate software testers in generating test cases from functional specifications. While the method is very useful, it is hindered by the lack of a systematic tree construction algorithm. This problem has been alleviated by Chen et al. via their ”integrated” classification-tree methodology (ICTM). In this paper, we describe and discuss a prototype system addict that is built on ICTM.

A STUDY ON A PATH-BASED STRATEGY FOR SELECTING BLACK-BOX GENERATED TEST CASES

Various black-box methods for the generation of test cases have been proposed in the literature. Many of these methods, including the category-partition method and the classification-tree method, follow the approach of partition testing, in which the input domain is partitioned into subdomains according to important aspects of the specification, and test cases are then derived from the subdomains. Though comprehensive in terms of these important aspects, execution of all the test cases so generated may not be feasible under the constraint of tight testing resources. In such circumstances, there is a need to select a smaller subset of test cases from the original test suite for execution. In this paper, we propose the use of white-box information to guide the selection of test cases from the original test suite generated by a black-box testing method. Furthermore, we have developed some techniques and algorithms to facilitate the implementation of our approach, and demonstrated its viability and benefits by means of a case study.

White on black: a white-box-oriented approach for selecting black box-generated test cases

Many useful test case construction methods that are based on important aspects of the specification have been proposed in the literature. A comprehensive test suite thus obtained is often very large and yet is non-redundant with respect to the aspects identified from the specification. This paper addresses the problem of selecting a subset of test cases from such a test suite. We propose the use of white box criteria to select test cases from the initial black-box-generated test suite. We illustrate our ideas with examples and demonstrate the viability and benefits of our approach by means of a case study.