F. T. Chan

In black and white: an integrated approach to class-level testing of object-oriented programs

Because of the growing importance of object-oriented programming, a number of testing strategies have been proposed. They are based either on pure black-box or white-box techniques. We propose in this article a methodology to integrate the black- and white-box techniques. The black-box technique is used to select test cases. The white-box technique is mainly applied to determine whether two objects resulting from the program execution of a test care are observationally equivalent. It is also used to select test cases in some situations. We define the concept of a fundamental pair as a pair of equivalent terms that are formed by replacing all the variables on both sides of an axiom by normal forms. We prove that an implementation is consistent with respect to all equivalent terms if and only if it is consistent with respect to all fundamental pairs. In other words, the testing coverage of fundamental pairs is as good as that of all possible term rewritings, and hence we need only concentrate on the testing of fundamental pairs. Our strategy is based on mathematical theorems. According to the strategy, we propose an algorithm for selecting a finite set of fundamental pairs as test cases. Given a pair of equivalent terms as a test case, we should then determine whether the objects that result from executing the implemented program are observationally equivalent. We prove, however, that the observational equivalence of objects cannot be determined using a finite set of observable contexts (which are operation sequences ending with an observer function) derived from any black-box technique. Hence we supplement our approach with a “relevant observable context” technique, which is a heuristic white-box technique to select a relevant finite subset of the set of observable contexts for determining the observational equivalence. The relevant observable contezxts are constructed from a data member relevance graph (DRG), which is an abstraction of the given implementation for a given specificatin. A semiautomatic tool hass been developed to support this technique.

Proportional sampling strategy: guidelines for software testing practitioners

Abstract Recently, several sufficient conditions have been developed that guarantee partition testing to have a higher probability of detecting at least one failure than random testing. One of these conditions is that the number of test cases selected from each partition is proportional to the size of the partition. We call such a method of allocating test cases the proportional sampling strategy. Although this condition is not the most general one, it is the most easily and practically applicable one. In this paper, we discuss how the proportional sampling strategy can be applied effectively in practice. Some practical issues that need to be attended are identified and guidelines to deal with these issues are suggested.

Test case selection with and without replacement

Previous theoretical studies on the effectiveness of partition testing and random testing have assumed that test cases are selected with replacement. Although this assumption has been well known to be less realistic, it has still been used in previous theoretical work because it renders the analyses more tractable. This paper presents a theoretical investigation aimed at comparing the effectiveness when test cases are selected with and without replacement, and exploring the relationships between these two scenarios. We propose a new effectiveness metric for software testing, namely the expected number of distinct failures detected, to re-examine existing partition testing strategies.

The application of Prolog to structured design

In this paper, we investigate the feasibility of applying logic programming to structured design. We propose to use Prolog as a common machinery for the representation of various structured tools. We illustrate through examples how to produce structure charts from data flow diagrams, and evaluate them according to commonly recommended design guidelines. If the structure charts produced are not satisfactory, the inherent backtracking mechanism in Prolog will help to produce other versions for further evaluation.

Software testing education and training in Hong Kong

While the use of computer applications is widely spread in every business and, hence, the reliability of software is critical, it is believed that many organizations involved in software development do not take software testing sufficiently seriously as an important task. It is worthwhile to find out how far organizations are carrying out software testing in a systematic and structured manner or still taking on an ad-hoc approach. A survey was conducted to understand the software testing practices and the level of related education and training in Hong Kong. It was found that most testing team members did not have formal training in software testing. University curricula generally did not prepare graduates with enough coverage in software testing. It is proposed that a review of the current software engineering curricula in the universities to examine the coverage of software testing will be useful to the development of quality software.

On the effectiveness of test case allocation schemes in partition testing

In partition testing, the input domain is divided into two or more disjoint subdomains according to some criteria and then test data are selected from each subdomain. Hence we must design a partitioning scheme that governs how to divide the input domain, and a test case allocation scheme that controls how to allocate test cases to the subdomains. Given any partitioning scheme, the test case allocation scheme plays a crucial part in the effectiveness of testing. This paper develops guidelines for comparing different test case allocation schemes. This provides solutions to the class of problems where various test case allocation schemes under the same partitioning scheme are compared for the effectiveness of testing. We illustrate with examples several potential applications of our findings.

On the effectiveness of the optimally refined proportional sampling testing strategy

Recently, the effectiveness of subdomain testing and random testing has been studied analytically. T.Y. Chen and Y.T. Yu (1994) found that, for the case of disjoint subdomains, as long as the number of test cases selected from each subdomain is proportional to its size (the proportional sampling strategy), the probability of revealing at least one failure using subdomain testing is not less than that using random testing. This paper investigates the effectiveness of the optimally refined proportional sampling (ORPS) strategy, which is a special case of the proportional sampling strategy. The ORPS strategy is simple in concept, and the implementation cost is usually low. An empirical study has been conducted for a sample of published programs with seeded errors. The performance of this strategy was found to be better than random testing.

An Axiom-based Test Case Selection Strategy for Object-oriented Programs

Because of the growing importance of object-oriented programming, a number of testing approaches have been proposed. Franld et al. propose the application of the functional approach, using algebraic specifications for the generation of test cases and the validation of methods. Given a specification, Frankl et al. propose that equivalent terms should give observably equivalent objects, and offer general heuristics on the selection of equivalent terms for testing. Their guidelines, however, are only supported by limited empirical results, do not have a theoretical basis, and provide no guarantee of effectiveness.