Jason McDonald

From Object-Z specifications to ClassBench test suites

This paper describes a method for specification‐based class testing that incorporates test case generation, execution, and evaluation based on formal specifications. This work builds on previous achievements in the areas of specification‐based testing and class testing by integrating the two within a single framework. The initial step of the method is to generate test templates for individual operations from a specification written in the Object‐Z specification language. These test templates are combined to produce a finite state machine for the class that is used as the basis for test case execution using the ClassBench test execution framework. An oracle derived from the Object‐Z specification is used to evaluate the outputs. The method is explained using a simple example and its application to a more substantial case study is also discussed. Copyright

Translating Object-Z specifications to passive test oracles

A test oracle provides a means for determining whether an implementation functions according to its specification. A passive test oracle checks the behaviour of the implementation, but does not attempt to reproduce this behaviour. The paper describes the translation of formal specifications of container classes to passive test oracles. Specifically, we use Object-Z for specifications and C++ for oracles. We discuss several practical issues for the use of formal specifications in test oracle generation. We then present the translation process and illustrate it with an example based on an integer set class. Our approach is illustrated with an example based on an integer set class.

Translating Object-Z specifications to object-oriented test oracles

This paper describes the translation of Object-Z specifications of container classes to C++ test oracle classes. It presents a three-stage translation process and describes how the derived test oracles are integrated into the ClassBench testing framework. The method caters for object-oriented features such as inheritance and aggregation. Translation issues and the limitations of the method are discussed. Our approach is illustrated with an example based on an integer set class.

Programmatic testing of the Standard Template Library containers

We describe part of an STL conformance test suite under development. Test suites for all of the STL containers have been written, demonstrating the feasibility of thorough and highly automated testing of industrial component libraries. We describe affordable test suites that provide good code and boundary value coverage, including the thousands of cases that naturally occur from combinations of boundary values. We show how two simple oracles can provide fully automated output checking for all the containers. We refine the traditional categories of black-box and white-box testing to specification-based, implementation-based and implementation-dependent testing, and show how these three categories highlight the key cost/thoroughness trade-offs.

Formal Derivation of Finite State Machines for Class Testing

Previous work on generating state machines for the purpose of class testing has not been formally based. There has also been work on deriving state machines from formal specifications for testing non-object-oriented software. We build on this work by presenting a method for deriving a state machine for testing purposes from a formal specification of the class under test. We also show how the resulting state machine can be used as the basis for a test suite developed and executed using an existing framework for class testing. To derive the state machine, we identify the states and possible interactions of the operations of the class under test. The Test Template Framework is used to formally derive the states from the Object-Z specification of the class under test. The transitions of the finite state machine are calculated from the derived states and the classs operations. The formally derived finite state machine is transformed to a ClassBench testgraph, which is used as input to the ClassBench framework to test a C++ implementation of the class. The method is illustrated using a simple bounded queue example.

Specification-based class testing with ClassBench

In this paper, we present an approach that combines specification-based testing and class testing. In particular, we provide a method for generating Finite State Machines (FSMs) from formal, object-oriented specifications, and use the ClassBench testing framework to build a test suite from those formally generated FSMs. We briefly outline our approach and focus on one step in the approach; the transformation of the formally derived FSM into a ClassBench testgraph, which is used by ClassBench to drive the test execution. We illustrate the method with a simple bounded queue class, and discuss the application of the method to a larger example, which is a simplified model of a process scheduling system.

Testing inheritance hierarchies in the ClassBench framework

Inheritance is a feature of the feature-oriented paradigm that permits substantial reuse of code. For us to have confidence in reused code, it must be adequately tested. Whilst object-oriented analysis, design and implementation techniques have received much attention in recent literature, object-oriented testing has been given comparatively little consideration. This paper presents an adaptation of the ClassBench methodology to the testing of inheritance hierarchies in C++. A small case study which applies the strategy to an inheritance hierarchy of a commercial class library is presented.

Module testing embedded software-an industrial pilot project

This paper reports on an industrial pilot project that introduces systematic, automated module testing for embedded software in distributed, real-time, control systems. The systems are used in safety-related applications, are complex in nature, and hence have strong requirements for test coverage, auditability and repeatability. This paper explores issues of isolating modules from the run-time environment, improving integration of testing into the development environment, automating testing, and improving test planning and documentation. Metrics were gathered throughout the project that allow a coarse cost-benefit evaluation. Code coverage metrics for statement and branch coverage were also gathered using a commercial code coverage analysis tool. The testing exposed a number of latent faults within the software, and the overall results of the project show that module testing is feasible for this complex, embedded software.