Alexandre Petrenko

Test selection based on communicating nondeterministic finite-state machines using a generalized Wp-method

Presents a method of generating test sequences for concurrent programs and communication protocols that are modeled as communicating nondeterministic finite-state machines (CNFSMs). A conformance relation, called trace-equivalence, is defined within this model, serving as a guide to test generation. A test generation method for a single nondeterministic finite-state machine (NFSM) is developed, which is an improved and generalized version of the Wp-method that generates test sequences only for deterministic finite-state machines. It is applicable to both nondeterministic and deterministic finite-state machines. When applied to deterministic finite-state machines, it yields usually smaller test suites with full fault coverage than the existing methods that also provide full fault coverage, provided that the number of states in implementation NFSMs are bounded by a known integer. For a system of CNFSMs, the test sequences are generated in the following manner: a system of CNFSMs is first reduced into a single NFSM by reachability analysis; then the test sequences are generated from the resulting NFSM using the generalized Wp-method. >

Testing deterministic implementations from nondeterministic FSM specifications

In this paper, conformance testing of protocols specified as nondeterministic finite state machines is considered. Protocol implementations are assumed to be deterministic. In this testing scenario, the conformance relation becomes a preorder, so-called reduction relation between FSMs. The reduction relation requires that an implementation machine produces a (sub)set of output sequences that can be produced by its specification machine in response to every input sequence. A method for deriving tests with respect to the reduction relation with full fault coverage for deterministic implementations is proposed based on certain properties of the product of specification and implementation machines.

Modular System Verification by Inference, Testing and Reachability Analysis

Verification of a modular system composed of communicating components is a difficult problem, especially when the models of the components are not available. Conventional testing techniques are not efficient in detecting erroneous interactions of components because such interactions often occur as interleavings of events that are difficult to reproduce in a modular system. The problem of detecting intermittent errors in the absence of models of components is addressed in this paper. A method to infer a controllable approximation of components through testing is elaborated. The inferred finite state models of components are used to detect intermittent errors and other compositional problems in the system through reachability analysis. The models are refined at each analysis step thus making the approach iterative.

Automating the process of test derivation from SDL specifications

Publisher Summary This chapter presents a set of automated tools for the development of conformance tests, following a methodology based on a partial unfolding of a given Specification and Description Language (SDL) and describes the behavior of the system under test. The methodology relies on finite state machine (FSM)-based test derivation methods that focus on the fault coverage aspect of testing. The tool kit offers the test designer a number of options for achieving different levels of fault coverage. In particular, it provides support for partial specifications, grouped transitions, and timers. The tests that are generated in SDL are completed by hand concerning certain aspects related to signal parameters, most of these adjustments are relatively straightforward and certain parts of the original SDL specification can be reused without change. The chapter also describes the experience of using the tool kit for the development of a test suite for the asynchronous transfer mode (ATM) PNNI signaling protocol.

A test generation tool for specifications in the form of state machines

This paper describes a software tool, TAG (test automatic generation), that automatically generates test cases for an FSM specification. It implements the so-called transition identification approach for test derivation, and may output test cases in the form of an SDL skeleton. The description focuses on the functions of the tool and the methods implemented in the tool, especially, the heuristic solution to the minimization of state identification sequences.

Test Generation for CEFSM Combining Specification and Fault Coverage

We discuss how specification coverage and fault coverage based test derivation strategies can be combined. In particular, the problem of deriving a test suffix, which raises tester confidence in the configuration of the system, reached after a test derived by a specification coverage criterion, is formalized and solved in the CEFSM setting. The traditional mutant-based test derivation approach is extended with nondeterministic mutants to cover more faults. An experimental tool and case study are reported.

Fault coverage of tests based on finite state models

The finite state models have been used extensively in protocol conformance testing, as well as in software and hardware testing. It is well known that exhaustive testing is often impractical since it may require the execution of a huge number of test cases. In practice, testing is a tradeoff between increased confidence in the correctness of the implementation under test and constraints on the amount of time and effort that can be spent in testing. Therefore, the fault coverage, or adequacy of the test suite, becomes a very important issue. In this paper, we analyze various techniques used to evaluate fault coverage based on finite state models in the form of finite state machines (FSM) or labeled transition systems (LTS). We also point out certain issues which need further study.

Fault coverage analysis in respect to an FSM specification

It is shown in this paper that the problem of deciding if a test suite generated from a finite state machine provides complete fault coverage can be converted into the problem of minimizing the test tree representing the test suite. A fault coverage analysis procedure, capable of deciding if a given test suite provides complete fault coverage in respect to a given FSM specification, is then developed. The core of this procedure is a state minimization procedure developed specifically for the class of FSMs whose graphic representations are trees. The fault coverage analysis procedure can cope with partially specified FSM specifications which need not be reduced and faults that increase the number of states up to a chosen upper bound. Two necessary and one sufficient conditions, which in some cases may simplify the fault coverage analysis, are also presented.<<ETX>>

Checking Experiments with Labeled Transition Systems for Trace Equivalence

We apply the state identification techniques for testing communication sysxadtems which are modeled labeled by transition systems (LTSs). The conforxadmance requirements of specifications are represented as the trace equivalence relation and derived tests have finite behavior and provide well-defined fault coverage. We redefine in the realm of LTSs the notions of state identification that were originally defined in the realm of input/output finite state machines (FSMs). Then we present the corresponding test generation methods and disxadcuss their fault coverage.

A structural analysis approach to the evaluation of fault coverage for protocol conformance testing

In this paper, we propose a structural analysis approach to the evaluation of fault coverage of protocol conformance testing based on the finite state machine model. The attractiveness of this approach is its low computational complexity. It allows us to calculate the fault coverage of a given test suite by directly analyzing the test suite against the specification machine. Therefore, it avoids the generation and execution of mutants. The approach has been implemented and a number of experiments has been carried out. Some of the experimental results are summarized in this paper to show the accuracy of this approach compared with the mutation analysis technique.