Alexander S. Kossatchev

UniTesK Test Suite Architecture

The article presents the main components of the test suite architecture underlying UniTesK test development technology, an automated specification based test development technology for use in industrial testing of general-purpose software. The architecture presented contains such elements as automatically generated oracles, components to monitor formally defined test coverage criteria, and test scenario specifications for test sequence generation with the help of an automata based testing mechanism. This work stems from the ISP RAS results of academic research and 7-years experience in industrial application of formal testing techniques [1].

KVEST: Automated Generation of Test Suites from Formal Specifications

KVEST - Kernel VErification and Specification Technology - is based on automated test generation from formal specifications in the RAISE specification language. The technology was developed under contract with Nortel Networks. As of 1999, the methodology and toolset have been applied in three industrial project dealing with verification of large-scale telecommunication software. The first project, the Kernel Verification project, gives its name to the methodology and the toolset as a whole. Results of this project are available from the Formal Methods Europe Application database [13]. It is one of the biggest formal method application presented in the database. This paper provides a brief description of the approach, comparison to related works, and statistics on completed projects.

Survey of compiler testing methods

Compilers are used for creating executable modules for programs written in high-level languages; therefore, the presence of errors in a compiler is a serious danger for the quality of the software developed with the use of this compiler. As in the case of any other software, testing is one of the most important methods of quality control and error detection in compilers. The survey is devoted to methods for generating, running, and checking the quality of compiler test suites, which are based on formal specifications of the programming language syntax and semantics.Compilers are used for creating executable modules for programs written in high-level languages; therefore, the presence of errors in a compiler is a serious danger for the quality of the software developed with the use of this compiler. As in the case of any other software, testing is one of the most important methods of quality control and error detection in compilers. The survey is devoted to methods for generating, running, and checking the quality of compiler test suites, which are based on formal specifications of the programming language syntax and semantics.

The UniTesK Approach to Designing Test Suites

Principles of the UniTesK test development technology based on the use of formal models of target software are presented. This technology was developed by the RedVerst group in the Institute for System Programming, Russian Academy of Sciences (ISPRAS) [1], which obtained rich experience in testing and verification of complex commercial software.

Irredundant Algorithms for Traversing Directed Graphs: The Nondeterministic Case

Problems of testing program systems modeled by deterministic finite automata are considered. The necessary (and, sometimes, sufficient) component of such testing is a traversal of the graph of the automaton state transitions. The main attention is given to the so-called irredundant traversal algorithms (algorithms for traversing unknown graphs, or on-line algorithms), which do not require an a priori knowledge of the total graph structure.

Integration of Functional and Timed Testing of Real-Time and Concurrent Systems

The article presents an approach to model based testing of complex systems based on a generalization of finite state machines (FSM) and input output state machines (IOSM). The approach presented is used in the context of UniTesK specification based test development method. The results of its practical applications are also discussed. Practical experience demonstrates the applicability of the approach for model based testing of protocol implementations, distributed and concurrent systems, and real-time systems. This work stems from ISPRAS results of academic research and industrial application of formal techniques in verification and testing [1].

Application of finite automatons for program testing

The application of the finite automaton theory to the problem of program testing is discussed. The problem is reduced to testing a finite automaton. Testing of automatons using their state graphs, factor graphs, testing using factor graphs, and methods for factor graphs construction are discussed.

Test Generation for Compilers and Other Formal Text Processors

A concept of automated test suites generation for testing compilers and other formal text processors is suggested. An approach based on the generation of tests from models is used.

Formal Conformance Testing of Systems with Refused Inputs and Forbidden Actions

The article introduces an extension of the well-known conformance relation ioco on labeled transition systems (LTS) with refused inputs and forbidden actions. This extension helps to apply the usual formal testing theory based on LTS models to incompletely specified systems, which are often met in practice. Another topic concerned in the article is compositional conformance. More precisely, we try to define a completion operation that turns any LTS into input-enabled one having the same set of ioco-conforming implementations. Such a completion enforces preservation of ioco conformance by parallel composition operation on LTSes.

Formalization of test experiments

Formal methods for testing conformance of the system under examination to its specification are examined. The operational interaction semantics is specified by a special testing machine that formally determines the testing capabilities. A set of theoretically powerful and practically important capabilities is distinguished that can be reduced to the observation of external actions and refusals (the absence of external actions). The novelties are as follows. (1) Parameterization of the semantics by the families of observable and not observable refusals, which makes it possible to take into account various constraints on the (correct) interactions. (2) Destruction as a forbidden action, which is possible but should not be performed in the case of a correct interaction. (3) Modeling of the divergence by the Δ-action, which also should be avoided in the case of a correct interaction. On the basis of this semantics, the concept of safe testing, the implementation safety hypothesis, and the safe conformance relation are proposed. The safe conformance relation corresponds to the principle of independent observations: a behavior of an implementation is correct or incorrect independently of its other possible behaviors. For a more narrow class of interactions, another version of the semantics based on the ready traces may be used along with the corresponding conformance relation. Some propositions concerning the relationships between the conformance relations under various semantics are formulated. The completion transformation that solves the problem of the conformance relation reflexivity and a monotone transformation that solves the monotonicity problem (preservation of the conformance under composition) are defined.