Alexander K. Petrenko

UniTesK Test Suite Architecture

The article presents the main components of the test suite architecture underlying UniTesK test development technology, an automated specification based test development technology for use in industrial testing of general-purpose software. The architecture presented contains such elements as automatically generated oracles, components to monitor formally defined test coverage criteria, and test scenario specifications for test sequence generation with the help of an automata based testing mechanism. This work stems from the ISP RAS results of academic research and 7-years experience in industrial application of formal testing techniques [1].

The UniTesK Approach to Designing Test Suites

Principles of the UniTesK test development technology based on the use of formal models of target software are presented. This technology was developed by the RedVerst group in the Institute for System Programming, Russian Academy of Sciences (ISPRAS) [1], which obtained rich experience in testing and verification of complex commercial software.

Integration of Functional and Timed Testing of Real-Time and Concurrent Systems

The article presents an approach to model based testing of complex systems based on a generalization of finite state machines (FSM) and input output state machines (IOSM). The approach presented is used in the context of UniTesK specification based test development method. The results of its practical applications are also discussed. Practical experience demonstrates the applicability of the approach for model based testing of protocol implementations, distributed and concurrent systems, and real-time systems. This work stems from ISPRAS results of academic research and industrial application of formal techniques in verification and testing [1].

Specification Based Testing: Towards Practice

Specification based testing facilities are gradually becoming software production aids. The paper shortly considers the current state of the art, original ISPRAS/RedVerst experience, and outlines the ways for further research and testing tool development. Both conceptual and technical problems of novel specification based testing technologies introduction are considered.

Test Generation for Compilers and Other Formal Text Processors

A concept of automated test suites generation for testing compilers and other formal text processors is suggested. An approach based on the generation of tests from models is used.

Automatic Test Generation for Model-Based Code Generators

This paper presents a novel testing approach for model-based design tools, termed GraphOTK, and applies it to the optimizing component of a code generator for Simulink and Stateflow models.

Practical approach to specification and conformance testing of distributed network applications

Standardization of infrastructure and services in distributed applications and frameworks requires ground methodological base. Design by Contract approach looks very promising as a candidate. It helps to obtain component-wise design, to separate concerns between developers accurately, and makes development of high quality complex systems a manageable process. Unfortunately, in its classic form it can hardly be applied to distributed network applications because of lack of adequate means to describe nondeterministic asynchronous events. We extend Design by Contract with capabilities to describe callbacks and asynchronous communication between components. The resulting method was used to specify distributed applications and to develop conformance test suites for them in automated manner. Specifications are developed in an extension of C language that makes them clear and useful for industrial developers and decreases greatly test construction effort. Practical results of numerous successful applications of the method are described. More information on the applications of the method can be found at the site of RedVerst group of ISP RAS [1].

The use of contract specifications for representing requirements and for functional testing of hardware models

Contract specifications in the form of pre-and postconditions are widely used in software engineering for formal description of interfaces of software components. On the one hand, such specifications are convenient for the developers since they can easily be attached to the system architecture. On the other hand, test oracles verifying conformance of the behavior of the target system to the specifications can automatically be generated from them. In the paper, it is suggested to use contract specifications for representing requirements and for functional testing of hardware models developed in languages such as VHDL, Verilog, SystemC, System Verilog, etc. An approach to specification of such systems is proposed and compared with the existing methods of hardware specification. An experience of its practical use is described. The approach is based on the UniTESK testing technology developed at the Institute for System Programming.

Configurable toolset for static verification of operating systems kernel modules

An operating system (OS) kernel is a critical software regarding to reliability and efficiency. Quality of modern OS kernels is already high enough. However, this is not the case for kernel modules, like, for example, device drivers that, due to various reasons, have a significantly lower level of quality. One of the most critical and widespread bugs in kernel modules are violations of rules for correct usage of a kernel API. One can find all such violations in modules or can prove their correctness using static verification tools that need contract specifications describing obligations of a kernel and modules relative to each other. This paper considers present methods and toolsets for static verification of kernel modules for different OSs. A new method for static verification of Linux kernel modules is proposed. This method allows one to configure the verification process at all its stages. It is shown how it can be adapted for checking kernel components of other OSs. An architecture of a configurable toolset for static verification of Linux kernel modules that implements the proposed method is described, and results of its practical application are presented. Directions for further development of the proposed method are discussed in conclusion.

Coverage-driven Automated Compiler Test Suite Generation

Abstract The paper presents a novel approach to automated compiler test suite generation based on the source level specification. Several coverage criteria are introduced. The application of the proposed methodology to testing the realistic programming language is discussed.