Pascale Le Gall

Symbolic execution techniques for test purpose definition

We propose an approach to test whether a system conforms to its specification given in terms of an Input/Output Symbolic Transition System (IOSTS). IOSTSs use data types to enrich transitions with data-based messages and guards depending on state variables. We use symbolic execution techniques both to extract IOSTS behaviours to be tested in the role of test purposes and to ground an algorithm of test case generation. Thus, contrarily to some already existing approaches, our test purposes are directly expressed as symbolic execution paths of the specification. They are finite symbolic subtrees of its symbolic execution. Finally, we give coverage criteria and demonstrate our approach on a running example.

Testing data types implementations from algebraic specifications

Algebraic specifications of data types provide a natural basis for testing data types implementations. In this framework, the conformance relation is based on the satisfaction of axioms. This makes it possible to state formally the fundamental concepts of testing: exhaustive test set, testability hypotheses, oracle. Various criteria for selecting finite test sets have been proposed. They depend on the form of the axioms, and on the possibilities of observation of the implementation under test. This last point is related to the well-known oracle problem. As the main interest of algebraic specifications is data type abstraction, testing a concrete implementation raises the issue of the gap between the abstract description and the concrete representation. The observational semantics of algebraic specifications bring solutions on the basis of the so-called observable contexts. After a description of testing methods based on algebraic specifications, the chapter gives a brief presentation of some tools and case studies, and presents some applications to other formal methods involving data types.

Formal Specifications and Test: Correctness and Oracle

This article presents a new formal approach to testing. In the field of dynamic testing, as soon as a program fails for a test set, it is flagged incorrect. The remaining question is: how far can a successful program be considered as correct? We give a definition of program correctness with respect to a specification which is adequate to dynamic testing. Similarly to the field of abstract implementation, the idea is that in order to declare a program as correct, it suffices that its behavior fulfills the specification requirements. An intermediate semantic level between the program and the specification, called the oracle framework, is introduced in order to interpret observable results obtained from dynamic experiments on the program. This allows to give algebraic semantics (i.e. a set of models) to the program, compatible with the program behavior. Program correctness is then defined by some adequacy criterion between the specification semantics and the program semantics. We point out that while for some specifications, there exist exhaustive test sets (the success of which means program correctness), for some other specifications, there only exist “complete” (but not exhaustive) test sets. Of course, all the programs rejected by a complete test set are incorrect but unfortunately, there still exist successful incorrect programs. We also explain how the test set selection can be formalized within our approach.

Symbolic modeling of genetic regulatory networks

Understanding the functioning of genetic regulatory networks supposes a modeling of biological processes in order to simulate behaviors and to reason on the model. Unfortunately, the modeling task is confronted to incomplete knowledge about the system. To deal with this problem we propose a methodology that uses the qualitative approach developed by Thomas. A symbolic transition system can represent the set of all possible models in a concise and symbolic way. We introduce a new method based on model-checking techniques and symbolic execution to extract constraints on parameters leading to dynamics coherent with known behaviors. Our method allows us to efficiently respond to two kinds of questions: is there any model coherent with a certain hypothetic behavior? Are there behaviors common to all selected models? The first question is illustrated with the example of the mucus production in Pseudomonas aeruginosa while the second one is illustrated with the example of immunity control in bacteriophage lambda.

Symbolic model based testing for component oriented systems

In a component oriented approach, components are designed, developed and validated in order to be widely used. However one cannot always foresee which specific uses will be made of components depending on the system they will constitute. In this paper we propose an approach to test each component of a system by extracting accurate behaviours using information given by the system specification. System specifications are defined as input/output symbolic transition systems structured by a communication operator (synchronized product) and an encapsulation operator (hiding communication channels). By projecting symbolic execution of a system on its components, we derive unitary symbolic behaviours to be used as test purposes at the component level. In practice, those behaviours can be seen as typical behaviours of the component in the context of the system. We will illustrate on an example that those behaviours could not have been extracted by reasoning uniquely at the component level.

Testing from algebraic specifications: test data set selection by unfolding axioms

This paper deals with test data set selection from algebraic specifications. Test data sets are generated from selection criteria which are usually defined to cover specification axioms. The unfolding selection criterion consists in covering the input domain of an operation using case analysis. The unfolding procedure can be iterated in order to split input domains of operations into finer subdomains. In this paper we propose to extend an unfolding procedure previously developed in [5, 19] that could only be performed on very low level, i.e. executable specifications. On the contrary, our new unfolding procedure can be applied to any positive conditional specification. We show that our unfolding procedure is sound (no test is added) and complete (no test is lost) with respect to the starting reference test data set.

Off-Line Test Case Generation for Timed Symbolic Model-Based Conformance Testing

Model-based conformance testing of reactive systems consists in taking benefit from the model for mechanizing both test data generation and verdicts computation. On-line test case generation allows one to apply adaptive on-the-fly analyzes to generate the next inputs to be sent and to decide if observed outputs meet intended behaviors. On the other hand, in off-line approaches, test suites are pre-computed from the model and stored under a format that can be later performed on test-beds. In this paper, we propose a two-passes off-line approach where: for the submission part, a test suite is a simple timed sequence of numerical input data and waiting delays, and then, the timed sequence of output data is post-processed on the model to deliver a verdict. As our models are Timed Output Input Symbolic Transition Systems, our off-line algorithms involve symbolic execution and constraint solving techniques.

Test Purpose Concretization through Symbolic Action Refinement

In a Model Driven Design process, model refinement methodologies allow one to denote system behaviors at several levels of abstraction. In the frame of a model-based testing process, benefits can be taken from such refinement processes by extracting test cases from the different intermediate models. As a consequence, test cases extracted from abstract models often have to be concretized in order to be executable on the System Under Test. In order to properly define a test concretization process, a notion of conformance relating SUTs and abstract models has to be defined. We define such a relation for models described in a symbolic manner as so-called IOSTSs (Input Output Symbolic Transition Systems) and for a particular kind of refinement, namely action refinement, which consists in replacing communication actions of abstract models with sets of sequences of more concrete communication actions. Our relation is defined as an extension of the ioco-conformance relation which relates SUTs and models whose communication actions are defined at the same level of abstraction. Finally we show from an example how a test purpose resulting from an abstract IOSTS-model can be concretized in a test purpose defined at the abstraction level of the SUT.

Symbolic execution techniques for refinement testing

We propose an approach to test whether an abstract specification is refined or not by a more concrete one. The specifications are input/output symbolic transition systems (IOSTS). The refinement relation requires that all traces of the abstract system are also traces of the concrete system, up to some signature inclusion. Our work takes inspiration from the conformance testing area. Symbolic execution techniques allow us to select traces of the abstract system and to submit them on the concrete specification. Each trace execution leads to a verdict Fail, Pass or Warning. The verdict Pass is provided with a formula which has to be verified by the values only manipulated at the level of the concrete specification in order to ensure the refinement relation. The verdict Warning reports that the concrete specification has not been sufficiently explored to give a reliable verdict. This is thus a partial verification process, related to the quality of the set of selected traces and of the exploration of the concrete specification. Our approach has been implemented and is demonstrated on a simple example.

Test selection criteria for quantifier-free first-order specifications

This paper deals with test case selection from axiomatic specifications whose axioms are quantifier-free first-order formulae. Test cases are modeled as ground formulae and any specification has an exhaustive test data set whose successful submission means correctness, provided that the software under verification can be modeled as a first-order structure over the same signature. As it has already been done for positive conditional equational specifications, we derive test cases from selection criteria based on axiom coverage. Our selection criteria allows us to select test cases by iteratively unfolding an initial target test purpose, given as a formula. The initial reference test set is iteratively split into successive subsets. Each subset of test cases is defined by constraints which are increasingly introduced by the unfolding procedure to ensure an appropriate matching between the current test purpose under unfolding and specification axioms. Our unfolding procedure is sound (no test is added) and complete (no test is lost) with respect to the starting test purpose. It is exemplified on a simple example.