Yves Le Traon

Improving test suites for efficient fault localization

The need for testing-for-diagnosis strategies has been identified for a long time, but the explicit link from testing to diagnosis (fault localization) is rare. Analyzing the type of information needed for efficient fault localization, we identify the attribute (called Dynamic Basic Block) that restricts the accuracy of a diagnosis algorithm. Based on this attribute, a test-for-diagnosis criterion is proposed and validated through rigorous case studies: it shows that a test suite can be improved to reach a high level of diagnosis accuracy. So, the dilemma between a reduced testing effort (with as few test cases as possible) and the diagnosis accuracy (that needs as much test cases as possible to get more information) is partly solved by selecting test cases that are dedicated to diagnosis.

Mutation analysis testing for model transformations

In MDE, model transformations should be efficiently tested so that it may be used and reused safely. Mutation analysis is an efficient technique to evaluate the quality of test data, and has been extensively studied both for procedural and object-oriented languages. In this paper, we study how it can be adapted to model oriented programming. Since no model transformation language has been widely accepted today, we propose generic fault models that are related to the model transformation process. First, we identify abstract operations that constitute this process: model navigation, models elements filtering, output model creation and input model modification. Then, we propose a set of specific mutation operators which are directly inspired from these operations. We believe that these operators are meaningful since a large part of the errors in a transformation are due to the manipulation of complex models regardless of the concrete implementation language.

A Requirement-Based Approach to Test Product Families

Use-cases have been identified as good inputs to generate test cases and oracles at requirement level. To have an automated generation, information is missing from use cases, such as the exact inputs of the system, and the sequential constraints between the use cases. The contribution of this paper is then two-fold. First we propose a contract language for PF functional requirements expressed as parameterized use cases; this language supports the specification of variant parts in the requirements. Then we provide a method, a formal model and a prototype tool to automatically generate both functional and robustness test cases specific to a product from the PF requirements. We study the efficiency of the generated test cases on a case study.

Reusable MDA components: a testing-for-trust approach

Making model transformations trustable is an obvious target for model-driven development since they impact on the design process reliability. Ideally, model transformations should be designed and tested so that they may be used and reused safely as MDA components. We present a method for building trustable MDA components. We first define the notion of MDA component as composed of its specification, one implementation and a set of associated test cases. The testing-for-trust approach checks the consistency between these three facets using the mutation analysis. It points out the lack of efficiency of the tests and the lack of precision of the specification. The mutation analysis thus gives a rate that evaluates: the level of consistency between the components facets and the level of trust we can have in a component. Relying on this estimation of the component trustability, developers can consciously trade reliability for resources.

System Testing of Product Lines: From Requirements to Test Cases

Product line processes still lack support for testing end-product functions by taking advantage of the specific features of a product line (commonality and variabilities). Indeed, classical testing approaches cannot be directly applied on each product since, due to the potentially huge number of products, the testing task would be far too long and expensive. There is thus a need for testing methods, adapted to the product line context, that allow reducing the testing cost. The approach we present is based on the automation of the generation of application system tests, for any chosen product, from the system requirements of a product line. These PL requirements are modeled using enhanced UML use cases which are the basis for the test generation. Product-specific test objectives, test scenarios, and test cases are successively tional variation points at requirement level to automatically generate the behaviors specific to any chosen product. With such a strategy, the designer may apply any method to produce the domain models of the product line and then instantiate a given product: the test cases check that the expected functionalities have been correctly implemented. The approach is adaptive and provides automated test generation for a new product as well as guided test generation support to validate the evolution of a given product.

Trustable components: yet another mutation-based approach

This paper presents the use of mutation analysis as the main qualification technique for: -estimating and automatically enhancing a test set (using genetic algorithms), -qualifying and improving a components contracts (that is the specification facet) - measuring the impact of contractable robust components on global system robustness and reliability. The methodology is based on an integrated design and test approach for OO software components. It is dedicated to design-by-contract, where the specification is systematically derived into executable assertions called contracts (invariant properties, pre/postconditions of methods). The testing-for-trust approach, using the mutation analysis, checks the consistency between specification, implementation and tests. It points out the tests lack of efficiency but also the lack of precision of the contracts. The feasibility of components validation by mutation analysis and its usefulness for test generation are studied as well as the robustness of trustable and self-testable components into an infected environment.