Willibald Krenn

MoMut::UML Model-Based Mutation Testing for UML

Model-based mutation testing (MBMT) is a promising testing methodology that relies on a model of the system under test (SUT) to create test cases. Hence, MBMT is a so-called black-box testing approach. It also is fault based, as it creates test cases that are guaranteed to reveal certain faults: after inserting a fault into the model of the SUT, it looks for a test case revealing this fault. This turns MBMT into one of the most powerful and versatile test case generation approaches available as its tests are able to demonstrate the absence of certain faults, can achieve both, control-flow and data-flow coverage of model elements, and also may include information about the behaviour in the failure case. The latter becomes handy whenever the test execution framework is bound in the number of observations it can make and - as a consequence - has to restrict them. However, this versatility comes at a price: MBMT is computationally expensive. The tool MoMuT::UML (https://www.momut.org) is the result of a multi-year research effort to bring MBMT from the academic drawing board to industrial use. In this paper we present the current stable version, share the lessons learnt when applying two generations of MoMuT::UML in an industrial setting, and give an outlook on the upcoming, third,generation.

Efficient Mutation Killers in Action

This paper presents the techniques and results of a novel model-based test case generation approach that automatically derives test cases from UML state machines. Mutation testing is applied on the modeling level to generate test cases. We present the test case generation approach, discuss the tool chain, and present the properties of the generated test cases. The main contribution of this paper is an empirical study of a car alarm system where different strategies for killing mutants are compared. We present detailed figures on the effectiveness of the test case generation technique. Although UML serves as an input language, all techniques are grounded on solid foundations: we give UML state transition diagrams a formal semantics by mapping them to Backs action systems.

UML in action: a two-layered interpretation for testing

This paper presents a novel model-based test case generation approach that automatically derives test cases from UML state machines. UML is given a two-layered formal semantics by (1) mapping UML class diagrams and state charts to Backs Action Systems, (2) by interpreting these action systems as labeled transition systems. The first semantics provides a formal framework to capture the object-oriented machinery: classes, objects, inheritance, transitions, time-outs, signals, nested and parallel regions. The second mapping represents the testers view on the interface in terms of input and output actions. Tretmans input-output conformance relation (ioco) forms the basis of our fault models. Mutation analysis on the models is used to generate test cases. A car alarm system serves as a running example

Model-based mutation testing of hybrid systems

This paper presents a novel model-based testing approach developed in the MOGENTES project. The aim is to test embedded systems controlling a continuous environment, i.e., hybrid systems. We present our two key abstractions against which we systematically test for conformance. (1) Classical action systems are used to model the discrete controller behavior. (2) Qualitative differential equations are used to model the evolutions of the environment. The latter is based on a technique from the domain of Artificial Intelligence called qualitative reasoning. Mutation testing on these models is used to generate effective test cases. A test case generator has been developed that searches for all test cases that would kill a mutant. The mutant models represent our fault models. The generated test cases are then executed on the implementation in order to systematically exclude the possibility that a mutant has been implemented.

Qualitative Action Systems

An extension to action systems is presented facilitating the modeling of continuous behavior in the discrete domain. The original action system formalism has been developed by Back et al. in order to describe parallel and distributed computations of discrete systems, i.e. systems with discrete state space and discrete control. In order to cope with hybrid systems, i.e. systems with continuous evolution and discrete control, two extensions have been proposed: hybrid action systems and continuous action systems . Both use differential equations (relations) to describe continuous evolution. Our version of action systems takes an alternative approach by adding a level of abstraction: continuous behavior is modeled by Qualitative Differential Equations that are the preferred choice when it comes to specifying abstract and possibly non-deterministic requirements of continuous behavior. Because their solutions are transition systems, all evolutions in our qualitative action systems are discrete. Based on hybrid action systems, we develop a new theory of qualitative action systems and discuss how we have applied such models in the context of automated test-case generation for hybrid systems.

Test Case Generation by Contract Mutation in Spec

Mutation testing is a well known fault-based testing technique that is normally used to assess the quality of a test suite. In this paper we use the mutation operation to derive test cases that demonstrate the absence of certain faults in an implementation: In difference to conventional mutation testing, which mutates program code, we mutate program contracts and generate test-input data that is able to distinguish the mutated contract from the original one. We show how existing development tools can be used as a foundation for the presented methodology: In particular we rely on the counter-example generation capabilities of the Spec#/Boogie/Z3 system.

Knowledge Extraction from C-Code

In this paper we present first ideas for extracting knowledge from C source code of control programs. The extracted knowledge is intended to be used in our smart control engine which takes a rule set and decides which rules to use based on the internal and environmental conditions. The extraction of rules is based on the control-flow graph of the supplied C program: Basically, our method extracts rules that correspond to paths to given high-level function calls. The advantage of this method is to get a first knowledge-base from available source code which makes using a smart control engine more applicable for industry. We use an industrial control program as example within the paper in order to justify the usefulness of our approach.

Intelligent, Fault Tolerant Control for Autonomous Systems

We present a methodology for intelligent control of an autonomous and resource constrained embedded system. Geared towards mastering permanent and transient faults by dynamic reconfiguration, our approach uses rules for describing device functionality, valid environmental interactions, and goals the system has to reach. Besides rules, we use functions that characterize a goals target activity profile. The target activity profile controls the frequency our system uses to reach the corresponding goal. In the paper we discuss a first implementation of the given methodology, and introduce useful extensions. In order to underline the feasibility and effectiveness of the presented control system, we present a case study that has been carried out on a prototype system.

Rule-Set Extraction from C-Code

We present an approach for extracting knowledge from C source code ofcontrol programs. The extracted knowledge is intended to be used in our smartcontrol engine which takes a rule set and decides which rules to use based onthe internal and environmental conditions. The extraction of rules is based onthe control-flow graph of the supplied C program: Basically, our methodextracts rules that correspond to paths to given high-level function calls. Theadvantage of this method is to get a first knowledge-base from availablesource code which makes using a smart control engine more applicable forindustry. We use an industrial control program as example within the paper inorder to justify the usefulness of our approach.

Intelligent, Fault Adaptive Control of Autonomous Systems

We present a methodology for intelligent control of an autonomous and resource constrained embedded system. Geared towards mastering permanent and transient faults by dynamic reconfiguration, our approach uses rules for describing device functionality, valid environmental interactions, and goals the system has to reach. Besides rules, we use functions that characterize a goal’s activity. These functions control the frequency our system uses to reach the goal. In this chapter we present the system model, discuss properties of the rule selection mechanism, and show results gained from running the approach on an embedded device.