Antonio García-Domínguez

EUnit: a unit testing framework for model management tasks

Validating and transforming models are essential steps in model-driven engineering. These tasks are often implemented as operations in general purpose programming languages or task-specific model management languages. Just like other software artefacts, these tasks must be tested to reduce the risk of defects. Testing model management tasks requires testers to select and manage the relevant combinations of input models, tasks and expected outputs. This is complicated by the fact that many technologies may be used in the same system, each with their own integration challenges. In addition, advanced test oracles are required: tests may need to compare entire models or directory trees. To tackle these issues, we propose creating an integrated unit testing framework for model management operations. We have developed the EUnit unit testing framework to validate our approach. EUnit tests specify how models and tasks are to be combined, while staying decoupled from the specific technologies used.

Evolutionary mutation testing

Abstract Context Mutation testing is a testing technique that has been applied successfully to several programming languages. However, it is often regarded as computationally expensive, so several refinements have been proposed to reduce its cost. Moreover, WS-BPEL compositions are being widely adopted by developers, but present new challenges for testing, since they can take much longer to run than traditional programs of the same size. Therefore, it is interesting to reduce the number of mutants required. Objective We present Evolutionary Mutation Testing (EMT), a novel mutant reduction technique for finding mutants that help derive new test cases that improve the quality of the initial test suite. It uses evolutionary algorithms to reduce the number of mutants that are generated and executed with respect to the exhaustive execution of all possible mutants, keeping as many difficult to kill and potentially equivalent mutants (strong mutants) as possible in the reduced set. Method To evaluate EMT we have developed GAmera, a mutation testing system powered by a co-evolutive genetic algorithm. We have applied this system to three WS-BPEL compositions to estimate its effectiveness, comparing it with random selection. Results The results obtained experimentally show that EMT can select all strong mutants generating 15% less mutants than random selection in over 20% less time for complex compositions. When generating a percentage of all mutants, EMT finds on average more strong mutants than random selection. This has been confirmed to be statistically significant within a 99.9% confidence interval. Conclusions EMT has reduced for the three tested compositions the number of mutants required to select those which are useful to derive new test cases that improve the quality of the test suite. The directed search performed by EMT makes it more effective than random selection, especially as compositions become more complex and the search space widens.

GAmera: An Automatic Mutant Generation System for WS-BPEL Compositions

Business processes based in WS-BPEL compositions are rapidly becoming commonplace in recent years, so it is important to pay special attention to testing in this context. Mutation testing is a white-box testing technique that has been applied successfully to programs written in several languages. In order to apply it we need a system for mutant generation. This paper presents the first results of applying a new system, GAmera, for automatic mutant generation for WS-BPEL compositions. It is composed by three different elements: an analyzer, a mutant generator and a system that executes and evaluates the mutants. GAmera is based in genetic algorithms and attempts to minimize the number of generated mutants, independently of the number and type of mutation operators, without losing relevant information. It can also detect potentially equivalent mutants allowing to improve the quality of the test suite.

Eugenia: towards disciplined and automated development of GMF-based graphical model editors

EMF and GMF are powerful frameworks for implementing tool support for modelling languages in Eclipse. However, with power comes complexity, implementing a graphical editor for a modelling language using EMF and GMF requires developers to handcraft and maintain several detailed interconnected models through a loosely guided, labour-intensive, and error-prone process. We demonstrate how the application of metamodel annotation and model transformation techniques can help to manage the complexity of GMF and EMF and deliver significant productivity, quality, and maintainability benefits. We present Eugenia, an open-source tool that implements the proposed approach, illustrate its functionality with an example, evaluate it through an empirical study, and report on the community’s response to the tool.

Quality metrics for mutation testing with applications to WS-BPEL compositions

Mutation testing is a successful testing technique based on fault injection. However, it can be very costly, and several cost‐reduction techniques for reducing the number of mutants have been proposed in the literature. Cost reduction can be aided by an analysis of mutation operators, but this requires the definition of specialized metrics. Several metrics have been proposed before, although their effectiveness and relative merits are not easy to assess. A step ahead in the evaluation of mutation‐reduction techniques would be a better metric to determine objectively the quality of a set of mutants with respect to a given test suite. This work introduces such a metric, which is naturally extended to mutation operators and may be used to reduce the number of mutants, particularly of equivalent mutants. Finally, a firm mutation analysis tool for WS‐BPEL service compositions is presented, and experimental results obtained by comparing different metrics on several compositions are presented. Copyright

Analogies and Differences between Mutation Operators for WS-BPEL 2.0 and Other Languages

Applying mutation testing to a program written in a certain language requires that a set of mutation operators is defined for that language. The mutation operators need to adequately cover the features of that language in order to be effective. In this work, we evaluate qualitatively the operators defined for the Web Services Business Process Execution Language 2.0 (WS-BPEL) and study the differences and similarities between WS-BPEL and other languages. We review the existing operators for several structured and object-oriented general-purpose programming languages, and for several domain-specific languages. Results confirm that WS-BPEL is very different from other languages, as half of the mutation operators for this language are equivalent to those of other languages. Our study concludes that the set of WS-BPEL mutation operators can be improved.

Takuan: A Dynamic Invariant Generation System for WS-BPEL Compositions

Web Services composition languages, such as the OASIS WS-BPEL 2.0 standard, open a vast new field for programming in the large. But they also present a challenge for traditional white-box testing techniques, due to the inclusion of specific instructions for concurrency, fault and compensation handling, dynamic service discovery and invocation, etc. Dynamic likely invariant generation has been successfully used to assist white-box testing of traditional imperative programs. We have implemented Takuan, a dynamic invariant generation system which infers likely invariants from actual execution logs of a WS-BPEL composition. We describe our experience during its development and show our first results based on the well-known loan example. These promising results point out that, with some refinements, Takuan can become an interesting aid in WS-BPEL composition white-box testing.

Assessment of class mutation operators for C + + with the MuCPP mutation system

Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C + + , despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C + + programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method:MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C + + programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C + + . In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.

A Comparison of BPMN 2.0 with other Notations for Manufacturing Processes

In order to study their current practices and improve on them, manufacturing firms need to view their processes from several viewpoints at various abstraction levels. Several notations have been developed for this purpose, such as Value Stream Mappings or IDEF models. Recently, the BPMN 2.0 standard from the Object Management Group has been proposed for modeling business processes. A process organizes several activities into a single higher-level entity, which can be reused elsewhere in the organization. Its potential for standardizing business interactions is well-known, but there is little work on using BPMN 2.0 to model manufacturing processes. In this work some of the previous notations are outlined and BPMN 2.0 is positioned among them after discussing it in more depth. Some guidelines on using BPMN 2.0 for manufacturing are offered, and its advantages and disadvantages in comparison with the other notations are presented.

GAmera: a Tool for WS-BPEL composition testing using mutation analysis

This paper shows a novel tool, GAmera, the first mutant generation tool for testing Web Service compositions written in the WS-BPEL language. After several improvements and the development of a graphical interface, we consider GAmera to be a mature tool that implements an optimization technique to reduce the number of generated mutants without significant loss of testing effectiveness. A genetic algorithm is used for generating and selecting a subset of high-quality mutants. This selection reduces the computational cost of mutation testing. The subset of mutants generated with this tool allows the user to improve the quality of the initial test suite.