Márcio Eduardo Delamaro

Interface Mutation: an approach for integration testing

The need for test adequacy criteria is widely recognized. Several criteria have been proposed for the assessment of adequacy of tests at the unit level. However, there remains a lack of criteria for the assessment of the adequacy of tests generated during integration testing. We present a mutation based interprocedural criterion, named Interface Mutation (IM), suitable for use during integration testing. A case study to evaluate the proposed criterion is reported. In the study, the UNIX sort utility was seeded with errors and Interface Mutation evaluated by measuring the cost of its application and its error revealing effectiveness. Alternative IM criteria using different sets of Interface Mutation operators were also evaluated. While comparing the error revealing effectiveness of these Interface Mutation-based test sets with same size randomly generated test sets, we observed that in most cases Interface Mutation based test sets are superior. The results suggest that Interface Mutation offers a viable test adequacy criteria for use at the integration level.

Mutation analysis testing for finite state machines

Proposes the application of the mutation analysis criterion in the context of specifications based on finite state machines. The main concepts of finite state machines and mutation analysis are briefly introduced. An experiment is reported which manually applies mutation analysis to a finite state machine modeling a Class 0 ISO transport protocol specification, using two test-sequence generator criteria-the W method and the TT (transition tours) method. The results obtained are presented, and evidences are given that the use of mutation analysis is effective in this context. Finally, the lines of evolution of the work presented in this paper are briefly discussed.<<ETX>>

Establishing Theoretical Minimal Sets of Mutants

Mutation analysis generates tests that distinguish variations, or mutants, of an artifact from the original. Mutation analysis is widely considered to be a powerful approach to testing, and hence is often used to evaluate other test criteria in terms of mutation score, which is the fraction of mutants that are killed by a test set. But mutation analysis is also known to provide large numbers of redundant mutants, and these mutants can inflate the mutation score. While mutation approaches broadly characterized as reduced mutation try to eliminate redundant mutants, the literature lacks a theoretical result that articulates just how many mutants are needed in any given situation. Hence, there is, at present, no way to characterize the contribution of, for example, a particular approach to reduced mutation with respect to any theoretical minimal set of mutants. This papers contribution is to provide such a theoretical foundation for mutant set minimization. The central theoretical result of the paper shows how to minimize efficiently mutant sets with respect to a set of test cases. We evaluate our method with a widely-used benchmark.

Integration testing using interface mutation

A criterion for assessing the adequacy of test sets during integration testing is proposed. The criterion is based on a testing technique named Interface Mutation. The technique itself is designed to be scalable with the size of the software under test; the size being measured in the number of subsystems integrated. Using Interface Mutation it is possible to assess the adequacy of tests incrementally while integrating various subsystems. Also reported are results from a pilot experiment conducted to study the cost and error defection effectiveness of Interface Mutation.

Proteum/IM 2.0: An Integrated Mutation Testing Environment

Mutation testing has been used mostly at the unit level. To support its application few tools have been developed and used, mainly in the academic environment. Interface Mutation has been proposed aiming at applying mutation at the integration level. A tool named Proteum/IM was implemented to support such criterion. With the definition of the Interface Mutation criterion the tester has the possibility of applying mutation testing concepts throughout the software development. It seems mandatory to have a single, integrated environment that would support mutation-based unit and integration testing. Such environment, which provides facilities to investigate low-cost and incremental testing strategies, is the focus of this paper.

Mutation Testing Applied to Validate Specifications Based on Petri Nets

Testing is one of the fundamental software development life cycle activities. Considering Reactive Systems such as: metro control, patient hospital monitoring and communication protocols, the testing activity becomes more relevant as errors in these systems can promote severe economical and social losses. The objective of this work is to evaluate the adequacy of applying the Mutation Analysis criterion to validate Petri Net based specifications. A set of mutation operators for Petri Nets, a key point for using Mutation Analysis, as well as the results of applying manually these operators to a Petri Net modeling a level 3 protocol are presented. We also examine the ideas of constrained and randomly selected mutation in this context.

Proteum/FSM: a tool to support finite state machine validation based on mutation testing

The quality of the VV&T-Verification, Validation and Testing-activity is extremely relevant to the software development process. Testing techniques and criteria have been investigated in the context of VV&T of reactive systems specifications, providing mechanisms to the VV&T activity quality assessment. The establishment of a low-cost, effective testing and validation strategy and the development of supporting tools have been pursued by many researchers for coding and specification as well. This paper discusses the main architectural and operational aspects of a tool, named Proteum/FSM, that supports the application of mutation testing for validating reactive systems specifications based on finite state machines (FSM). Further improvements and research issues are briefly discussed.

A strategy to perform coverage testing of mobile applications

The development of wireless application has recently received more attention due to the increment in the number and in the power of mobile devices such as PDAs and cellular phones. Different methods and techniques have been developed to ease the design and development of applications for these kind of devices. Also, different languages have been proposed to provide support for such platform, such as J2ME and Brew. On the other hand, few attention has been given to testing activity in this scenario. Some works try to test the functional aspects of a given application, others try to perform load, usability and stress testing. In this article we present a strategy to support coverage testing for mobile device software in such a way that the applications can be tested not only on emulators, but also on their real target mobile devices with the aid of structural coverage assessment. We also present an environment which supports the proposed strategy. Such environment is implemented in a tool, named JaBUTi/ME. A simple case illustrating how JaBUTi/ME can be used is also presented.

Coverage testing of Java programs and components

Although software testing is a crucial activity in the software development process, only recently have more sound and consistent testing tools become available for testing Java programs and their components. These tools support mostly functional and control-flow based structural criteria. In this paper we explore control-flow and data-flow based coverage criteria to support the testing of Java programs and/or components. We also describe a testing tool, named JaBUTi, which can be used by both the component developer and the component user to test Java-based components and/or systems. To achieve this goal, the tool works at the bytecode level such that no source code is required during the testing activity. We illustrate these ideas and concepts with an example extracted from the literature.

Proteum: a family of tools to support specification and program testing based on mutation

The qualification of the VV&T- Verification, Validation and Testing - activity is extremely relevant to the software development process. The establishment of a low-cost, effective testing and validation strategy and the development of supporting tools have been pursued by many researchers. This presentation discusses the main architectural and operational aspects of a family of tools that support specification and program testing based on mutation. The testing of C programs is supported by Proteum/IM 2.0, at the unit and at the integration level as well. Proteum is an acronym for PROgram Testing Using Mutants. At the specification level the application of mutation testing for validating Reactive Systems (RS) specifications based on Finite State Machines (FSM), Statecharts and Petri Nets is support by Proteum/RS.