Sandra Camargo Pinto Ferraz Fabbri

Replicating software engineering experiments: addressing the tacit knowledge problem

Recently the awareness of the importance of replicating studies has been growing in the empirical software engineering community. The results of any one study cannot simply be extrapolated to all environments because there are many uncontrollable sources of variation between different environments. In our work, we have reasoned that the availability of laboratory packages for experiments can encourage better replications and complementary studies. However, even with effectively specified laboratory packages, transfer of experimental know-how can still be difficult. A cooperation between Brazilian and American researchers addressing effective running of replications was formed in 1999. One of the specific issues being addressed is the problem of transferring tacit knowledge. We discuss what we learned about the tacit knowledge transfer problem and the evolution of laboratory packages in the description of a replication performed in Brazil using a PBR (Perspective Based Reading) laboratory package; also how further issues are addressed.

Mutation testing applied to validate specifications based on statecharts

The establishment of a low-cost, effective testing and validation strategy has been pursued by many researchers at the program level as well as at the specification level. The application of mutation testing for validating specifications based on statecharts is proposed. A mutation operator set for statecharts, one of the crucial points for effectively applying mutation testing is defined; in this scope these operators can be taken as a fault model. We also provide strategies to abstract the statechart components according to different statechart features that may comprise the testing and validation activity aims, providing in this way mechanisms for the establishment of an incremental, hierarchical, mutation-based testing strategy. Implementation and functional aspects of PROTEUM/ST, a tool under development are also presented.

Knowledge-Sharing Issues in Experimental Software Engineering

Recently the awareness of the importance of replicating studies has been growing in the empirical software engineering community. The results of any one study cannot simply be extrapolated to all environments because there are many uncontrollable sources of variation between different environments.In our work, we have reasoned that the availability of laboratory packages for experiments can encourage better replications and complementary studies. However, even with effectively specified laboratory packages, transfer of experimental know-how can still be difficult. In this paper, we discuss the collaboration structures we have been using in the Readers’ Project, a bilateral project supported by the Brazilian and American national science agencies that is investigating replications and transfer of experimental know-how issues. In particular, we discuss how these structures map to the Nonaka–Takeuchi knowledge sharing model, a well-known paradigm used in the knowledge management literature. We describe an instantiation of the Nonaka–Takeuchi Model for software engineering experimentation, establishing a framework for discussing knowledge sharing issues related to experimental software engineering. We use two replications to illustrate some of the knowledge sharing issues we have faced and discuss the mechanisms we are using to tackle those issues in Readers’ Project.

Mutation testing applied to Estelle specifications

Many researchers have pursued the establishment of a low-cost, effective testing and validation strategy at the program level as well as at the specification level. Mutation Testing is an error-based approach, originally introduced for program testing, that provides testers a systematic way to evaluate how good a given test set is. Some studies have also investigated its use to generate test sets. In this article, the application of Mutation Testing for validating Estelle specifications is proposed. A mutant operator set for Estelle—one of the crucial points for effectively applying Mutation Testing—is defined, addressing: the validation of the behavior of the modules, the communication among modules and the architecture of the specification. In this scope, these operators can be taken as a fault model. Considering this context, a strategy for validating Estelle-based specification is proposed and exemplified using the Alternating-bit protocol.

Mutation Testing Applied to Validate Specifications Based on Petri Nets

Testing is one of the fundamental software development life cycle activities. Considering Reactive Systems such as: metro control, patient hospital monitoring and communication protocols, the testing activity becomes more relevant as errors in these systems can promote severe economical and social losses. The objective of this work is to evaluate the adequacy of applying the Mutation Analysis criterion to validate Petri Net based specifications. A set of mutation operators for Petri Nets, a key point for using Mutation Analysis, as well as the results of applying manually these operators to a Petri Net modeling a level 3 protocol are presented. We also examine the ideas of constrained and randomly selected mutation in this context.

Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness

This paper describes a replication conducted to compare the effectiveness of inspectors using Perspective Based Reading (PBR) to the effectiveness of inspectors using a checklist. The goal of this replication was to better understand the complementary aspects of the PBR perspectives. To this end, a brief discussion of the original study is provided as well as a more detailed description of the replication. A detailed statistical analysis is then provided along with analysis of the PBR perspectives.For the individual PBR perspectives, we saw an interesting dichotomy: In the original study there was little overlap among the sets of defects found by each of the three perspectives, while in the replication two of the three perspectives found similar sets of defects on one of the two documents used in the study. Interestingly this document was the only case where the users of PBR were not more effective than the users of a checklist. This result leads to a new hypothesis that the complementary aspect of the PBR perspectives is the characteristic that provides the benefit over other defect detection techniques.

Proteum/FSM: a tool to support finite state machine validation based on mutation testing

The quality of the VV&T-Verification, Validation and Testing-activity is extremely relevant to the software development process. Testing techniques and criteria have been investigated in the context of VV&T of reactive systems specifications, providing mechanisms to the VV&T activity quality assessment. The establishment of a low-cost, effective testing and validation strategy and the development of supporting tools have been pursued by many researchers for coding and specification as well. This paper discusses the main architectural and operational aspects of a tool, named Proteum/FSM, that supports the application of mutation testing for validating reactive systems specifications based on finite state machines (FSM). Further improvements and research issues are briefly discussed.

Proteum: a family of tools to support specification and program testing based on mutation

The qualification of the VV&T- Verification, Validation and Testing - activity is extremely relevant to the software development process. The establishment of a low-cost, effective testing and validation strategy and the development of supporting tools have been pursued by many researchers. This presentation discusses the main architectural and operational aspects of a family of tools that support specification and program testing based on mutation. The testing of C programs is supported by Proteum/IM 2.0, at the unit and at the integration level as well. Proteum is an acronym for PROgram Testing Using Mutants. At the specification level the application of mutation testing for validating Reactive Systems (RS) specifications based on Finite State Machines (FSM), Statecharts and Petri Nets is support by Proteum/RS.

TUCCA: improving the effectiveness of use case construction and requirement analysis

Most of the delivered software products are delivered with defects usually introduce in the communications and transformations among different levels of abstraction along the software development phases. Defects remain in the artifacts even though high quality, rigorous design, inspections and testing activities are carried out during software development. Therefore, every software development activity should be systematic and rigorous and whenever possible should embody the aim of finding defects. This paper presents results of a feasibility study that was carried out aiming at evaluating TUCCA

Externalising tacit knowledge of the systematic review process

Technique for use case model construction and construction-based requirements document analysis. TUCCA is composed of two reading techniques: AGRT - actor goal reading technique whose objective is to determine the actors of the system and their goals and UCRT - use case reading technique whose objective is to determine the use case model. These techniques support the construction of the use case model and also embody a review of the requirements document. In this paper we show that TUCCA is effective in use case construction and defect detection; we detailed the results of TUCCASs contribution for finding defects, comparing this technique with Checklist and PBR-User (perspective based reading - user view). The results motivate to investigate the relationship of inspection techniques embodying construction steps with the construction techniques embodying inspection steps, such as TUCCA, aiming at reducing the overall cost and increasing the effectiveness of the software development.