Omar S. Gómez

Replications types in experimental disciplines

Experiment replication is a key component of the scientific paradigm. The purpose of replication is to verify previously observed findings. Although some Software Engineering (SE) experiments have been replicated, yet, there is still disagreement about how replications should be run in our field. With the aim of gaining a better understanding of how replications are carried out, this paper examines different replication types in other scientific disciplines. We believe that by analysing the replication types proposed in other disciplines it is possible to clarify some of the question marks still hanging over experimental SE replication.

Understanding replication of experiments in software engineering: A classification

Abstract Context Replication plays an important role in experimental disciplines. There are still many uncertainties about how to proceed with replications of SE experiments. Should replicators reuse the baseline experiment materials? How much liaison should there be among the original and replicating experimenters, if any? What elements of the experimental configuration can be changed for the experiment to be considered a replication rather than a new experiment? Objective To improve our understanding of SE experiment replication, in this work we propose a classification which is intend to provide experimenters with guidance about what types of replication they can perform. Method The research approach followed is structured according to the following activities: (1) a literature review of experiment replication in SE and in other disciplines, (2) identification of typical elements that compose an experimental configuration, (3) identification of different replications purposes and (4) development of a classification of experiment replications for SE. Results We propose a classification of replications which provides experimenters in SE with guidance about what changes can they make in a replication and, based on these, what verification purposes such a replication can serve. The proposed classification helped to accommodate opposing views within a broader framework, it is capable of accounting for less similar replications to more similar ones regarding the baseline experiment. Conclusion The aim of replication is to verify results, but different types of replication serve special verification purposes and afford different degrees of change. Each replication type helps to discover particular experimental conditions that might influence the results. The proposed classification can be used to identify changes in a replication and, based on these, understand the level of verification.

15 Years of Software Regression Testing Techniques — A Survey

Software regression testing techniques verify previous functionalities each time software modifications occur or new characteristics are added. With the aim of gaining a better understanding of this subject, in this work we present a survey of software regression testing techniques applied in the last 15 years; taking into account its application domain, kind of metrics they use, its application strategies and the phase of the software development process where they are applied. From an outcome of 460 papers, a set of 25 papers describing the use of 31 software testing regression techniques were identified. Results of this survey suggest that at the time of applying a regression testing techniques, metrics like cost and fault detection efficiency are the most relevant. Most of the techniques were assessed with instrumented programs (experimental cases) under academic settings. Conversely, we observe a minimum set of software regression techniques applied in industrial settings, mainly, under corrective and maintenance approaches. Finally, we observe a trend using some regression techniques under agile approaches.

An Empirical Study on the Impact of an IDE Tool Support in the Pair and Solo Programming

The adoption of Agile software development approaches has been widespread. One well-known Agile approach is extreme programming, which encompasses twelve practices of which pair programming is one of them. Although various aspects of pair programming have been studied, we have not found, under a traditional setting of pair programming, studies that examine the impact of using a tool support, such as an integrated development environment (IDE) or a simple text editor. In an attempt to obtain a better understanding of the impact of using an IDE in this field, we present the results of a controlled experiment that expose the influence on quality, measured as the number of defects injected per hour, and cost, measured as the time necessary to complete a programming assignment, of pair and solo programming with and without the use of an IDE. For quality, our findings suggest that the use of an IDE results in significantly higher defect injection rates (for both pairs and solos) when the programming assignment is not very complicated. Nevertheless, defect injection rates seem to decrease when pairs work on more complicated programming assignments irrespective of the tool support that they use. For cost, the programming assignment significantly affects the time needed to complete the assignment. In relation to the programming type, pairs and solos performed in a similar way with regards to quality and cost.

Regression Testing of Database Applications Under an Incremental Software Development Setting

Software regression testing verifies previous features on a software product when it is modified or new features are added to it. Because of the nature of regression testing it is a costly process. Different approaches have been proposed to reduce the costs of this activity, among which are: minimization, prioritization, and selection of test cases. Recently, soft computing techniques, such as data mining, machine learning, and others have been used to make regression testing more efficient and effective. Currently, in different contexts, to a greater or lesser extent, software products have access to databases (DBs). Given this situation, it is necessary to consider regression testing also for software products such as information systems that are usually integrated with or connected to DBs. In this paper, we present a selection regression testing approach that utilizes a combination of unsupervised clustering with random values, unit tests, and the DB schema to determine the test cases related to modifications or new features added to software products connected to DBs. Our proposed approach is empirically evaluated with two database software applications in a production context. Effectiveness metrics, such as test suite reduction, fault detection capability, recall, precision, and the F-measure are examined. Our results suggest that the proposed approach is enough effective with the resulting clusters of test cases.

An approach for regression testing of database applications in incremental development settings

In an iterative and incremental development environment software regression testing plays an important role; it helps to ensure the reliability in the building process of a software product. The optimization of a regression test depends on the size of the test suite to be executed. Regression testing helps to verify existing modifications (fixing bugs) or verify new features added to a software product. When regression testing is applied to database applications it is necessary to consider aspects of the database along with the product code in order to guarantee a proper verification of the product. In this paper we present an approach for conducting regression tests on database applications under an iterative and incremental development setting.

Assessment of software defect detection efficiency and cost through an intelligent collaborative virtual environment

The use of collaborative virtual environments facilitates communication, coordination and cooperation within a group of people. In the field of information and communications technology (ICT), collaborative virtual environments have been developed to support the learning of programming. In this work, we report an empirical study (controlled experiment) which assess the efficiency and cost of detecting defects in instrumented programs with and without the support of an intelligent collaborative virtual environment (ICVE). The ICVE was used as part of a programming course at undergraduate level. The results suggest an equivalent efficiency in defect detection for both types of participants, those who used the ICVE (3.76 observed defects per hour) and those who worked in a traditional manner (4.07 observed defects per hour). With regard to cost, it was observed that participants who used the ICVE completed the activities within a slightly shorter time frame (49.25 minutes) than those who worked without the environment (52.90 minutes); however, the difference was not significant. It was observed that with regard to efficiency and cost, for this type of task (the detection of defects), working virtually through an ICVE versus working ad hoc (in a tradition manner, at the same time and in the same place) yields equivalent results. When collaborative work cannot be done in a traditional way, the use an ICVE is an alternative approach equally effective.

Paradigma Experimental en la Ingeniería de Software – Caso Programación en Pareja

Resumen—Contexto: La Ingenieria de Software (IS) es una disciplina compleja donde a dia de hoy el resultado de la aplicacion de alguna tecnologia para apoyar el desarrollo de software es impredecible. De manera similar a otras disciplinas ingenieriles, la IS requiere de construccion de modelos, experimentacion y aprendizaje. La experimentacion es un recurso valioso que permite evaluar de manera objetiva las distintas tecnologias disponibles para desarrollar software. Objetivo: Con la finalidad de promover el paradigma experimental en la IS, en este articulo se aplica el paradigma experimental para estudiar algunos efectos de la programacion en pareja. Metodo: A traves de un diseno experimental cuadrado latino con dos factores de bloqueo, se emplearon 21 estudiantes del programa de licenciatura en IS de la Universidad Autonoma de Yucatan (UADY) para examinar la duracion y esfuerzo que conlleva la programacion en pareja. Los estudiantes se agruparon en 7 parejas y 7 individuos donde codificaron durante dos sesiones dos programas pequenos. Resultados: Los resultados del experimento sugieren una diferencia significativa (en un nivel α=0.1) en favor de la programacion en pareja con respecto a la duracion de codificacion de los ejercicios asignados (28% decremento en tiempo). Por el contrario, los resultados sugieren una diferencia significativa (en un nivel α=0.1) en favor de la programacion individual con respecto al esfuerzo (30% decremento en esfuerzo). Conclusiones: En este articulo se presenta el paradigma experimental aplicado a la IS. A traves del paradigma experimental es posible obtener observaciones de manera objetiva para acumular un cuerpo de conocimientos que pueda ser usado de manera confiable por profesionales en IS.