Eduardo Aranha

An Estimation Model for Test Execution Effort

Testing is an important activity to ensure software quality. Big organizations can have several development teams with their products being tested by overloaded test teams. In such situations, test team managers must be able to properly plan their schedules and resources. Also, estimates for the required test execution effort can be an additional criterion for test selection, since effort may be restrictive in practice. Nevertheless, this information is usually not available for test cases never executed before. This paper proposes an estimation model for test execution effort based on the test specifications. For that, we define and validate a measure of size and execution complexity of test cases. This measure is obtained from test specifications written in a controlled natural language. We evaluated the model through an empirical study on the mobile application domain, which results suggested an accuracy improvement when compared with estimations based only on historical test productivity.

Search based constrained test case selection using execution effort

Software testing is essential to guarantee high quality products. However, it is a very expensive activity, particularly when manually performed. One way to cut down costs is by reducing the input test suites, which are usually large in order to fully satisfy the test goals. Yet, since large test suites usually contain redundancies (i.e., two or more test cases (TC) covering the same requirement/piece of code), it is possible to reduce them in order to respect time/people constraints without severely compromising coverage. In this light, we formulated the TC selection problem as a constrained search based optimization task, using requirements coverage as the fitness function to be maximized (quality of the resultant suite), and the execution effort (time) of the selected TCs as a constraint in the search process. Our work is based on the Particle Swarm Optimization (PSO) algorithm, which is simple and efficient when compared to other widespread search techniques. Despite that, besides our previous works, we did not find any other proposals using PSO for TC selection, neither we found solutions treating this task as a constrained optimization problem. We implemented a Binary Constrained PSO (BCPSO) for functional TC selection, and two hybrid algorithms integrating BCPSO with local search mechanisms, in order to refine the solutions provided by BCPSO. These algorithms were evaluated using two different real-world test suites of functional TCs related to the mobile devices domain. In the performed experiments, the BCPSO obtained promising results for the optimization tasks considered. Also, the hybrid algorithms obtained statistically better results than the individual search techniques.

A Model-Driven Approach to Specifying and Monitoring Controlled Experiments in Software Engineering

This paper presents a process-oriented model-driven approach that supports the conduction of controlled experiments in software engineering. The approach consists of: (i) a domain specific language (DSL) for process specification and statistical design of controlled experiments; (ii) model-driven transformations that allow workflow models generations specific to each experiment participant and according to the experiment statistical design; and (iii) a workflow execution environment that allows the monitoring of participant activities in the experiment, besides gathering participants feedback from the experiment. The paper also presents the results of an exploratory study that analyzes the feasibility of the approach and the expressivity of the DSLs in the modeling of a non-trivial software engineering experiment.

Ensino de programação utilizando jogos digitais: uma revisão sistemática da literatura

Iniciativas internacionais renomadas indicam a importância do ensino de programacao desde o ensino basico para formacao nos profissionais de TI, mas existem ainda diversos desafios para a implantacao desta ideia. Nesse contexto, este artigo tem como objetivo apresentar uma revisao sistematica dos artigos referentes a utilizacao de jogos digitais para o ensino de programacao, publicados nos ultimos cinco anos (2008- 2012) em tres principais eventos nacionais na area de informatica na educacao e tambem em tres relevantes revistas nacionais na area. Os resultados obtidos mostraram que as pesquisas focam principalmente no ensino de programacao para o ensino medio e a maioria esta sendo realizada por pesquisadores localizados nas em tres regioes brasileira.

Test case prioritization based on data reuse an experimental study

The order in which tests are executed can significantly impact the total test execution time. In this paper, we evaluate two test prioritization techniques (manual and automatic) in the context of mobile phone testing. The manual technique produces test sequences created by test experts, while the automatic one generates sequences mechanically based on the permutation of the tests. Both techniques take into account a data reuse: the more the data is reused among tests, the faster the sequence is executed. In order to evaluate the benefits of these two techniques, we carried out an experiment with 8 testers and 2 test suites arranged in a 2×2 Latin square design replicated four times. The automatic technique reduced approximately 25% of the data generation time and 13.5% of the execution time. The automatic technique is clearly better than the manual one with respect to the generation of sequences. Our experiment showed that the automatic technique also generates sequences whose execution is faster than those created manually by test experts.

Test Effort Estimation Models Based on Test Specifications

In competitive markets, test teams should be able to estimate the required effort to perform their test activities on the schedule. In this paper, we present a test execution and a test automation effort estimation model and their use for test selection. All these models are based on the test specifications written in a controlled natural language. This characteristic may allow us to have better estimation accuracy, even for tests never executed before. The evaluation of these models and some additional benefits to industry are also discussed in the paper.

Estimating Manual Test Execution Effort and Capacity Based On Execution Points

Abstract In some domains, such as the mobile application domain, functional tests may not be feasible to automate due to limited test automation technology (multimedia support, etc.) or high cost to port automated tests for different phone models and operating systems. In this context, test teams should be able to estimate the effort required to manually execute tests on schedule. Although several effort estimation approaches have been proposed over the years, none of them are appropriate for estimating manual test execution effort. The main reason for that is the lack of size measures for test projects. In this paper, we present execution points, a measure for the size and execution complexity of tests that is calculated based on test specifications. We also show how to estimate manual test execution effort and capacity based on this measure. Finally, results from an empirical study suggested the viability of our proposed measure.

Using process simulation to assess the test design effort reduction of a model-based testing approach

Several researches are being performed to address current softwaredevelopment problems in industry. However, quantifying the benefits of usingthese solutions in the practice is also a challenge. Usually, pilot studies are run toget evidences about these benefits. Nevertheless, it may be difficult to run thesestudies due to the required changes in the development process and the lack ofavailable resources. In this work, we address the problem of assessing the testdesign effort reduction provided by TaRGeT, a tool that supports a Model-Based Testing (MBT) approach. We used process simulation to provide evidence ofthis effort reduction in a multi-site industry. For that, we modeled, simulated andcompared the use of the current and the model-based test design processes. Weidentified interesting advantages of using process simulation, such as its reducedcosts and the possibility to analyze multiple scenarios. We also show somedrawbacks of this approach, such as the difficult to create models close to the realityand the lack of processes simulation and comparison support.

Online game-based programming learning for high school students — A case study

[Context] The inclusion of programming concepts in high school education has been emerging, since applying computing in high school education is a way to motivate and prepare students into the field of technology. [Objective] This work evaluates the teaching programming games for high school students based on different forms of distance education (online video lessons, broadcasted live or recorded, and tutorials in PDF). [Method] A case study was performed in the second half of 2014 with 30 high school students enrolled in an online game programming challenge. The students had access to the different formats of online lessons and they were monitored by tutors in a online learning environment. [Results] Over 75% of the students performed and sent all the activities requested in the study, suggesting that all investigated online formats are effective. However, 85% of students indicated their preference to use recorded video classes to learn programming. Finally, 100% of the students said that they would participate again in such challenge, showing the attractiveness of the game development. [Conclusions] We considered that online formats (specially recorded video) can be used to promote the development of games to introduce programming knowledge in high school education. However, the lessons learned (difficults found and improvement suggestions) should be taken in consideration to implement online solutions in large scale.

Unveiling developers contributions behind code commits: an exploratory study

The process and activities of software development are very dynamic and diverse. For instance, source code has to be written, tested and revised, e-mails have to be sent, bugs have to be communicated, managed and fixed. As a consequence, the contributions of the developers are very diversified. This paper describes an empirical study whose goal was to assess and compare the contributions of the developers through software repository mining. Two medium-sized projects -- an open source and a commercial project -- were analyzed. Overall, 17,490 commits and 10,308 bugs reports were analyzed. In the first part of our study, we have classified the developers based on their contribution to the software repository in three groups -- core, active and peripheral developers. After that, we have collected a series of metrics -- code contribution, buggy commits and resolution of priority bugs -- for all the developers of the investigated projects. Finally, we have analyzed the collected metrics for the different developer groups. Our study findings show significant differences in the contribution provided by the developers groups considering the open-source and the commercial project.