Marcelo Medeiros Eler

Built-In Structural Testing of Web Services

Testing Service Oriented Architecture applications is a challenging task due to the high dynamism, the low coupling and the low testability of services. Web services, a popular implementation of services, are usually provided as black box and using testing techniques based on implementation is limited. This paper presents an approach to support the use of the structural testing technique on web service testing. The approach improves web service testability by developing web services with built-in structural testing capabilities. Testers can run test cases against such web services and obtain a coverage analysis on structural testing criteria. A set of metadata provided with the testable web service helps testers to evaluate the coverage reached and the quality of their test cases. An implementation of the approach is presented using a service called JaBUTiWS that performs instrumentation and coverage analysis of Java web services. We also present a usage scenario of the approach.

JaBUTiService: A Web Service for Structural Testing of Java Programs

Web services are an emerging Service-Oriented Architecture technology to integrate applications using open standards based on XML. Software Engineering tools integration is a promising area since companies adopt different software processes and need different tools on each activity. Software engineers could take advantage of software engineering tools available as web services and create their own workflow for integrating the required tools. In this paper, we propose the development of testing tools designed as web services and discuss the pros and cons of this idea. We developed a web service for structural testing of Java programs called JaBUTiService, which is based on the stand-alone tool JaBUTi. We also present an usage example of this service with the support of a desktop front-end and pre prepared scripts. A set of 62 classes of the library Apache-Commons-BeanUtils was used for this test and the results are discussed.

More testable service compositions by test metadata

In previous work we proposed testable services as a solution to provide third-party testers with structural coverage information after a test session, yet without revealing their internal details. However, service testers, e.g., integrators that use testable services into their compositions, do not have enough information to improve their test set when they get a low coverage measure because they do not know which test requirements have not been covered. This paper proposes an approach in which testable services are provided along with test metadata that will help their testers to get a higher coverage. We show the approach on a case study of a real system that uses orchestrations and testable services.

Quantifying the Characteristics of Java Programs That May Influence Symbolic Execution from a Test Data Generation Perspective

Testing plays a key role in assessing the quality of a software product. During testing, a program is run in hopes of finding faults. As exhaustive testing is seldom possible, specific testing criteria have been proposed to help testers to devise test cases that cover the most relevant faulty scenarios. Manually creating test cases that satisfy these criteria is time consuming, error prone, and unwieldy. Symbolic execution has been used as an effective way of automatically generating test data that meets those criteria. Although this technique has been used for over three decades, several challenges remain, such as path explosion, precision of floating-point data, constraints with complex expressions, and dependency of external libraries. In this paper, we explore a sample of 100 open source Java projects in order to analyze characteristics that are relevant to generate test data using symbolic execution. The results provide valuable insight into how researchers and practitioners can tailor symbolic execution techniques and tools to better suit the needs of different Java applications.

An empirical study to quantify the characteristics of Java programs that may influence symbolic execution from a unit testing perspective

We analyze two benchmarks: SF100 (100 projects) and R47 (47 projects).We show the distributions of issues on generating test data using symbolic execution.We show that few methods have potential for path explosion.We show that dependency and exception-dependent paths are very frequent.We show that the constraints complexity is mainly affected by object types. In software testing, a program is executed in hopes of revealing faults. Over the years, specific testing criteria have been proposed to help testers to devise test cases that cover the most relevant faulty scenarios. Symbolic execution has been used as an effective way of automatically generating test data that meet those criteria. Although this technique has been used for over three decades, several challenges remain and there is a lack of research on how often they appear in real-world applications. In this paper, we analyzed two samples of open source Java projects in order to understand the characteristics that may hinder the generation of unit test data using symbolic execution. The first sample, named SF100, is a third party corpus of classes obtained from 100 projects hosted by SourceForge. The second sample, called R47, is a set of 47 well-known and mature projects we selected from different repositories. Both samples are compared with respect to four dimensions that influence symbolic execution: path explosion, constraint complexity, dependency, and exception-dependent paths. The results provide valuable insight into how researchers and practitioners can tailor symbolic execution techniques and tools to better suit the needs of different Java applications.

Aspects as components

An adaptation of the UML Component method to design crosscutting components is briefly presented. Such components are allowed to crosscut only the public interface of base (convencional) components. The design and implementation of crosscutting components using the language JAsCO is discussed.

Evaluation studies of software testing research in Brazil and in the world: A survey of two premier software engineering conferences

This paper reports on a historical perspective of the evaluation studies present in software testing research published in the Brazilian Symposium on Software Engineering (SBES) in comparison to the International Conference on Software Engineering (ICSE). The survey characterizes the software testing-related papers published in the 25-year history of SBES, investigates the types of evaluation presented in these publications, and how the rate of evaluations has evolved over the years. A similar analysis within the same period is made for ICSE, allowing for a comparison between the national and international scenario. Results show that the rate of papers that present evaluation studies in SBES has significantly increased over the years. However, among the papers that described some kind of evaluation, only around 20% performed more rigorous evaluations (i.e. case studies, quasi experiments, or controlled experiments). Such percentage is low when compared to ICSE, which presented 40% of papers with more rigorous evaluations within the same period. Nevertheless, we noticed that both venues still lack the publication of research reporting controlled experiments: only a single paper in each conference presented this type of evaluation.

Evaluation Studies of Software Testing Research in the Brazilian Symposium on Software Engineering

Experimentation is the traditional way of identifying cause-effect relationships in scientific research. Lately, there has been an increasing understanding that experiments and other forms of evaluation should be more thoroughly disseminated among computer science and, in particular, Software Engineering (SE) researchers. Software testing (ST) is an important SE topic, where experiments are particularly valuable: since cost constraints and high effectiveness goals are common within this subfield, cost/benefit characteristics have to be adequately evaluated to deem a specific approach useful or not. This paper reports on a historical perspective of the evaluation studies present in ST research published in the Brazilian Symposium on Software Engineering (SBES). The survey characterizes the software testing-related papers published in the 24-year history of SBES, investigates the types of evaluation presented in these studies - when they were presented at all - and whether the number of evaluations has increased over the years. Additionally, the survey also brings a preliminary characterization of the Brazilian software testing community that adopts SBES as a vehicle to publish its research. Results show that the number of papers that present evaluation studies have significantly increased over the years. However, on the downside, amongst the papers that described some kind of evaluation, only 20% performed more rigorous evaluations (e.g. experiments or case studies in the industrial context), whereas 80% described exploratory, less rigorous case studies.

An analysis of automated tests for mobile Android applications

Mobile computing has become ubiquitous, thus the amount and complexity of mobile applications have challenged software engineering practices. To overcome the challenges brought by mobile applications, the adoption of repeatable, systematic and mainly automated tests have been researched. In this paper, we look into open source projects in hopes of identifying how automated tests are applied to mobile applications developed for the Android platform. We analyzed the automated tests to identify the frameworks adopted, the relation between the production and the testing code, and how the following challenges have been dealt with: connectivity, rich GUIs, limited resources, sensors, and multiple configurations.

Dynamic product line for Business Process Management

Purpose – The purpose of this paper is to present the proposal of a Product Line (PL)-based approach for Business Process Management (BPM) projects that cover the entire BPM lifecycle and proposes integrating it with dynamic techniques still not used together. Design/methodology/approach – The authors carried out this work using the design science research methodology. The authors assessed the proposed approach using a classification procedure created through a series of specific attributes, which enables a comparison of the proposed integrated approach with related works selected from a systematic literature review. Findings – The comparative assessment has shown that the proposed approach presents the most comprehensive solution than any other similar one suggested for the same purpose, mainly in terms of the coverage of the entire BPM lifecycle and dynamic techniques. Research limitations/implications – Due to the high-level conceptual nature of the proposed approach, the authors could not evaluate it ...