José García-Fanjul

Testing in Service Oriented Architectures with dynamic binding: A mapping study

Context: Service Oriented Architectures (SOA) have emerged as a new paradigm to develop interoperable and highly dynamic applications. Objective: This paper aims to identify the state of the art in the research on testing in Service Oriented Architectures with dynamic binding. Method: A mapping study has been performed employing both manual and automatic search in journals, conference/workshop proceedings and electronic databases. Results: A total of 33 studies have been reviewed in order to extract relevant information regarding a previously defined set of research questions. The detection of faults and the decision making based on the information gathered from the tests have been identified as the main objectives of these studies. To achieve these goals, monitoring and test case generation are the most proposed techniques testing both functional and non-functional properties. Furthermore, different stakeholders have been identified as participants in the tests, which are performed in specific points in time during the life cycle of the services. Finally, it has been observed that a relevant group of studies have not validated their approach yet. Conclusions: Although we have only found 33 studies that address the testing of SOA where the discovery and binding of the services are performed at runtime, this number can be considered significant due to the specific nature of the reviewed topic. The results of this study have contributed to provide a body of knowledge that allows identifying current gaps in improving the quality of the dynamic binding in SOA using testing approaches.

Generation of Conformance Test Suites for Compositions of Web Services Using Model Checking

Testing compositions of Web services is complex, due to their distributed nature and asynchronous behaviour. However, research in this field is scarce. We propose a new testing method for compositions of Web services. A formal verification tool (the SPIN model checker) is used to automatically generate test suites for compositions specified in an industry standard language: BPEL. Adequacy criteria is employed to define a systematic procedure to select the test cases. Preliminary results have been obtained using a transition coverage criterion

A First Approach to Test Case Generation for BPEL Compositions of Web Services Using Scatter Search

A challenging part of Software Testing entails the generation of test cases, which cost can be reduced by means of the use of techniques for automating this task. In this paper we present an approach based on the metaheuristic technique Scatter Search for the automatic test case generation of the BPEL business process. A transition coverage criterion is used as adequacy criterion.

Coverage-Based Testing for Service Level Agreements

Service level agreements (SLAs) are typically used to specify rules regarding the consumption of services that are agreed between the providers of the service-based applications (SBAs) and their consumers. An SLA includes a list of terms that contain the guarantees that must be fulfilled during the provisioning and consumption of the services. Since the violation of such guarantees may lead to the application of potential penalties, it is important to assure that the SBA behaves as expected. In this paper, we propose a proactive approach to test SLA-aware SBAs by means of identifying test requirements, which represent situations that are relevant to be tested. To address this issue, we define a four-valued logic that allows evaluating both the individual guarantee terms and their logical relationships. Grounded in this logic, we devise a test criterion based on the modified condition decision coverage (MCDC) in order to obtain a cost-effective set of test requirements from the structure of the SLA. Furthermore by analyzing the syntax and semantics of the agreement, we define specific rules to avoid non-feasible test requirements. The whole approach has been automated and applied over an eHealth case study.

A Proactive Approach to Test Service Level Agreements

Service Oriented Architectures (SOA) have emerged as a promising solution to develop interoperable and highly dynamic applications. In the domain of SOA, Service Level Agreements (SLAs) are used to specify the stipulated terms between the service provider and the consumer. Due to the unique features of this paradigm such as SLA management, testing SOA presents new challenges to researchers because traditional testing techniques need to be suitably adapted. It is important for both stakeholders to develop proactive approaches that allow the detection of problems before they may later derive in SLA violations. This paper tackles the problem of testing the terms of the SLA, specified in WS-Agreement language, by introducing a stepwise method based on a well-known partition testing technique. The approach is illustrated over a case study.

Identifying Test Requirements by Analyzing SLA Guarantee Terms

Service Level Agreements (SLAs) are used to specify the negotiated conditions between the provider and the consumer of services. In this paper we present a stepwise method to identify and categorize a set of test requirements that represent the potential situations that can be exercised regarding the specification of each isolated guarantee term of an SLA. This identification is addressed by means of devising a set of coverage levels that allow grading the thoroughness of the tests. The utilization of these test requirements would focus on twofold objectives: (1) the generation of a test suite that allows exercising the situations described in the test requirements and (2) the support for the derivation of a monitoring plan that checks the compliance of these requirements at runtime. The approach is illustrated over an eHealth case study.

A Partition-Based Approach for XPath Testing

The XML language is becoming the preferred means of data interchange and representation in web based applications. Usually, XML data is stored in XML repositories, which can be accessed efficiently using the standard XPath as query language. However, the specific techniques for testing these queries often ignore the functional testing. This work addresses this problem by introducing a technique based on the category-partition method for the systematic design of test input data for an XPath query. The method permits the automatic identification of categories and choices in the XPath and XML Schema implementations and the construction of constraints in order to obtain complete and valid test cases. The technique is illustrated over a practical example.

Automatic test case generation for WS-Agreements using combinatorial testing

In the scope of the applications developed under the service-based paradigm, Service Level Agreements (SLAs) are a standard mechanism used to flexibly specify the Quality of Service (QoS) that must be delivered. These agreements contain the conditions negotiated between the service provider and consumers as well as the potential penalties derived from the violation of such conditions. In this context, it is important to assure that the service based application (SBA) behaves as expected in order to avoid potential consequences like penalties or dissatisfaction between the stakeholders that have negotiated and signed the SLA. In this article we address the testing of SLAs specified using the WS-Agreement standard by means of applying testing techniques such as the Classification Tree Method and Combinatorial Testing to generate test cases. From the content of the individual terms of the SLA, we identify situations that need to be tested. We also obtain a set of constraints based on the SLA specification and the behavior of the SBA in order to guarantee the testability of the test cases. Furthermore, we define three different coverage strategies with the aim at grading the intensity of the tests. Finally, we have developed a tool named SLACT (SLA Combinatorial Testing) in order to automate the process and we have applied the whole approach to an eHealth case study. We propose a method to test SLAs using standard testing techniques.Constraints that avoid generating non feasible test cases are automatically obtained.Different coverage strategies are defined to grade the thoroughness of the tests.A tool named SLACT that automates the generation of the test cases is implemented.Application of the approach to a real and critical eHealth service-based scenario.

Effort measurement in student software engineering projects

Teaching software engineering by means of student involvement in the team development of a product is the most effective way to teach the main issues of software engineering. Some of its difficulties are those of coordinating their work, measuring the time spent by the students (both in individual work and in meetings) and making sure that meeting time will not be excessive. Starting in the academic year 1998/1999, we assessed, improved and documented the development process for the student projects and found that measurement is one of the outstanding issues to be considered. Each week, the students report the time spent on the different project activities. We present and analyze the measurement results for our 16 student teams (each one with around 6 students). It is interesting to note that the time spent in meetings is usually too long, ranging from 46% in the requirements analysis phase to 21% in coding, mainly due to problems of coordination. Results from previous years are analyzed and presented to the following years students for feedback. In the present year (2000), we have decreased the amount of time spent by the student doing group work, and improved the effectiveness and coordination of the teams.

Systematic Design of Test Case for XPath Queries using Partition-based Techniques

The XML language is becoming the preferred means of data interchange and representation in web based applications. Usually, XML data is stored in XML repositories, which can be accessed efficiently using the standard XPath as query language. However, the specific techniques for testing these queries often ignore the functional testing. This work addresses this problem by introducing a technique based on the category-partition method for the systematic design of test input data for an XPath query. The method permits the automatic identification of categories and choices in the XPath and XML Schema implementations and the construction of constraints in order to obtain complete and valid test cases. The technique is illustrated over a practical example.