Maurizio Leotta

Capture-replay vs. programmable web testing: An empirical assessment during test case evolution

There are several approaches for automated functional web testing and the choice among them depends on a number of factors, including the tools used for web testing and the costs associated with their adoption. In this paper, we present an empirical cost/benefit analysis of two different categories of automated functional web testing approaches: (1) capture-replay web testing (in particular, using Selenium IDE); and, (2) programmable web testing (using Selenium WebDriver). On a set of six web applications, we evaluated the costs of applying these testing approaches both when developing the initial test suites from scratch and when the test suites are maintained, upon the release of a new software version. Results indicate that, on the one hand, the development of the test suites is more expensive in terms of time required (between 32% and 112%) when the programmable web testing approach is adopted, but on the other hand, test suite maintenance is less expensive when this approach is used (with a saving between 16% and 51%). We found that, in the majority of the cases, after a small number of releases (from one to three), the cumulative cost of programmable web testing becomes lower than the cost involved with capture-replay web testing and the cost saving gets amplified over the successive releases.

Visual vs. DOM-Based Web Locators: An Empirical Study

Automation in Web testing has been successfully supported by DOM-based tools that allow testers to program the interactions of their test cases with the Web application under test. More recently a new generation of visual tools has been proposed where a test case interacts with the Web application by recognising the images of the widgets that can be actioned upon and by asserting the expected visual appearance of the result.

Improving Test Suites Maintainability with the Page Object Pattern: An Industrial Case Study

The page object pattern is used in the context of web testing for abstracting the applications web pages in order to reduce the coupling between test cases and application under test. This paper reports on an industrial case study in a small Italian company (eXact learning solutions S.p.A.) investigating the potential benefits of adopting the page object pattern to improve the maintainability of Selenium WebDriver test cases. After a maintenance/evolution activity performed on the application under test, we compared two equivalent test suites, one built using the page object pattern and one without it. The results of our case study indicate a strong reduction in terms of time required (by a factor of about three) and number of modified LOCs (by a factor of about eight) to repair the test suite when the page object pattern is used.

“Precise is better than light” a document analysis study about quality of business process models

Business process modelling is often used in the initial phases of traditional software development to reduce faulty requirements and as starting point for building SOA based applications. Often, modellers produce business process models without following recognized guidelines and opt for “light” models where nodes representing the actions are simply decorated with natural language text. The potential consequence of this practice is that the quality of built business process models may be low. In this paper, we propose a method based on manual transformations to detect flaws in “light” business process models expressed as activity diagrams. Using that method we have executed a document analysis study with 14 business process models taken by books and websites. Preliminary results of this study show that almost all the analysed business process models contain errors and style violations (precisely 92% of them).

Who Knows/Uses What of the UML: A Personal Opinion Survey

UML is a comprehensive notation, offering a very large set of diagrams and constructs covering any possible modelling need. As consequence, on one hand, it is difficult and time consuming to master it, and on the other hand, people tend, naturally, to consider only a part of it. In practice, many UML diagrams/constructs seem scarcely used or even their existence is not known. By means of a study covering any possible source of information (e.g. UML books and tools), we started to assess which part of the UML is considered and used in practice. Here, we present some results about knowledge and usage of the UML diagrams by means of a personal opinion survey with 275 participants from both industry and academy, analysing also the influence of different factors: working environment (academia vs. industry), working role, seniority, education, and gender.

Reducing Web Test Cases Aging by Means of Robust XPath Locators

In the context of web regression testing, the main aging factor for a test suite is related to the continuous evolution of the underlying web application that makes the test cases broken. This rapid decay forces the quality experts to evolve the test ware. One of the major costs of test case evolution is due to the manual effort necessary to repair broken web page element locators. Locators are lines of source code identifying the web elements the test cases interact with. Web test cases rely heavily on locators, for instance to identify and fill the input portions of a web page (e.g., The form fields), to execute some computations (e.g., By locating and clicking on buttons) and to verify the correctness of the output (by locating the web page elements showing the results). In this paper we present ROBULA (ROBUst Locator Algorithm), a novel algorithm able to partially prevent and thus reduce the aging of web test cases by automatically generating robust XPath-based locators that are likely to work also when new releases of the web application are created. Preliminary results show that XPath locators produced by ROBULA are substantially more robust than absolute and relative locators, generated by state of the practice tools such as Fire Path. Fragility of the test suites is reduced on average by 56% for absolute locators and 41% for relative locators.

Robula+: an algorithm for generating robust XPath locators for web testing

Automated test scripts are used with success in many web development projects, so as to automatically verify key functionalities of the web application under test, reveal possible regressions and run a large number of tests in short time. However, the adoption of automated web testing brings advantages but also novel problems, among which the test code fragility problem. During the evolution of the web application, existing test code may easily break and testers have to correct it. In the context of automated DOM‐based web testing, one of the major costs for evolving the test code is the manual effort necessary to repair broken web page element locators – lines of source code identifying the web elements (e.g. form fields and buttons) to interact with.

Approaches and Tools for Automated End-to-End Web Testing

The importance of test automation in web engineering comes from the widespread use of web applications and the associated demand for code quality. Test automation is considered crucial for delivering the quality levels expected by users, since it can save a lot of time in testing and it helps developers to release web applications with fewer defects. The main advantage of test automation comes from fast, unattended execution of a set of tests after some changes have been made to a web application. Moreover, modern web applications adopt a multitier architecture where the implementation is scattered across different layers and run on different machines. For this reason, end-to-end testing techniques are required to test the overall behavior of web applications. In the last years, several approaches have been proposed for automated end-to-end web testing and the choice among them depends on a number of factors, including the tools used for web testing and the costs associated with their adoption. They can be classified using two main criteria: the first concerns how test cases are developed (ie, Capture-Replay and Programmable approaches), while, the second concerns how test cases localize the web elements to interact with (ie, Coordinates-based, DOM-based, and Visual approaches), that is what kind of locators are used for selecting the target GUI components. For developers and project managers it is not easy to select the most suitable automated end-to-end web testing approach for their needs among the existing ones. This chapter provides a comprehensive overview of the automated end-to-end web testing approaches and summarizes the findings of a long term research project aimed at empirically investigating their strengths and weaknesses.

Using Multi-Locators to Increase the Robustness of Web Test Cases

The main reason for the fragility of web test cases is the inability of web element locators to work correctly when the web page DOM evolves. Web elements locators are used in web test cases to identify all the GUI objects to operate upon and eventually to retrieve web page content that is compared against some oracle in order to decide whether the test case has passed or not. Hence, web element locators play an extremely important role in web testing and when a web element locator gets broken developers have to spend substantial time and effort to repair it. While algorithms exist to produce robust web element locators to be used in web test scripts, no algorithm is perfect and different algorithms are exposed to different fragilities when the software evolves. Based on such observation, we propose a new type of locator, named multi-locator, which selects the best locator among a candidate set of locators produced by different algorithms. Such selection is based on a voting procedure that assigns different voting weights to different locator generation algorithms. Experimental results obtained on six web applications, for which a subsequent release was available, show that the multi-locator is more robust than the single locators (about -30% of broken locators w.r.t. the most robust kind of single locator) and that the execution overhead required by the multiple queries done with different locators is negligible (2-3% at most).

Towards a lightweight model driven method for developing SOA systems using existing assets

Developing SOA based systems and migrating legacy systems to SOA are difficult and error prone tasks, where approaches, methods and tools play a fundamental role. For this reason, several proposals have been brought forward in literature to help SOA developers. This paper sketches a novel method for the development of systems based on services, i.e., adhering to the SOA paradigm, which follows the model driven paradigm. Our method is based on a “meet-in-the-middle” approach that allows the reuse of existing assets (e.g., legacy systems). The starting point of this method is a UML model representing the target business process and the final result is a detailed design model of the SOA system. The method, explained in this paper using a simple running example, has been applied successfully within an industrial project.