Nadia Alshahwan

Automated Session Data Repair for Web Application Regression Testing

This paper introduces an approach to web application regression testing, based upon repair of user session data. The approach is entirely automated. It consists of a white box examination of the structure of the changed web application to detect changes and a set of techniques to map these detected changes onto repair actions. The paper reports the results of experiments that explore both the performance and effectiveness of the approach. The effectiveness experiment uses an implementation of the repair algorithm applied to the online bookstore application over a series of 10 releases.

Coverage-Based Test Case Prioritisation: An Industrial Case Study

This paper presents an industrial case study of coverage-based prioritisation techniques on a real world system with real regression faults. The study evaluates four common and different test case prioritisation techniques and examines the effects of using various coverage criteria on the fault detection rates of the prioritised test suites. The results show that prioritisation techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates. The study also reveals that using modification information does not significantly enhance fault detection rates.

Automated testing for SQL injection vulnerabilities: an input mutation approach

Web services are increasingly adopted in various domains, from finance and e-government to social media. As they are built on top of the web technologies, they suffer also an unprecedented amount of attacks and exploitations like the Web. Among the attacks, those that target SQL injection vulnerabilities have consistently been top-ranked for the last years. Testing to detect such vulnerabilities before making web services public is crucial. We present in this paper an automated testing approach, namely μ4SQLi, and its underpinning set of mutation operators. μ4SQLi can produce effective inputs that lead to executable and harmful SQL statements. Executability is key as otherwise no injection vulnerability can be exploited. Our evaluation demonstrated that the approach is effective to detect SQL injection vulnerabilities and to produce inputs that bypass application firewalls, which is a common configuration in real world.

Coverage-based regression test case selection, minimization and prioritization: a case study on an industrial system

This paper presents a case study of coverage‐based regression testing techniques on a real world industrial system with real regression faults. The study evaluates four common prioritization techniques, a test selection technique, a test suite minimization technique and a hybrid approach that combines selection and minimization. The study also examines the effects of using various coverage criteria on the effectiveness of the studied approaches. The results show that prioritization techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates. The study also reveals that using modification information in prioritization techniques does not significantly enhance fault detection rates. The results show that test selection does not provide significant savings in execution cost (<2%), which might be attributed to the nature of the changes made to the system. Test suite minimization using finer grained coverage criteria could provide significant savings in execution cost (79.5%) while maintaining a fault detection capability level above 70%, thus representing a possible trade‐off. The hybrid technique did not provide a significant improvement over traditional minimization techniques. Copyright

Augmenting test suites effectiveness by increasing output diversity

The uniqueness (or otherwise) of test outputs ought to have a bearing on test effectiveness, yet it has not previously been studied. In this paper we introduce a novel test suite adequacy criterion based on output uniqueness. We propose 4 definitions of output uniqueness with varying degrees of strictness. We present a preliminary evaluation for web application testing that confirms that output uniqueness enhances fault-finding effectiveness. The approach outperforms random augmentation in fault finding ability by an overall average of 280% in 5 medium sized, real world web applications.

State aware test case regeneration for improving web application test suite coverage and fault detection

This paper introduces two test cases regeneration approaches for web applications, one uses standard Def-Use testing but for state variables, the other uses a novel value-aware dataflow approach. Our overall approach is to combine requests from a test suite to form client-side request sequences, based on dataflow analysis of server-side session variables and database tables. We implemented our approach as a tool SART (State Aware Regeneration Tool) and used it to evaluate our proposed approaches on 4 real world web applications. Our results show that for all 4 applications, both server-side coverage and fault detection were statistically significantly improved. Even on relatively high quality test suites our algorithms improve average coverage by 14.74% and fault detection by 9.19%.

Crawlability metrics for automated web testing

Web applications are exposed to frequent changes both in requirements and involved technologies. At the same time, there is a continuously growing demand for quality and trust and such a fast evolution and quality constraints claim for mechanisms and techniques for automated testing. Web application automated testing often involves random crawlers to navigate the application under test and automatically explore its structure. However, owing to the specific challenges of the modern Web systems, automatic crawlers may leave large portions of the application unexplored. In this paper, we propose the use of structural metrics to predict whether an automatic crawler with given crawling capabilities will be sufficient or not to achieve high coverage of the application under test. In this work, we define a taxonomy of such capabilities and we determine which combination of them is expected to give the highest reward in terms of coverage increase. Our proposal is supported by an experiment in which 19 web applications have been analyzed.

Improving Web Application Testing using testability measures

One of the challenges of testing web applications derives from their dynamic content and structure. As we test a website, we may discover more about its structure and behaviour. This paper proposes a framework for collection of testability measures during the automated testing process (termed ‘in-testing’ measure collection). The measures gathered in this way can take account of dynamic and content driven aspects of web applications, such as form structure, client-side scripting and server-side code. Their goal is to capture measurements related to on-going testing activity, indicating where additional testing can best lead to higher overall coverage. They denote a form of ‘web testability’ measures. The paper reports on the implementation of a prototype Web Application Testing Tool, WATT, illustrating the in-testing measure collection approach on 34 forms taken from 14 real world web applications.

Assessing the Impact of Firewalls and Database Proxies on SQL Injection Testing

This paper examines the effects and potential benefits of utilising Web Application Firewalls (WAFs) and database proxies in SQL injection testing of web applications and services. We propose testing the WAF itself to refine and evaluate its security rules and prioritise fixing vulnerabilities that are not protected by the WAF. We also propose using database proxies as oracles for black-box security testing instead of relying only on the output of the application under test. The paper also presents a case study of our proposed approaches on two sets of web services. The results indicate that testing through WAFs can be used to prioritise vulnerabilities and that an oracle that uses a database proxy finds more vulnerabilities with fewer tries than an oracle that relies only on the output of the application.

Crawlability Metrics for Web Applications

Automated web crawlers can be used to explore and exercise portions of a web application under test. However, the possibility to achieve full exploration of a web application through automated crawling is severely limited by the choice of the input values submitted with forms. Depending on the crawlers capabilities, a larger or smaller portion of web application will be automatically explored. In this paper, we introduce web crawl ability metrics to quantify properties of application pages and forms that affect crawl ability. Moreover, we show that our metrics can be used to identify the boundaries between those parts of the application that can be successfully crawled automatically and those parts that will require manual intervention or other crawl ability support. We have validated our crawl ability metrics on real web applications, for which low crawl ability was indeed associated with the existence of pages never exercised during automated crawling.