Bernhard Garn

On the applicability of combinatorial testing to web application security testing: a case study

Case studies for evaluating tools in security testing are powerful. Although they cannot achieve the scientific rigor of formal experiments, the results can provide sufficient information to help professionals judge if a specific technology being evaluated will benefit their organization. This paper reports on a case study done for evaluating and revisiting a recently introduced combinatorial testing methodology used for web application security purposes. It further reports on undertaken practical experiments thus strengthening the applicability of combinatorial testing to web application security testing.

Attack Pattern-Based Combinatorial Testing with Constraints for Web Security Testing

Security testing of web applications remains a major problem of software engineering. In order to reveal vulnerabilities, manual and automatic testing approaches use different strategies for detection of certain kinds of inputs that might lead to a security breach. In this paper we compared a state-of-the-art manual testing tool with an automated one that is based on model-based testing. The first tool requires user input from the tester whereas the second one reduces the necessary amount of manual manipulation. Both approaches depend on the corresponding test case generation technique and its produced inputs are executed against the system under test (SUT). For this case we enhance a novel technique, which combines a combinatorial testing technique for input generation and a model-based technique for test execution. In this work the input parameter modelling is improved by adding constraints to generate more comprehensive and sophisticated testing inputs. The evaluated results indicate that both techniques succeed in detecting security leaks in web applications with different results, depending on the background logic of the testing approach. Last but not least, we claim that attack pattern-based combinatorial testing with constraints can be an alternative method for web application security testing, especially when we compare our method to other test generation techniques like fuzz testing.

Eris: A Tool for Combinatorial Testing of the Linux System Call Interface

In this paper, we show the applicability of combinatorial testing to the system call interface of the Linux kernel. Our approach is two-fold: first we analyze the Trinity fuzz tester and in the aftermath we adapt the input parameter modeling of Trinity to the field of combinatorial testing. Furthermore, apart from the modeling itself, we target to provide a configurable testing framework for executing tests obtained by the ACTS combinatorial test generation tool, called Eris.

Evaluation of the IPO-Family algorithms for test case generation in web security testing

Security testing of web applications remains a major problem of software engineering. In order to reveal vulnerabilities, testing approaches use different strategies for detection of certain kinds of inputs that might lead to a security breach. Such approaches depend on the corresponding test case generation technique that are executed against the system under test. In this work we examine how two of the most popular algorithms for combinatorial test case generation, namely the IPOG and IPOG-F algorithms, perform in web security testing. For generating comprehensive and sophisticated testing inputs we have used input parameter modelling which includes also constraints between the different parameter values. To handle the test execution, we make use of a recently introduced methodology which is based on model-based testing. Our evaluation indicates that both algorithms generate test inputs that succeed in revealing security leaks in web applications with IPOG-F giving overall slightly better results w.r.t. the test quality of the generated inputs. In addition, using constraints during the modelling of the attack grammars results in an increase on the number of test inputs that cause security breaches. Last but not least, a detailed analysis of our evaluation results confirms that combinatorial testing is an efficient test case generation method for web security testing as the security leaks are mainly due to the interaction of a few parameters. This statement is further supported by some combinatorial coverage measurement experiments on the successful test inputs.

Testing TLS Using Combinatorial Methods and Execution Framework

The TLS protocol is the standard for secure Internet communication between two parties. Unfortunately, there have been recently successful attacks like DROWN or BREACH that indicate the necessity for thoroughly testing TLS implementations. In our research work, we focus on automated test case generation and execution for the TLS security protocol, where the aim is to make use of combinatorial methods for providing test cases that ideally also reveal previously unknown attacks. This is made feasible by creating appropriate input parameter models for different messages that can appear in a TLS message sequence. In this paper, we present the resulting test case generation and execution framework together with the corresponding testing oracle. Furthermore, we discuss first empirical results obtained using different TLS implementations and their releases.

A Combinatorial Approach to Analyzing Cross-Site Scripting (XSS) Vulnerabilities in Web Application Security Testing

Web applications typically employ sanitization functions to sanitize user inputs, independently whether this input is assumed to be legitimate, invalid or malicious. When such functions do not work correctly, a web application immediately becomes vulnerable to security attacks such as XSS. In this paper, we report a combinatorial approach to analyze XSS vulnerabilities in web applications. Our approach first performs combinatorial testing where a set of test vectors is executed against a subject application. If one or more XSS vulnerabilities are triggered during testing, we analyze the structure of each test vector to identify XSS-inducing combinations of its parameter model. If an attack vector contains an XSS-inducing combination, then the execution of this vector will successfully exploit an XSS vulnerability. Identification of XSS-inducing combinations provides insights about which kinds of user input might still be leverageable for XSS attacks and how to correct the function to provide better security guarantees. We conducted an experiment in which our approach was applied to four sanitization functions from the Web Application Vulnerability Scanner Evaluation Project (WAVSEP). The experimental results show that our approach can effectively identify XSS-inducing combinations for these sanitization functions.

Combinatorial Methods for Modelling Composed Software Systems

In this paper, we review and prove certain properties regarding t-way coverage when covering arrays are merged together to form larger arrays. Moreover, we build upon these theoretical observations to formulate a procedure on how, from the input space model of a composed SUT and t-way test suites for its components, to create a larger and unified t-way test suite. The proposed modelling methodologies for a composed SUT involve a hierarchy between the different input models of the components of the SUT, which is nested in such a way that it can be linked to certain combinatorial constructions for covering arrays. As a proof of concept of our modelling methodology, that arises from such combinatorial constructions, we apply it to a sample of composed SUTs to validate our approach in practice.

KERIS: A CT Tool of the Linux Kernel with Dynamic Memory Analysis Capabilities.

We present KERIS, a configurable, non-centralized server-based framework which enables the combinatorial testing of the Linux kernel’s system call interface. The tool constitutes an improvement over our previously developed tool called ERIS by incorporating dynamic memory analysis capabilities among other improvements. The testing framework is designed to offer large-scale automation and requires only minimal high-level input from the user. Several experiments performed with KERIS demonstrate the capabilities of finding and reproducing Linux kernel bugs in an automated manner.

Testing TLS using planning-based combinatorial methods and execution framework

The TLS protocol is the standard for secure Internet communication between two parties. Unfortunately, there have been recently successful attacks like DROWN, ROBOT, or BREACH that indicate the necessity for thoroughly testing TLS implementations. In our research work, we focus on automated test case generation and execution for the TLS security protocol, where the aim is to combine planning with combinatorial methods for providing test cases that ideally also reveal previously unknown attacks. This is made feasible by creating appropriate input parameter models for different messages that can appear in a TLS message sequence. In this paper, we present the resulting test case generation and execution framework together with the corresponding test oracle. Furthermore, we discuss in detail empirical results obtained via testing different TLS implementations.

Covering Arrays via Set Covers

Abstract In this paper, we provide yet another construction on how to map problems pertaining covering arrays to specialized set cover problems. With this mapping it is possible to compute optimal covering arrays via minimal solutions of the generated set cover problem.