Josip Bozic

Attack pattern-based combinatorial testing

The number of potential security threats rises with the increasing number of web applications, which cause tremendous financial and existential implications for developers and users as well. The biggest challenge for security testing is to specify and implement ways in order to detect potential vulnerabilities of the developed system in a never ending quest against new security threats but also to cover already known ones so that a program is suited against typical attack vectors. For these purposes many approaches have been developed in the area of model-based security testing in order to come up with solutions for real-world application problems. These approaches provide theoretical background as well as practical solutions for certain security issues. In this paper, we partially rely on previous work but focus on the representation of attack patterns using UML state diagrams. We extend previous work in combining the attack pattern models with combinatorial testing in order to provide concrete test input, which is submitted to the system under test. With combinatorial testing we capture different combinations of inputs and thus increasing the likelihood to find weaknesses in the implementation under test that can be exploited. Besides the foundations of our approach we further report on first experiments that indicate its practical use.

XSS pattern for attack modeling in testing

Security issues of web applications are still a current topic of interest especially when considering the consequences of unintended behaviour. Such services might handle sensitive data about several thousands or millions of users. Hence, exploiting services or other undesired effects that cause harm on users has to be avoided. Therefore, for software developers of such applications one of the major tasks in providing security is to embed testing methodologies into the software development cycle, thus minimizing the subsequent damage resulting in debugging and time intensive upgrading. Model-based testing evolved as one of the methodologies which offer several theoretical and practical approaches in testing the system under test (SUT) that combine several input generation strategies like mutation testing, using of concrete and symbolic execution etc. by putting the emphasis on specification of the model of an application. In this work we propose an approach that makes use of an attack pattern model in form of a UML state machine for test case generation and execution. The paper also discusses the current implementation of our attack pattern testing tool using a XSS attack pattern and demonstrates the execution in a case study.

Attack Pattern-Based Combinatorial Testing with Constraints for Web Security Testing

Security testing of web applications remains a major problem of software engineering. In order to reveal vulnerabilities, manual and automatic testing approaches use different strategies for detection of certain kinds of inputs that might lead to a security breach. In this paper we compared a state-of-the-art manual testing tool with an automated one that is based on model-based testing. The first tool requires user input from the tester whereas the second one reduces the necessary amount of manual manipulation. Both approaches depend on the corresponding test case generation technique and its produced inputs are executed against the system under test (SUT). For this case we enhance a novel technique, which combines a combinatorial testing technique for input generation and a model-based technique for test execution. In this work the input parameter modelling is improved by adding constraints to generate more comprehensive and sophisticated testing inputs. The evaluated results indicate that both techniques succeed in detecting security leaks in web applications with different results, depending on the background logic of the testing approach. Last but not least, we claim that attack pattern-based combinatorial testing with constraints can be an alternative method for web application security testing, especially when we compare our method to other test generation techniques like fuzz testing.

Evaluation of the IPO-Family algorithms for test case generation in web security testing

Security testing of web applications remains a major problem of software engineering. In order to reveal vulnerabilities, testing approaches use different strategies for detection of certain kinds of inputs that might lead to a security breach. Such approaches depend on the corresponding test case generation technique that are executed against the system under test. In this work we examine how two of the most popular algorithms for combinatorial test case generation, namely the IPOG and IPOG-F algorithms, perform in web security testing. For generating comprehensive and sophisticated testing inputs we have used input parameter modelling which includes also constraints between the different parameter values. To handle the test execution, we make use of a recently introduced methodology which is based on model-based testing. Our evaluation indicates that both algorithms generate test inputs that succeed in revealing security leaks in web applications with IPOG-F giving overall slightly better results w.r.t. the test quality of the generated inputs. In addition, using constraints during the modelling of the attack grammars results in an increase on the number of test inputs that cause security breaches. Last but not least, a detailed analysis of our evaluation results confirms that combinatorial testing is an efficient test case generation method for web security testing as the security leaks are mainly due to the interaction of a few parameters. This statement is further supported by some combinatorial coverage measurement experiments on the successful test inputs.

Security Testing Based on Attack Patterns

Testing for security related issues is an important task of growing interest due to the vast amount of applications and services available over the internet. In practice testing for security often is performed manually with the consequences of higher costs, and no integration of security testing with todays agile software development processes. In order to bring security testing into practice, many different approaches have been suggested including fuzz testing and model-based testing approaches. Most of these approaches rely on models of the system or the application domain. In this paper we suggest to formalize attack patterns from which test cases can be generated and even executed automatically. Hence, testing for known attacks can be easily integrated into software development processes where automated testing, e.g., for daily builds, is a requirement. The approach makes use of UML state charts. Besides discussing the approach, we illustrate the approach using a case study.

PURITY: A Planning-based secURITY Testing Tool

Despite sophisticated defense mechanisms security testing still plays an important role in software engineering. Because of their latency, security flaws in web applications always bear the risk of being exploited sometimes in the future. In order to avoid potential damage, appropriate prevention measures should be incorporated in time and in the best case already during the software development cycle. In this paper, we contribute to this this goal and present the PURITY tool for testing web applications. PURITY executes test cases against a given website. It detects whether the website is vulnerable against some of the most common vulnerabilities, i.e., SQL injections and cross-site scripting. The goal is to resemble a malicious activity by following typical sequences of actions potentially leading to a vulnerable state. The test execution proceeds automatically. In contrast to other penetration testing tools, PURITY relies on planning. Concrete test cases are obtained from a plan, which in turn is generated from specific initial values and given actions. The latter are intended to mimic actions usually performed by an attacker. In addition, PURITY also allows a tester to configure input parameters and also tests a website in a manual manner.

Plan It! Automated Security Testing Based on Planning

Testing of web applications for common vulnerabilities still represents a major challenge in the area of security testing. The objective here is not necessarily to find new vulnerabilities but to ensure that the web application handles well-known attack patterns in a reliable way. Previously developed methods based on formalizing attack patterns contribute to the underlying challenge. However, the adaptation of the attack models is not easy and requires substantial effort. In order to make modeling easier we suggest representing attacks as a sequence of known actions that have to be carried out in order to be successful. Each action has some pre conditions and some effects. Hence, we are able to represent testing in this context as a planning problem where the goal is to break the application under test. In the paper, we discuss the proposed planning based testing approach, introduce the underlying concepts and definitions, and present some experimental results obtained from an implementation.

Testing TLS Using Combinatorial Methods and Execution Framework

The TLS protocol is the standard for secure Internet communication between two parties. Unfortunately, there have been recently successful attacks like DROWN or BREACH that indicate the necessity for thoroughly testing TLS implementations. In our research work, we focus on automated test case generation and execution for the TLS security protocol, where the aim is to make use of combinatorial methods for providing test cases that ideally also reveal previously unknown attacks. This is made feasible by creating appropriate input parameter models for different messages that can appear in a TLS message sequence. In this paper, we present the resulting test case generation and execution framework together with the corresponding testing oracle. Furthermore, we discuss first empirical results obtained using different TLS implementations and their releases.

Planning-Based Security Testing of the SSL/TLS Protocol

With a growing amount of transferred data in an interconnected world, the insurance of a secure communication between two peers becomes a critical task in the software industry. A leak of critical data can cause tremendous costs in a financial, social but also political manner. For this sake, cryptographic protocols are implemented and regulate the data transfer, thus ensuring the safety of transferred data between two peers. The widespread security protocol SSL/TLS provides the mechanisms for this request, however, not without drawbacks since several security leaks have been identified up to now. Since vulnerabilities act as a starting point for a potential malicious action, the identification of such leaks is of highest priority. In this paper a novel testing approach is presented, which adapts planning for security testing of cryptographic protocols. The whole approach is implemented in one testing framework. Its purpose is to automatically test for known vulnerabilities in protocol implementations but to trigger other unintended behavior as well so eventually new security flaws can be identified. Additionally, the planning specification can be extended further so new testing possibilities can be generated. New test cases can be generated dynamically according to changing conditions.

Retaining Consistency for Knowledge-Based Security Testing

Testing of software and systems requires a set of inputs to the system under test as well as test oracles for checking the correctness of the obtained output. In this paper we focus on test oracles within the domain of security testing, which require consistent knowledge of security policies. Unfortunately, consistency of knowledge cannot always be ensured. Therefore, we strongly require a process of retaining consistencies in order to provide a test oracle. In this paper we focus on an automated approach for consistency handling that is based on the basic concepts and ideas of model-based diagnosis. Using a brief example, we discuss the underlying method and its application in the domain of security testing. The proposed algorithm guarantees to find one root cause of an inconsistency and is based on theorem proving.