Evan Martin

A fault model and mutation testing of access control policies

To increase confidence in the correctness of specified policies, policy developers can conduct policy testing by supplying typical test inputs (requests) and subsequently checking test outputs (responses) against expected ones. Unfortunately, manual testing is tedious and few tools exist for automated testing of access control policies. We present a fault model for access control policies and a framework to explore it. The framework includes mutation operators used to implement the fault model, mutant generation, equivalent-mutant detection, and mutant-killing determination. This framework allows us to investigate our fault model, evaluate coverage criteria for test generation and selection, and determine a relationship between structural coverage and fault-detection effectiveness. We have implemented the framework and applied it to various policies written in XACML. Our experimental results offer valuable insights into choosing mutation operators in mutation testing and choosing coverage criteria in test generation and selection.

Defining and measuring policy coverage in testing access control policies

To facilitate managing access control in a system, security officers increasingly write access control policies in specification languages such as XACML, and use a dedicated software component called a Policy Decision Point (PDP). To increase confidence on written policies, certain types of policy testing (often in an ad hoc way) are usually conducted, which probe the PDP with some typical requests and check PDPs responses against expected ones. This paper develops a first step toward systematic policy testing by defining and measuring policy coverage when testing policies. We have developed a coverage-measurement tool to measure policy coverage given a set of XACML policies and a set of requests. We have developed a tool for request generation, which randomly generates requests for a given set of policies, and a tool for request reduction, which greedily selects a nearly minimal set of requests for achieving the same coverage as the originally generated requests. To evaluate coverage-based request reduction and its effect on fault detection, we have conducted an experiment with mutation testing on a set of real policies. Our experimental results show that the coveragebased test reduction can substantially reduce the size of generated requests and incur only relatively low loss on fault detection. We also conduct a study on the policy coverage achieved by manually generated requests.

Automated test generation for access control policies

Access control policies are increasingly written in specification languages such as XACML. To increase confidence in the correctness of specified policies, policy developers can conduct policy testing to probe the Policy Decision Point (PDP) with some typical test inputs (in the form of requests) and check test outputs (in the form of responses) against expected ones. Unfortunately, manual test generation is tedious and manually generated tests are often not sufficient to exercise various policy behaviors. In this paper we present an efficient test generation approach and its supporting tool called Targen. We further reduce the number of generated requests based on structural coverage information to facilitate manual inspection. If a rule is unreachable due to an unsatisfiable set of constraints, it is redundant. We also present an approach for redundant-rule detection based on change-impact analysis and its supporting tool call Cirg. We have evaluated Targen on policies collected from various sources, some of which are complex policies being used in real systems. Our results show that Targen can effectively generate tests to achieve high structural coverage of policies and outperforms the existing random test generation in terms of structural coverage and fault-detection capability. Cirg can identify a large number of redundant rules among rules defined in a complex, real policy.

Conformance Checking of Access Control Policies Specified in XACML

Access control is one of the most fundamental and widely used security mechanisms. Access control mechanisms control which principals such as users or processes have access to which resources in a system. To facilitate managing and maintaining access control, access control policies are increasingly written in specification languages such as XACML. The specification of access control policies itself is often a challenging problem. Furthermore, XACML is intentionally designed to be generic: it provides the freedom in describing access control policies, which are well-known or invented ones. But the flexibility and expressiveness provided by XACML come at the cost of complexity, verbosity, and lack of desirable-property enforcement. Often common properties for specific access control policies may not be satisfied when these policies are specified in XACML, causing the discrepancy between what the policy authors intend to specify and what the actually specified XACML policies reflect. In this position paper, we propose an approach for conducting conformance checking of access control policies specified in XACML based on existing verification and testing tools for XACML policies.

Inferring access-control policy properties via machine learning

To ease the burden of implementing and maintaining access-control aspects in a system, a growing trend among developers is to write access-control policies in a specification language such as XACML and integrate the policies with applications through the use of a policy decision point (PDP). To assure that the specified polices reflect the expected ones, recent research has developed policy verification tools; however, their applications in practice are still limited, being constrained by the limited set of supported policy language features and the unavailability of policy properties. This paper presents a data-mining approach to the problem of verifying that expressed access-control policies reflect the true desires of the policy author. We developed a tool to investigate this approach by automatically generating requests, evaluating those requests to get responses, and applying machine learning on the request-response pairs to infer policy properties. These inferred properties facilitate the inspection of the policy behavior. We applied our tool on an access-control policy of a central grades repository system for a university. Our results show that machine learning algorithms can provide valuable insight into basic policy properties and help identify specific bug-exposing requests

Automatic extraction of abstract-object-state machines from unit-test executions

An automatic test-generation tool can produce a large number of test inputs to exercise the class under test. However, without specifications, developers cannot inspect the execution of each automatically generated test input practically. To address the problem, we have developed an automatic test abstraction tool, called Abstra, to extract high level object-state-transition information from unit-test executions, without requiring a priori specifications. Given a class and a set of its generated test inputs, our tool extracts object state machines (OSM): a state in an OSM represents an object state of the class and a transition in an OSM represents method calls of the class. When an object state in an OSM is concrete (being represented by the values of all fields reachable from the object), the size of the OSM could be too large to be useful for inspection. To address this issue, we have developed techniques in the tool to abstract object states based on returns of observer methods, branch coverage of methods, and individual object fields, respectively. The tool provides useful object-state-transition information for programmers to inspect unit-test executions effectively. In particular, the tool helps facilitate correctness inspection, program understanding, fault isolation, and test characterization.

Assessing Quality of Policy Properties in Verification of Access Control Policies

Access control policies are often specified in declarative languages. In this paper, we propose a novel approach, called mutation verification, to assess the quality of properties specified for a policy and, in doing so, the quality of the verification itself. In our approach, given a policy and a set of properties, we first mutate the policy to generate various mutant policies, each with a single seeded fault. We then verify whether the properties hold for each mutant policy. If the properties still hold for a given mutant policy, then the quality of these properties is determined to be insufficient in guarding against the seeded fault, indicating that more properties are needed to augment the existing set of properties to provide higher confidence of the policy correctness. We have implemented Mutaver, a mutation verification tool for XACML, and applied it to policies and properties from a real-world software system.

WebSob: A Tool for Robustness Testing of Web Services

Web services are a popular way of implementing a service-oriented architecture. Testing can be used to help assure both the correctness and robustness of a web service. Because manual testing is tedious, tools are needed to automate test generation and execution for web services. This paper presents WebSob, a tool for automatically generating and executing web-service requests given a service providers Web service description language (WSDL) specification. We have applied WebSob to freely available web services and our experiences show that WebSob can be used to quickly generate and execute web-service requests that may reveal robustness problems with no knowledge of the underlying web service implementation.

Testing and Analysis of Access Control Policies

Policy testing and analysis are important techniques for high assurance of correct specification of access control policies. We propose a set of testing and analysis techniques for access control policies and tools for empirically investigating and evaluating the proposed techniques. We propose a fault model for access control policies and investigate various fault types and their frequencies of occurrence in policy development; we develop a mutation testing framework that implements the fault model; we propose and investigate various coverage criteria for testing access control policies; we develop various test generation techniques and evaluate them using the coverage criteria and mutation testing framework; we develop a policy model to facilitate refactoring, performance optimizations, dependency identification, and other types of static analysis. To make our discussion concrete, we choose to present our techniques in the context of XACML. Note that since XACML is an application- independent, generic access control policy language, our techniques can be equally applied to test policies written in other languages.

Understanding software application interfaces via string analysis

In software systems, different software applications often interact with each other through specific interfaces by exchanging data in string format. For example, web services interact with each other through XML strings. Database applications interact with a database through strings of SQL statements. Sometimes these interfaces between different software applications are complex and distributed. For example, a table in a database can be accessed by multiple methods in a database application and a single method can access multiple tables. In this paper, we propose an approach to understanding software application interfaces through string analysis. The approach first performs a static analysis of source code to identify interaction points (in the form of interface-method-call sites). We then leverage existing string analysis tools to collect all possible string data that can be sent through these different interaction points. Then we manipulate collected string data by grouping similar data together. For example, we group together all collected SQL statements that access the same table. Then we associate various parts of aggregated data with interaction points in order to show the connections between entities from interacting applications. Our preliminary results show that the approach can help us understand the characteristics of interactions between database applications and databases. We also identify some challenges in this approach for our future work.