Guoqiang Shu

Testing Security Properties of Protocol Implementations - a Machine Learning Based Approach

Security and reliability of network protocol implementations are essential for communication services. Most of the approaches for verifying security and reliability, such as formal validation and black-box testing, are limited to checking the specification or conformance of implementation. However, in practice, a protocol implementation may contain engineering details, which are not included in the system specification but may result in security flaws. We propose a new learning-based approach to systematically and automatically test protocol implementation security properties. Protocols are specified using symbolic parameterized extended finite state machine (SP-EFSM) model, and an important security property - message confidentiality under the general Dolev-Yao attacker model - is investigated. The new testing approach applies black-box checking theory and a supervised learning algorithm to explore the structure of an implementation under test while simulating the teacher with a conformance test generation scheme. We present the testing procedure, analyze its complexity, and report experimental results.

A model-based approach to security flaw detection of network protocol implementations

A lot of efforts have been devoted to the analysis of network protocol specification for reliability and security properties using formal techniques. However, faults can also be introduced during system implementation; it is indispensable to detect protocol implementation flaws, yet due to the black-box nature of protocol implementation and the unavailability of protocol specification most of the approaches resort to random or manual testing. In this paper we propose a model-based approach for security flaw detection of protocol implementation with a high fault coverage, measurability, and automation. Our approach first synthesizes an abstract behavioral model from a protocol implementation and then uses it to guide the testing process for detecting security and reliability flaws. For protocol specification synthesis we reduce the problem a trace minimization with a finite state machine model and an efficient algorithm is presented for state space reduction. Our method is implemented and applied to real network protocols. Guided by the synthesized model our testing tool reveals a number of unknown reliability and security issues by automatically crashing the implementations of the Microsoft MSN instant messaging (MSNIM) protocol. Analytical comparison between our model-based and prevalent syntax-based flaw detection schemes is also provided with the support of experimental results.

Detecting Communication Protocol Security Flaws by Formal Fuzz Testing and Machine Learning

Network-based fuzz testing has become an effective mechanism to ensure the security and reliability of communication protocol systems. However, fuzz testing is still conducted in an ad-hoc manner with considerable manual effort, which is mainly due to the unavailability of protocol model. In this paper we present our on-going work of developing an automated and measurable protocol fuzz testing approach that uses a formally synthesized approximate formal protocol specification to guide the testing process. We adopt the Finite State Machine protocol model and study two formal methods for protocol synthesis: an active black-box checking algorithm that has provable optimality and a passive trace minimization algorithm that is less accurate but much more efficient. We also present our preliminary results of using this method to implementations of the MSN instant messaging protocol: MSN clients Gaim (pidgin) and aMSN. Our testing reveals some serious reliability and security flaws by automatically crashing both of them.

Network Protocol System Fingerprinting - A Formal Approach

Network protocol system fingerprinting has been recognized as an important issue and a major threat to network security. Prevalent works rely largely on human experiences and insight of the protocol system specifications and implementations. Such ad-hoc approaches are inadequate in dealing with large complex protocol systems. In this paper we propose a formal approach for automated protocol system fingerprinting analysis and experiment. Parameterized Extended Finite State Machine is used to model protocol systems, and four categories of fingerprinting problems are formally defined. We propose and analyze algorithms for both active and passive fingerprinting and present our experimental results on Internet protocols. Furthermore, we investigate protection techniques against malicious fingerprinting and discuss the feasibility of two defense schemes, based on the protocol and application scenarios.

Message confidentiality testing of security protocols: passive monitoring and active checking

Security protocols provide critical services for distributed communication infrastructures. However, it is a challenge to ensure the correct functioning of their implementations, particularly, in the presence of malicious parties. We study testing of message confidentiality – an essential security property. We formally model protocol systems with an intruder using Dolev-Yao model. We discuss both passive monitoring and active testing of message confidentiality. For adaptive testing, we apply a guided random walk that selects next input on-line based on transition coverage and intruders knowledge acquisition. For mutation testing, we investigate a class of monotonic security flaws, for which only a small number of mutants need to be tested for a complete checking. The well-known Needham-Schroeder-Lowe protocol is used to illustrate our approaches.

A note on broadcast encryption key management with applications to large scale emergency alert systems

Emergency alerting capability is crucial for the prompt response to natural disasters and terrorist attacks. The emerging network infrastructure and secure broadcast techniques enable prompt and secure delivery of emergency notification messages. With the ubiquitous deployment of alert systems, scalability and heterogeneity pose new challenges for the design of secure broadcast schemes. In this paper, we discuss the key generation problem with the goal of minimizing the total number of keys which need to be generated by the alert center and distributed to the users. Two encryption schemes, zero message scheme and extended header scheme, are modeled formally. For both schemes we show the equivalence of the general optimal key generation (OKG) problem and the bipartite clique cover (BCC) problem, and show that OKG problem is NP-hard. The result is then generalized to the case with resource constraints, and we provide a heuristic algorithm for solving the restricted BCC (and OKG) problem.

Virtual Cyber-Security Testing Capability for Large Scale Distributed Information Infrastructure Protection

Security, reliability and interoperability are indispensable in todays distributed heterogeneous information infrastructure. For government and military applications, it is crucial to conduct effective and efficient testing of security properties for newly developed systems, which are to be integrated into existing information system. Yet little progress has been made in the technology advancement of rigorous and automated security testing. In this contribution we present virtual cyber security testing capability (VCSTC) - a DoD funded project-for developing an automated testing capability that can assess the operational functions and security impact of a target system without physically integrating it into an intended network infrastructure. VCSTC first synthesizes a model to emulate the real network infrastructure; then it automatically generates and executes test cases with guaranteed coverage of the features and security properties under test. This report presents the architecture of VCSTC, its key techniques and experimental results on real systems.

Minutiae: A Formal Methodology for Accurate Protocol Fingerprinting

We study the new problem of network protocol fingerprinting, which has been recognized as both a threat to cyberspace privacy and a useful technique for intrusion detection. This paper provides the first taxonomy of fingerprint matching and discovery problems based on a formal fingerprint model call Minutiae. The FSM based Minutiae model captures more structural characteristics of a protocol implementation than the traditional trace-based representation, and therefore enables rigid analysis and more accurate fingerprinting algorithms. We first introduce our formal model and classification of fingerprinting problems; then propose solution for each category of problem illustrated with examples. For all algorithms we also present analysis of their time complexity.

A Formal Methodology for Network Protocol Fingerprinting

Network protocol fingerprinting refers to the process of identifying a protocol implementation by their input and output behaviors. It has been regarded as both a potential threat to network security and also as a useful mechanism for network management. Existing protocol fingerprinting tools share common disadvantages such as being protocol-specific and difficult to automate. This paper proposes a formal methodology for fingerprinting experiments using which we can model a broad spectrum of fingerprinting problems and design-efficient algorithms. We present a formal behavioral model that specifies a protocol principal by its states and transitions, then identify a complete taxonomy of fingerprint matching and discovery problems is identified based on 1) whether the fingerprinting experiment is active or passive and 2) the information available about the specifications and implementations. Algorithms to solve the problems are discussed. In particular, for fingerprint matching algorithm, we propose an efficient PEFSM online separation algorithm for active experiment and concurrent passive testing for passive experiments. For fingerprint discovery problem, there are two cases: if the protocol specification is available as a nondeterministic PEFSM, we apply across verification and back-tracing technique for active and passive discovery, respectively; if no specification is available, we take the machine learning approach and discover the fingerprint by active testing.

VCSTC: Virtual Cyber Security Testing Capability --- An Application Oriented Paradigm for Network Infrastructure Protection

Network security devices are becoming more sophisticated and so are the testing processes. Traditional network testbeds face challenges in terms of fidelity, scalability and complexity of security features. In this paper we propose a new methodology of testing security devices using network virtualization techniques, and present an integrated solution, including network emulation, test case specification and automated test execution. Our hybrid network emulation scheme provides high fidelity by host virtualization and scalability by lightweight protocol stack emulation. We also develop an intermediate level test case description language that is suitable for security tests at various network protocol layers and that can be executed automatically on the emulated network. The methodology presented in this paper has been implemented and integrated into a security infrastructure testing system for US Department of Defense and we report the experimental results.