Xinzhou Qin

Statistical Causality Analysis of Infosec Alert Data

With the increasingly widespread deployment of security mechanisms, such as firewalls, intrusion detection systems (IDSs), antiviras software and authentication services, the problem of alert analysis has become very important. The large amount of alerts can overwhelm security administrators and prevent them from adequately understanding and analyzing the security state of the network, and initiating appropriate response in a timely fashion. Recently, several approaches for alert correlation and attack scenario analysis have been proposed. However, these approaches all have limited capabilities in detecting new attack scenarios. In this paper, we study the problem of security alert correlation with an emphasis on attack scenario analysis. In our framework, we use clustering techniques to process low-level alert data into high-level aggregated alerts, and conduct causal analysis based on statistical tests to discover new relationships among attacks. Our statistical causality approach complements other approaches that use hard-coded prior knowledge for pattern matching. We perform a series of experiments to validate our method using DARPA’s Grand Challenge Problem (GCP) datasets, the 2000 DARPA Intrusion Detection Scenario datasets, and the DEF CON 9 datasets. The results show that our approach can discover new patterns of attack relationships when the alerts of attacks are statistically correlated.

HoneyStat: Local Worm Detection Using Honeypots

Worm detection systems have traditionally used global strategies and focused on scan rates. The noise associated with this approach requires statistical techniques and large data sets (e.g., 220 monitored machines) to yield timely alerts and avoid false positives. Worm detection techniques for smaller local networks have not been fully explored.

Proactive detection of distributed denial of service attacks using MIB traffic variables-a feasibility study

We propose a methodology for utilizing network management systems for the early detection of distributed denial of service (DDoS) attacks. Although there are quite a large number of events that are prior to an attack (e.g. suspicious log-ons, start of processes, addition of new files, sudden shifts in traffic, etc.), in this work we depend solely on information from MIB (management information base) traffic variables collected from the systems participating in the attack. Three types of DDoS attacks were effected on a research test bed, and MIB variables were recorded. Using these datasets, we show how there are indeed MIB-based precursors of DDoS attacks that render it possible to detect them before the target is shut down. Most importantly, we describe how the relevant MIB variables at the attacker can be extracted automatically using statistical tests for causality. It is shown that statistical tests applied in the time series of MIB traffic at the target and the attacker are effective in extracting the correct variables for monitoring in the attacker machine. Following the extraction of these key variables at the attacker, it is shown that an anomaly detection scheme, based on a simple model of the normal rate of change of the key MIBs can be used to determine statistical signatures of attacking behavior. These observations suggest the possibility of an entirely automated procedure centered on network management systems for detecting precursors of distributed denial of service attacks, and responding to them.

Attack plan recognition and prediction using causal networks

Correlating and analyzing security alerts is a critical and challenging task in security management. Recently, some techniques have been proposed for security alert correlation. However, these approaches focus more on basic or low-level alert correlation. In this paper, we study how to conduct probabilistic inference to correlate and analyze attack scenarios. Specifically, we propose an approach to solving the following problems: 1) How to correlate isolated attack scenarios resulted from low-level alert correlation? 2) How to identify attackers high-level strategies and intentions? 3) How to predict the potential attacks based on observed attack activities? We evaluate our approaches using DARPAs grand challenge problem (GCP) data set. The results demonstrate the capability of our approach in correlating isolated attack scenarios, identifying attack strategies and predicting future attacks.

Worm detection, early warning and response based on local victim information

Worm detection systems have traditionally focused on global strategies. In the absence of a global worm detection system, we examine the effectiveness of local worm detection and response strategies. This paper makes three contributions: (1) we propose a simple two-phase local worm victim detection algorithm, DSC (Destination-Source Correlation), based on worm behavior in terms of both infection pattern and scanning pattern. DSC can detect zero-day scanning worms with a high detection rate and very low false positive rate. (2) We demonstrate the effectiveness of early worm warning based on local victim information. For example, warning occurs with 0.19% infection of all vulnerable hosts on Internet when using a /12 monitored network. (3) Based on local victim information, we investigate and evaluate the effectiveness of an automatic real-time local response in terms of slowing down the global Internet worms propagation. (2) and (3) are general results, not specific to certain detection algorithm like DSC. We demonstrate (2) and (3) with both analytical models and packet-level network simulator experiments.

Discovering Novel Attack Strategies from Infosec Alerts

Correlating security alerts and discovering attack strategies are important and challenging tasks for security analysts. Recently, there have been several proposed techniques to analyze attack scenarios from security alerts. However, most of these approaches depend on a priori and hard-coded domain knowledge that lead to their limited capabilities of detecting new attack strategies. In this paper, we propose an approach to discover novel attack strategies. Our approach includes two complementary correlation mechanisms based on two hypotheses of attack step relationship. The first hypothesis is that attack steps are directly related because an earlier attack enables or positively affects the later one. For this type of attack relationship, we develop a Bayesian-based correlation engine to correlate attack steps based on security states of systems and networks. The second hypothesis is that for some related attack steps, even though they do not have obvious and direct relationship in terms of security and performance measures, they still have temporal and statistical patterns. For this category of relationship, we apply time series and statistical analysis to correlate attack steps. The security analysts are presented with aggregated information on attack strategies from these two correlation engines. We evaluate our approach using DARPA’s Grand Challenge Problem (GCP) data sets. The results show that our approach can discover novel attack strategies and provide a quantitative analysis of attack scenarios.

Proactive Intrusion Detection and Distributed Denial of Service Attacks—A Case Study in Security Management

Little or no integration exists today between Intrusion Detection Systems (IDSs) and SNMP-based Network Management Systems (NMSs), in spite of the extensive monitoring and alarming capabilities offered by commercial NMSs. This difficulty is mainly associated with the distinct data sources used by the two systems: packet traffic and audit records for IDSs and SNMP MIB variables for NMSs. In this paper we propose and evaluate a methodology for utilizing NMSs for the early detection of Distributed Denial of Service attacks (DDoS). A principled approach is described for discovering precursors to DDoS attacks in databases formed by MIB variables recorded from multiple domains in networked information systems. The approach is rooted in time series quantization, and in the application of the Granger Causality Test of classical statistics for selecting variables that are likely to contain precursors. A methodology is proposed for discovering precursor rules from databases containing time series related to different regimes of a system. These precursor rules relate precursor events extracted from input time series with phenomenon events extracted from output time series. Using MIB datasets collected from real experiments involving Distributed Denial of Service Attacks, it is shown that precursor rules relating activities at attacking machines with traffic floods at target machines can be extracted by the methodology. The technology has extensive applications for security management: it enables security analysts to better understand the evolution of complex computer attacks, it can be used to trigger alarms indicating that an attack is imminent, or it can be used to reduce the false alarm rates of conventional IDSs.

Integrating intrusion detection and network management

The problems of detecting and resolving performance in distributed systems have become increasingly important and challenging due to the tremendous growth in network-based services. There is a need for a predictive and proactive approach so that appropriate and timely actions can be taken before service disruptions escalate and become widespread. A network management system (NMS) is responsible for monitoring the performance of a network. An intrusion detection system (IDS) is responsible for detecting and responding to intrusions. The current practice is that NMS and IDS are independent to each other in a network. There is little integration and information sharing between the two. We outline an approach to integrate NMS and IDS so that the security capabilities of network management can be enhanced and the performance of IDS can be improved.

Proactive intrusion detection and SNMP-based security management: new experiments and validation

In our earlier work we have proposed and developed a methodology for the early detection of distributed denial of service (DDoS) attacks. In this paper, we examine the applicability of proactive intrusion detection on a considerably more complex set-up, with hosts associated with three clusters, connected by routers. Background TCP, UDP and ICMP traffic following interrupted Poisson processes are superimposed on the attack traffic. We have examined six types of DDoS attacks. In four of the attacks we have obtained valid MIB-based precursors with no false alarms in all experiments. In the remaining two attacks precursors were obtained, but false alarms were observed. Procedures for eliminating these false alarms are discussed.

Using MIB II Variables for Network Intrusion Detection

Detecting and resolving security and performance problems in distributed systems have become increasingly important and challenging because of the tremendous growth in network-based services. Intrusion detection is an important security technique for networks and systems. In this paper, we propose a methodology for utilizing MIB II objects for network intrusion detection. We establish the normal profiles of network activities based on the information provided by the MIB II variables and use data mining techniques and information-theoretic measures to build an intrusion detection model. We test our MIB II-based intrusion detection model with several Denial of Service (DoS) and probing attacks. The results have shown that the model can detect these attacks effectively.