Ayesha Binte Ashfaq

A Comparative Evaluation of Anomaly Detectors under Portscan Attacks

Since the seminal 1998/1999 DARPA evaluations of intrusion detection systems, network attacks have evolved considerably. In particular, after the CodeRed worm of 2001, the volume and sophistication of self-propagating malicious code threats have been increasing at an alarming rate. Many anomaly detectors have been proposed, especially in the past few years, to combat these new and emerging network attacks. At this time, it is important to evaluate existing anomaly detectors to determine and learn from their strengths and shortcomings. In this paper, we evaluate the performance of eight prominent network-based anomaly detectors under malicious portscan attacks. These ADSs are evaluated on four criteria: accuracy (ROC curves), scalability (with respect to varying normal and attack traffic rates, and deployment points), complexity (CPU and memory requirements during training and classification,) and detection delay. These criteria are evaluated using two independently collected datasets with complementary strengths. Our results show that a few of the anomaly detectors provide high accuracy on one of the two datasets, but are unable to scale their accuracy across the datasets. Based on our experiments, we identify promising guidelines to improve the accuracy and scalability of existing and future anomaly detectors.

Information theoretic feature space slicing for statistical anomaly detection

Abstract Anomaly detection accuracy has been a serious limitation in commercial ADS deployments. A main reason for this limitation is the expectation that an ADS should achieve very high accuracy while having extremely low computational complexity. The constraint of low computational cost has recently been relaxed with the emergence of cheap high-performance platforms (e.g., multi-core, GPU, SCC, etc.). Moreover, current ADSs perform anomaly detection on aggregate feature spaces, with large volumes of benign and close-to-benign feature instances that overwhelm the feature space and hence yield low accuracies. In this paper, we ask and address the following question: Can the accuracy of an ADS be improved if we slice ADS feature space at the cost of higher computational resource utilization? We first observe that existing ADSs are not designed to exploit better computational platforms to achieve higher accuracies. To mitigate this problem, we identify the fundamental accuracy limiting factors for statistical network and host-based ADSs. We then show that these bottlenecks can be alleviated by our proposed feature space slicing framework. Our framework slices a statistical ADS׳ feature space into multiple disjoint subspaces and then performs anomaly detection separately on each subspace by utilizing more computational resources. We propose generic information-theoretic methods for feature space slicing and for determining the appropriate number of subspaces for any statistical ADS. Performance evaluation on three independently-collected attack datasets and multiple ID algorithms shows that the enhanced ADSs are able to achieve dramatic improvements in detection (up to 75%) and false alarm (up to 99%) rates.

Accuracy improving guidelines for network anomaly detection systems

An unprecedented growth in computer and communication systems in the last two decades has resulted in a proportional increase in the number and sophistication of network attacks. In particular, the number of previously-unseen attacks has increased exponentially in the last few years. Due to the rapidly evolving nature of network attacks, a considerable paradigm shift has taken place in the intrusion detection community. The main focus is now on Network Anomaly Detection Systems (NADSs) which model and flag deviations from normal/benign behavior of a network and can hence detect previously-unseen attacks. Contemporary NADS borrow concepts from a variety of theoretical fields (e.g., Information theory, stochastic and machine learning, signal processing, etc.) to model benign behavior. These NADSs, however, fall short of achieving acceptable performance levels as therefore widespread commercial deployments. Thus, in this paper, we firstly evaluate the performance of eight prominent network-based anomaly detectors under malicious portscan attacks to identify which NADSs perform better than others and why. These NADSs are evaluated on three criteria: accuracy (ROC curves), scalability (with respect to varying normal and attack traffic rates, and deployment points) and detection delay. These criteria are evaluated using two independently collected datasets with complementary strengths. We then propose novel methods and promising guidelines to improve the accuracy and scalability of existing and future anomaly detectors. Experimental analysis of the proposed guidelines is also presented for the proof of concept.

On the Inefficient Use of Entropy for Anomaly Detection

Entropy-based measures have been widely deployed in anomaly detection systems (ADSes) to quantify behavioral patterns. The entropy measure has shown significant promise in detecting diverse set of anomalies present in networks and end-hosts. We argue that the full potential of entropy-based anomaly detection is currently not being exploited because of its inefficient use. In support of this argument, we highlight three important shortcomings of existing entropy-based ADSes. We then propose efficient entropy usage --- supported by preliminary evaluations --- to mitigate these shortcomings.

Diagnosing bot infections using Bayesian inference

Prior research in botnet detection has used the bot lifecycle to build detection systems. These systems, however, use rule-based decision engines which lack automated adaptability and learning, accuracy tunability, the ability to cope with gaps in training data, and the ability to incorporate local security policies. To counter these limitations, we propose to replace the rigid decision engines in contemporary bot detectors with a more formal Bayesian inference engine. Bottleneck, our prototype implementation, builds confidence in bot infections based on the causal bot lifecycle encoded in a Bayesian network. We evaluate Bottleneck by applying it as a post-processing decision engine on lifecycle events generated by two existing bot detectors (BotHunter and BotFlex) on two independently-collected datasets. Our experimental results show that Bottleneck consistently achieves comparable or better accuracy than the existing rule-based detectors when the test data is similar to the training data. For differing training and test data, Bottleneck, due to its automated learning and inference models, easily surpasses the accuracies of rule-based systems. Moreover, Bottleneck’s stochastic nature allows its accuracy to be tuned with respect to organizational needs. Extending Bottleneck’s Bayesian network into an influence diagram allows for local security policies to be defined within our framework. Lastly, we show that Bottleneck can also be extended to incorporate evidence trustscore for false alarm reduction.

Towards a science of anomaly detection system evasion

A fundamental drawback of current anomaly detection systems (ADSs) is the ability of a skilled attacker to evade detection. This is due to the flawed assumption that an attacker does not have any information about an ADS. Advanced persistent threats that are capable of monitoring network behavior can always estimate some information about ADSs which makes these ADSs susceptible to evasion attacks. Hence in this paper, we first assume the role of an attacker to launch evasion attacks on anomaly detection systems. We show that the ADSs can be completely paralyzed by parameter estimation attacks. We then present a mathematical model to measure evasion margin with the aim to understand the science of evasion due to ADS design. Finally, to minimize the evasion margin, we propose a key-based randomization scheme for existing ADSs and discuss its robustness against evasion attacks. Case studies are presented to illustrate the design methodology and extensive experimentation is performed to corroborate the results.

POSTER: Revisiting anomaly detection system design philosophy

The inherent design of anomaly detection systems (ADSs) make them highly susceptible to evasion attacks and hence their wide-spread commercial deployment has not been witnessed. There are two main reasons for this: 1) ADSs incur high false positives; 2) Are highly susceptible to evasion attacks (false negatives). While efforts have been made to minimize false positives, evasion is still an open problem. We argue that ADSs design is inherently flawed since it relies on the ADSs detection logic and feature space which is trivial to estimate. In information security e.g. cryptographic algorithms (such as DES), security is inherently dependent upon the key and not the algorithm, which makes these systems very robust by rendering evasion computationally infeasible. We believe there is a need to redesign the anomaly detection systems similar to cryptographic systems. We propose to randomize the feature space of an ADS such that it acts as a cryptographic key for the ADS and hence this randomized feature space is used by the ADS logic for detection of anomalies. This would make the evasion of the ADS computationally infeasible for the attacker.

Joint network-host based malware detection using information-theoretic tools

In this paper, we propose two joint network-host based anomaly detection techniques that detect self-propagating malware in real-time by observing deviations from a behavioral model derived from a benign data profile. The proposed malware detection techniques employ perturbations in the distribution of keystrokes that are used to initiate network sessions. We show that the keystrokes’ entropy increases and the session-keystroke mutual information decreases when an endpoint is compromised by a self-propagating malware. These two types of perturbations are used for real-time malware detection. The proposed malware detection techniques are further compared with three prominent anomaly detectors, namely the maximum entropy detector, the rate limiting detector and the credit-based threshold random walk detector. We show that the proposed detectors provide considerably higher accuracy with almost 100% detection rates and very low false alarm rates.