Monowar H. Bhuyan

Network Anomaly Detection: Methods, Systems and Tools

Network anomaly detection is an important and dynamic research area. Many network intrusion detection methods and systems (NIDS) have been proposed in the literature. In this paper, we provide a structured and comprehensive overview of various facets of network anomaly detection so that a researcher can become quickly familiar with every aspect of network anomaly detection. We present attacks normally encountered by network intrusion detection systems. We categorize existing network anomaly detection methods and systems based on the underlying computational techniques used. Within this framework, we briefly describe and compare a large number of network anomaly detection methods and systems. In addition, we also discuss tools that can be used by network defenders and datasets that researchers in network anomaly detection can use. We also highlight research directions in network anomaly detection.

Detecting Distributed Denial of Service Attacks: Methods, Tools and Future Directions

The minimal processing and best-e↵ort forwarding of any packet, malicious or not, was the prime concern when the Internet was designed. This architecture creates an unregulated network path, which can be exploited by any cyber attacker motivated by revenge, prestige, politics or money. Denial-of-service (DoS) attacks exploit this to target critical Web services [1, 2, 3, 4, 5]. This type of attack is intended to make a computer resource unavailable to its legitimate users. Denial of service attack programs have been around for many years. Old single source attacks are now countered easily by many defense mechanisms and the source of these attacks can be easily rebu↵ed or shut down with improved tracking capabilities. However, with the astounding growth of the Internet during the last decade, an increasingly large number of vulnerable systems are now available to attackers. Attackers can now employ a large number of these vulnerable hosts to launch an attack instead of using a single server, an approach which is not very e↵ective and detected easily. A distributed denial of service (DDoS) attack [1, 6] is a large-scale, coordinated attack on the availability of services of a victim system or network resources, launched indirectly through many compromised computers on the Internet. The first well-documented DDoS attack appears to have occurred in August 1999, when a DDoS tool called Trinoo was deployed in at least 227 systems, to flood a single University of Minnesota computer, which was knocked down for more than two days1. The first largescale DDoS attack took place on February 20001. On February 7, Yahoo! was the victim of a DDoS attack during which its Internet portal was inaccessible for three hours. On February 8, Amazon, Buy.com, CNN and eBay were all hit by DDoS attacks that caused them to either stop functioning completely or slowed them down significantly1. DDoS attack networks follow two types of architectures: the Agent-Handler architecture and the Internet Relay Chat (IRC)-based architecture as discussed by [7]. The Agent-Handler architecture for DDoS attacks is comprised of clients, handlers, and agents (see Figure 6). The attacker communicates with the rest of the DDoS attack system at the client systems. The handlers are often software packages located throughout the Internet that are used by the client to communicate with the agents. Instances of the agent software are placed in the compromised systems that finally carry out the attack. The owners and users of the agent systems are generally unaware of the situation. In the IRC-based DDoS attack architecture, an IRC communication channel is used to connect the client(s) to the agents. IRC

Network attacks: Taxonomy, tools and systems

To prevent and defend networks from the occurrence of attacks, it is highly essential that we have a broad knowledge of existing tools and systems available in the public domain. Based on the behavior and possible impact or severity of damages, attacks are categorized into a number of distinct classes. In this survey, we provide a taxonomy of attack tools in a consistent way for the benefit of network security researchers. This paper also presents a comprehensive and structured survey of existing tools and systems that can support both attackers and network defenders. We discuss pros and cons of such tools and systems for better understanding of their capabilities. Finally, we include a list of observations and some research challenges that may help new researchers in this field based on our hands-on experience.

An empirical evaluation of information metrics for low-rate and high-rate DDoS attack detection

Abstract Distributed Denial of Service (DDoS) attacks represent a major threat to uninterrupted and efficient Internet service. In this paper, we empirically evaluate several major information metrics, namely, Hartley entropy, Shannon entropy, Renyi’s entropy, generalized entropy, Kullback–Leibler divergence and generalized information distance measure in their ability to detect both low-rate and high-rate DDoS attacks. These metrics can be used to describe characteristics of network traffic data and an appropriate metric facilitates building an effective model to detect both low-rate and high-rate DDoS attacks. We use MIT Lincoln Laboratory, CAIDA and TUIDS DDoS datasets to illustrate the efficiency and effectiveness of each metric for DDoS detection.

A multi-step outlier-based anomaly detection approach to network-wide traffic

Abstract Outlier detection is of considerable interest in fields such as physical sciences, medical diagnosis, surveillance detection, fraud detection and network anomaly detection. The data mining and network management research communities are interested in improving existing score-based network traffic anomaly detection techniques because of ample scopes to increase performance. In this paper, we present a multi-step outlier-based approach for detection of anomalies in network-wide traffic. We identify a subset of relevant traffic features and use it during clustering and anomaly detection. To support outlier-based network anomaly identification, we use the following modules: a mutual information and generalized entropy based feature selection technique to select a relevant non-redundant subset of features, a tree-based clustering technique to generate a set of reference points and an outlier score function to rank incoming network traffic to identify anomalies. We also design a fast distributed feature extraction and data preparation framework to extract features from raw network-wide traffic. We evaluate our approach in terms of detection rate, false positive rate, precision, recall and F -measure using several high dimensional synthetic and real-world datasets and find the performance superior in comparison to competing algorithms.

Packet and Flow Based Network Intrusion Dataset

With exponential growth in the number of computer applications and the size of networks, the potential damage that can be caused by attacks launched over the internet keeps increasing dramatically. A number of network intrusion detection methods have been developed with their respective strengths and weaknesses. The majority of research in the area of network intrusion detection is still based on the simulated datasets because of non-availability of real datasets. A simulated dataset cannot represent the real network intrusion scenario. It is important to generate real and timely datasets to ensure accurate and consistent evaluation of methods. We propose a new real dataset to ameliorate this crucial shortcoming. We have set up a testbed to launch network traffic of both attack as well as normal nature using attack tools. We capture the network traffic in packet and flow format. The captured traffic is filtered and preprocessed to generate a featured dataset. The dataset is made available for research purpose.

NADO: network anomaly detection using outlier approach

Anomaly detection, which is an important task in any Network Intrusion Detection System (NIDS), enables discovery of known as well as unknown attacks. Anomaly detection using outlier approach is a successful network anomaly identification technique. In this paper, we describe NADO (Network Anomaly Detection using Outlier approach), an effective outlier based approach for detection of anomalies in networks. It initially clusters the normal data using a variant of the k-means clustering technique for high dimensional data. Then it calculates the reference point from each cluster and builds profiles for each cluster. Finally, it calculates the score for each candidate point w.r.t the reference points and reports as anomaly if it exceeds a user defined threshold value. We evaluate the performance of our approach with KDDcup99 intrusion dataset and other real life datasets. We show that NADO has high detection rate and low false positive rate.

An effective unsupervised network anomaly detection method

In this paper, we present an effective tree based subspace clustering technique (TreeCLUS) for finding clusters in network intrusion data and for detecting unknown attacks without using any labelled traffic or signatures or training. To establish its effectiveness in finding all possible clusters, we perform a cluster stability analysis. We also introduce an effective cluster labelling technique (CLUSLab) to generate labelled dataset based on the stable cluster set generated by TreeCLUS. CLUSLab is a multi-objective technique that exploits an ensemble approach for stability analysis of the clusters generated by TreeCLUS. We evaluate the performance of both TreeCLUS and CLUSLab in terms of several real world intrusion datasets to identify unknown attacks and find that both outperform the competing algorithms.

Network attacks

To prevent and defend networks from the occurrence of attacks, it is highly essential that we have a broad knowledge of existing tools and systems available in the public domain. Based on the behavior and possible impact or severity of damages, attacks are categorized into a number of distinct classes. In this survey, we provide a taxonomy of attack tools in a consistent way for the benefit of network security researchers. This paper also presents a comprehensive and structured survey of existing tools and systems that can support both attackers and network defenders. We discuss pros and cons of such tools and systems for better understanding of their capabilities. Finally, we include a list of observations and some research challenges that may help new researchers in this field based on our hands-on experience.

Review: Network attacks: Taxonomy, tools and systems

To prevent and defend networks from the occurrence of attacks, it is highly essential that we have a broad knowledge of existing tools and systems available in the public domain. Based on the behavior and possible impact or severity of damages, attacks are categorized into a number of distinct classes. In this survey, we provide a taxonomy of attack tools in a consistent way for the benefit of network security researchers. This paper also presents a comprehensive and structured survey of existing tools and systems that can support both attackers and network defenders. We discuss pros and cons of such tools and systems for better understanding of their capabilities. Finally, we include a list of observations and some research challenges that may help new researchers in this field based on our hands-on experience.