Félix Iglesias

Analysis of network traffic features for anomaly detection

Anomaly detection in communication networks provides the basis for the uncovering of novel attacks, misconfigurations and network failures. Resource constraints for data storage, transmission and processing make it beneficial to restrict input data to features that are (a) highly relevant for the detection task and (b) easily derivable from network observations without expensive operations. Removing strong correlated, redundant and irrelevant features also improves the detection quality for many algorithms that are based on learning techniques. In this paper we address the feature selection problem for network traffic based anomaly detection. We propose a multi-stage feature selection method using filters and stepwise regression wrappers. Our analysis is based on 41 widely-adopted traffic features that are presented in several commonly used traffic data sets. With our combined feature selection method we could reduce the original feature vectors from 41 to only 16 features. We tested our results with five fundamentally different classifiers, observing no significant reduction of the detection performance. In order to quantify the practical benefits of our results, we analyzed the costs for generating individual features from standard IP Flow Information Export records, available at many routers. We show that we can eliminate 13 very costly features and thus reducing the computational effort for on-line feature generation from live traffic observations at network nodes.

Entropy-Based Characterization of Internet Background Radiation

Network security requires real-time monitoring of network traffic in order to detect new and unexpected attacks. Attack detection methods based on deep packet inspection are time consuming and costly, due to their high computational demands. This paper proposes a fast, lightweight method to distinguish different attack types observed in an IP darkspace monitor. The method is based on entropy measures of traffic-flow features and machine learning techniques. The explored data belongs to a portion of the Internet background radiation from a large IP darkspace, i.e., real traffic captures that exclusively contain unsolicited traffic, ongoing attacks, attack preparation activities and attack aftermaths. Results from an in-depth traffic analysis based on packet headers and content are used as a reference to label data and to evaluate the quality of the entropy-based classification. Full IP darkspace traffic captures from a three-week observation period in April, 2012, are used to compare the entropy-based classification with the in-depth traffic analysis. Results show that several traffic types present a high correlation to the respective traffic-flow entropy signals and can even fit polynomial regression models. Therefore, sudden changes in traffic types caused by new attacks or attack preparation activities can be identified based on entropy variations.

DAT detectors: uncovering TCP/IP covert channels by descriptive analytics

Covert channels provide means to conceal information transfer between hosts and bypass security barriers in communication networks. Hidden communication is of paramount concern for governments and companies, because it can conceal data leakage and malware communication, which are crucial building blocks used in cyber crime. We propose detectors based on descriptive analytics of traffic DAT to facilitate revealing network and transport layer covert channels originated from a wide spectrum of published data-hiding techniques. DAT detectors transform communication data into flexible feature vectors that represent traffic by a set of extracted calculations and estimations. For the case of covert channels, the core of the detection is performed by the combined application of autocorrelation calculations and multimodality measures built upon kernel density estimations and Pareto charts. DAT detectors are devised to be embedded as extensions of network intrusion detection systems, being able to perform fast, lightweight analysis of numerous flows. The present paper focuses specifically on TCP/IP traffic and provides suitable classifications of TCP/IP fields and related covert channel techniques from the perspective of the statistical detection. The proposed methodology is evaluated with public traffic datasets as well as covert channels generated according to main techniques described in the related literature. Copyright

Modelling IP darkspace traffic by means of clustering techniques.

An IP darkspace is an unused IP address range. Addresses are announced by routing, but no hosts are attached. Therefore all traffic directed to IP darkspace addresses is unsolicited and usually originates from attacks, attack preparation activities or misconfigurations. Most of the observed traffic belongs to known phenomena (e.g. horizontal scanning targeting a specific port) and is of limited interest to security analysts. But hidden in the vast amount of common attacks, smaller unusual events may indicate new malicious activities. In this paper we present a methodology to distinguish IP darkspace sources with common traffic patterns from sources that show uncommon behavior and may be the origin of novel attacks. For this, we model IP darkspace sources based on clustering techniques. We extract data from one complete month of a large /8 darkspace capture and use a very simple feature vector. Our analysis is purely based on clustering techniques and does not require any pre-knowledge about phenomena in darkspace traffic. We found that about 75% of the darkspace IP sources contributes to a set of very stable clusters, 4% to less stable clusters and 21% to outliers. This allows us to concentrate the effort for searching for new attacks in just 21% of the sources.

Decision Tree Rule Induction for Detecting Covert Timing Channels in TCP/IP Traffic

The detection of covert channels in communication networks is a current security challenge. By clandestinely transferring information, covert channels are able to circumvent security barriers, compromise systems, and facilitate data leakage. A set of statistical methods called DAT (Descriptive Analytics of Traffic) has been previously proposed as a general approach for detecting covert channels. In this paper, we implement and evaluate DAT detectors for the specific case of covert timing channels. Additionally, we propose machine learning models to induce classification rules and enable the fine parameterization of DAT detectors. A testbed has been created to reproduce main timing techniques published in the literature; consequently, the testbed allows the evaluation of covert channel detection techniques. We specifically applied Decision Trees to infer DAT-rules, achieving high accuracy and detection rates. This paper is a step forward for the actual implementation of effective covert channel detection plugins in modern network security devices.

Time-activity footprints in IP traffic

This paper studies the temporal behavior of communication flows in the Internet. Characterization of flows by temporal patterns supports traffic classification and filtering for network management and network security in situations where full packet data is not accessible (e.g., obfuscated or encrypted traffic) or cannot be analyzed due to privacy concerns or resource limitations. In this paper we define a time activity feature vector that describes the temporal behavior of flows. Later, we use cluster analysis to capture the most common time activity patterns in real internet traffic using traces from the MAWI dataset. We discovered a set of seven time-activity footprints and show that 95.3% of the analyzed flows can be characterized based on such footprints, which represent different behaviors for the three main protocols (4 in TCP, 1 in ICMP and 2 in UDP). In addition, we found that the majority of the observed flows consisted of short, one-time bursts. An in-depth inspection revealed, besides some DNS traffic, the preponderance of a large number of scanning, probing, DoS attacks and backscatter traffic in the network. Flows transmitting meaningful data became outliers among short, one-time bursts of unwanted traffic.

Are Network Covert Timing Channels Statistical Anomalies

Covert channels exploit communication protocols to clandestinely transfer information. They enable criminals to hide malicious activities and can be used for secret data exfiltration, malware spreading or for the stealthy establishment of command and control structures. In this paper we study covert timing channels from a statistical perspective and investigate whether they can be identified as anomalies with unsupervised learning methods. We use a testbed to generate covert timing channels based on seven popular techniques and inject them in real captured traffic. Final datasets are analyzed with diverse outlier detection and classification algorithms. Our results show that, based on their statistical properties, covert channels do not occupy low density regions or take extreme values in the problem space, and therefore are not detectable as strong anomalies. However, they present traceable profiles that can be abstracted by supervised learning models. Such findings reveal that facing the detection of novel (and classic) covert timing channels from an anomaly-detection perspective will probably fail or not suffice; instead, they must be identified based on the similarity to known schemes, using supervised and semi-supervised approaches.

Pattern Discovery in Internet Background Radiation

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

Crucial pitfall of DPA Contest V4.2 implementation

Differential power analysis (DPA) is a powerful side-channel key recovery attack that efficiently breaks cryptographic algorithm implementations. In order to prevent these types of attacks, hardware designers and software programmers make use of masking and hiding techniques. DPA contest is an international framework that allows researchers to compare their power analysis attacks under the same conditions. The latest version of DPA contest, denoted as V4.2, provides an improved implementation of the rotating S-box masking scheme where low-entropy boolean masking is combined with the shuffling technique to protect Advanced Encryption Standard implementation on a smart card. The improvements were designed based on the awareness of implementation lacks analyzed from attacks carried out during the previous DPA contest V4. Therefore, this new approach is devised to resist most of the proposed attacks to the original rotating S-box masking implementation. In this paper, we investigate the security of this new implementation in practice. Our analysis, focused on exploiting the first-order leakage, discovered important lacks. The main vulnerability observed is that an adversary can mount a standard DPA attack aimed at the S-box output in order to recover the whole secret key even when a shuffling technique is used. We tested this observation on a public dataset and implemented a successful attack that revealed the secret key using only 35 power traces. Copyright