Philippe Owezarski

Non-Gaussian and Long Memory Statistical Characterizations for Internet Traffic with Anomalies

The goals of the present contribution are twofold. First, we propose the use of a non-Gaussian long-range dependent process to model Internet traffic aggregated time series. We give the definitions and intuition behind the use of this model. We detail numerical procedures that can be used to synthesize artificial traffic exactly following the model prescription. We also propose original and practically effective procedures to estimate the corresponding parameters from empirical data. We show that this empirical model relevantly describes a large variety of Internet traffic, including both regular traffic obtained from public reference repositories and traffic containing legitimate (flash crowd) or illegitimate (DDoS attack) anomalies. We observe that the proposed model accurately fits the data for a wide range of aggregation levels. The model provides us with a meaningful multiresolution (i.e., aggregation level dependent) statistics to characterize the traffic: the evolution of the estimated parameters with respect to the aggregation level. It opens the track to the second goal of the paper: anomaly detection. We propose the use of a quadratic distance computed on these statistics to detect the occurrences of DDoS attack and study the statistical performance of these detection procedures. Traffic with anomalies was produced and collected by us so as to create a controlled and reproducible database, allowing for a relevant assessment of the statistical performance of the proposed (modeling and detection) procedures

Design and Deployment of a Passive Monitoring Infrastructure

This paper presents the architecture of a passive monitoring system installed within the Sprint IP backbone network. This system differs from other packet monitoring systems in that it collects packet-level traces from multiple links within the network and provides the capability to correlate the data using highly accurate GPS timestamps. After a thorough description of the monitoring systems, we demonstrate the systems capabilities and the diversity of the results that can be obtained from the collected data. These results include workload characterization, packet size analysis, and packet delay incurred through a single backbone router. We conclude with lessons learned from the development of the monitoring infrastructure and present future research goals.

Modeling Internet backbone traffic at the flow level

Our goal is to design a traffic model for noncongested Internet backbone links, which is simple enough to be used in network operation, while being as general as possible. The proposed solution is to model the traffic at the flow level by a Poisson shot-noise process. In our model, a flow is a generic notion that must be able to capture the characteristics of any kind of data stream. We analyze the accuracy of the model with real traffic traces collected on the Sprint Internet protocol (IP) backbone network. Despite its simplicity, our model provides a good approximation of the real traffic observed in the backbone and of its variation. Finally, we discuss the application of our model to network design and dimensioning.

Unsupervised Network Intrusion Detection Systems: Detecting the Unknown without Knowledge

Traditional Network Intrusion Detection Systems (NIDSs) rely on either specialized signatures of previously seen attacks, or on expensive and difficult to produce labeled traffic datasets for user-profiling to hunt out network attacks. Despite being opposite in nature, both approaches share a common downside: they require the knowledge provided by an external agent, either in terms of signatures or as normal-operation profiles. In this paper we present UNIDS, an Unsupervised Network Intrusion Detection System capable of detecting unknown network attacks without using any kind of signatures, labeled traffic, or training. UNIDS uses a novel unsupervised outliers detection approach based on Sub-Space Clustering and Multiple Evidence Accumulation techniques to pin-point different kinds of network intrusions and attacks such as DoS/DDoS, probing attacks, propagation of worms, buffer overflows, illegal access to network resources, etc. We evaluate UNIDS in three different traffic datasets, including the well-known KDD99 dataset as well as real traffic traces from two operational networks. We particularly show the ability of UNIDS to detect unknown attacks, comparing its performance against traditional misuse-detection-based NIDSs. In addition, we also evidence the supremacy of our outliers detection approach with respect to different previously used unsupervised detection techniques. Finally, we show that the algorithms used by UNIDS are highly adapted for parallel computation, which permits to drastically reduce the overall analysis time of the system.

A flow-based model for internet backbone traffic

Our goal is to design a traffic model for uncongested IP backbone links that is simple enough to be used in network operation, and that is protocol and application agnostic in order to be as general as possible. The proposed solution is to model the traffic at the flow level by a Poisson shot-noise process. In our model, a flow is a generic notion that must be able to capture the characteristics of any kind of data stream. We analyze the accuracy of the model with real traffic traces collected on the Sprint IP backbone network. Despite its simplicity, our model provides a good approximation of the real traffic observed in the backbone and of its variation. Finally, we discuss three applications of our model to network design and management.

Internet Traffic Characterization – An Analysis of Traffic Oscillations

Internet traffic has been changing a lot since few years in particular with the arrival of new P2P applications for exchanging audio files or movies and nowadays the knowledge we have on it is quite limited. Especially, new applications and new traffic are creating a lot of troubles and performance issues. Based on some traffic traces captured in the framework of the METROPOLIS network monitoring project, this paper exhibits the highly oscillating nature of Internet traffic, thus explaining why it is almost impossible nowadays to guarantee a stable QoS in the Internet, and also that such oscillations provoke a huge decrease of the global network QoS and performance. This paper then demonstrates that traffic oscillations can be characterized by the Hurst (LRD) parameter. In particular, this demonstration relies on a comparative study of Internet traffic depending on the transport protocol used to generate it. It is then shown that using TFRC – a congestion control mechanism whose purpose deals with providing smooth sending rates for stream oriented applications – instead of TCP, makes traffic oscillations and LRD almost disappear. This result, i.e. limiting as much as possible the oscillations of traffic sources in the Internet, then gives research directions for future Internet protocols and architectures.

OSNT: Open Source Network Tester

Despite network monitoring and testing being critical for computer networks, current solutions are both extremely expensive and inflexible. Into this lacuna we launch the Open Source Network Tester, a fully open source traffic generator and capture system. Our prototype implementation on the NetFPGA-10G supports 4 × 10 Gb/s traffic generation across all packet sizes, and traffic capture is supported up to 2 × 10Gb/s with naïve host software. Our system implementation provides methods for scaling and coordinating multiple generator/capture systems, and supports 6.25 ns timestamp resolution with clock drift and phase coordination maintained by GPS input. Additionally, our approach has demonstrated lower-cost than comparable commercial systems while achieving comparable levels of precision and accuracy; all within an open-source framework extensible with new features to support new applications, while permitting validation and review of the implementation.

UNADA: unsupervised network anomaly detection using sub-space outliers ranking

Current network monitoring systems rely strongly on signature-based and supervised-learning-based detection methods to hunt out network attacks and anomalies. Despite being opposite in nature, both approaches share a common downside: they require the knowledge provided by an expert system, either in terms of anomaly signatures, or as normal-operation profiles. In a diametrically opposite perspective we introduce UNADA, an Unsupervised Network Anomaly Detection Algorithm for knowledge-independent detection of anomalous traffic. UNADA uses a novel clustering technique based on Sub-Space-Density clustering to identify clusters and outliers in multiple low-dimensional spaces. The evidence of traffic structure provided by these multiple clusterings is then combined to produce an abnormality ranking of traffic flows, using a correlation-distance-based approach. We evaluate the ability of UNADA to discover network attacks in real traffic without relying on signatures, learning, or labeled traffic. Additionally, we compare its performance against previous unsupervised detection methods using traffic from two different networks.

Evaluation of active measurement tools for bandwidth estimation in real environment

Available bandwidth - as well as capacity or achievable bandwidth - on a path or a link is one of the very important parameters to measure or estimate in a network: it is of high interest for many networking functions (routing, admission and congestion control, load balancing, etc). Active probing techniques provide the easiest and the more flexible approach, for estimating available bandwidth. In addition, they can be used for different network technologies or structures. Many techniques and tools for available bandwidth estimation appeared recently, but little attention has been given to the accuracy of the estimated values in the real Internet, most of previous studies focusing on validating the accuracy of these tools on local platform. Therefore, this paper deals with evaluating the accuracy of active estimation tools in the real wide area Internet. We use passive monitoring tools for this purpose. We then built a platform combining active and passive equipments, and define a methodology for evaluating active probing techniques using passive tools. The passive evaluation relies on DAG system cards that represent references for such kind of measurements. This paper then discusses the results we got in the different experiments with different tools. In particular, we use traffic generators for changing the characteristics of the traffic on the Internet paths, which we are making our measurements on. It is useful for analyzing the accuracy of active estimation tools according to network and traffic conditions.

On the impact of monitoring router energy consumption for greening the Internet

Research in the field of green-networking is raising more and more interest, in particular driven by energy saving purposes. The global Internet and its thousands of equipments consume an enormous energy amount, have an impact on global warming. In addition, nobody has a precise idea about what the Internet - or at least one of its AS (Autonomous System) - consumes. It is obvious designing new routing or management strategies for greening the Internet relies on an initial study of the energy consumption of network equipments at large, and routers on a more focuses way. That is why we study in this paper the power consumption of a router depending on several factors as the traffic rate it has to compute, and its configuration (in particular depending on queue management policy). This work then aims to establish an effective method to measure and analyze the power consumption of a router, as well as to provide data from a real router. This work was motivated by the fact that very little data on the power consumption of network devices is available, despite its huge importance for greening network communication. Based on these first results, a discussion is started on how it would be possible to change routing and management strategies and policies in the Internet for saving energy.