Paulo Salvador

Multiscale Fitting Procedure Using Markov Modulated Poisson Processes

This paper proposes a parameter fitting procedure using Markov Modulated Poisson Processes (MMPPs) that leads to accurate estimates of queuing behavior for network traffic exhibiting long-range dependence behavior. The procedure matches both the autocovariance and marginal distribution of the counting process. A major feature is that the number of states is not fixed a priori, and can be adapted to the particular trace being modeled. The MMPP is constructed as a superposition of L 2-MMPPs and one M-MMPP. The 2-MMPPs are designed to match the autocovariance and the M-MMPP to match the marginal distribution. Each 2-MMPP models a specific time-scale of the data. The procedure starts by approximating the autocovariance by a weighted sum of exponential functions that model the autocovariance of the 2-MMPPs. The autocovariance tail can be adjusted to capture the long-range dependence characteristics of the traffic, up to the time-scales of interest to the system under study. The procedure then fits the M-MMPP parameters in order to match the marginal distribution, within the constraints imposed by the autocovariance matching. The number of states is also determined as part of this step. The final MMPP with M2L states is obtained by superposing the L 2-MMPPs and the M-MMPP. We apply the inference procedure to traffic traces exhibiting long-range dependence and evaluate its queuing behavior through simulation. Very good results are obtained, both in terms of queuing behavior and number of states, for the traces used, which include the well-known Bellcore traces.

Modeling IP traffic: joint characterization of packet arrivals and packet sizes using BMAPs

This paper proposes a traffic model and a parameter fitting procedure that are capable of achieving accurate prediction of the queuing behavior for IP traffic exhibiting long-range dependence. The modeling process is a discrete-time batch Markovian arrival process (dBMAP) that jointly characterizes the packet arrival process and the packet size distribution. In the proposed dBMAP, packet arrivals occur according to a discrete-time Markov modulated Poisson process (dMMPP) and each arrival is characterized by a packet size with a general distribution that may depend on the phase of the dMMPP. The fitting procedure is designed to provide a close match of both the autocovariance and the marginal distribution of the packet arrival process, using a dMMPP; a packet size distribution is fitted individually to each state of the dMMPP. A major feature of the procedure is that the number of states of the fitted dBMAP is not fixed a priori; it is determined as part of the procedure itself. In this way, the procedure allows establishing a compromise between the accuracy of the fitting and the number of parameters, while maintaining a low computational complexity.We apply the inference procedure to several traffic traces exhibiting long-range dependence. Very good results were obtained since the fitted dBMAPs match closely the autocovariance, the marginal distribution and the queuing behavior of the measured traces. Our results also show that ignoring the packet size distribution and its correlation with the packet arrival process can lead to large errors in terms of queuing behavior.

Robust feature selection and robust PCA for internet traffic anomaly detection

Robust statistics is a branch of statistics which includes statistical methods capable of dealing adequately with the presence of outliers. In this paper, we propose an anomaly detection method that combines a feature selection algorithm and an outlier detection method, which makes extensive use of robust statistics. Feature selection is based on a mutual information metric for which we have developed a robust estimator; it also includes a novel and automatic procedure for determining the number of relevant features. Outlier detection is based on robust Principal Component Analysis (PCA) which, opposite to classical PCA, is not sensitive to outliers and precludes the necessity of training using a reliably labeled dataset, a strong advantage from the operational point of view. To evaluate our method we designed a network scenario capable of producing a perfect ground-truth under real (but controlled) traffic conditions. Results show the significant improvements of our method over the corresponding classical ones. Moreover, despite being a largely overlooked issue in the context of anomaly detection, feature selection is found to be an important preprocessing step, allowing adaption to different network conditions and inducing significant performance gains.

Classification of Internet users using discriminant analysis and neural networks

The (reliable) classification of Internet users, based on their hourly traffic profile, can be advantageous in several traffic engineering tasks and in the selection of suitable tariffing plans. For example, it can be used to optimize the routing by mixing users with contrasting hourly traffic profiles in the same network resources or to advise users on the tariffing plan that best suits their needs. In this paper we compare the use of discriminant analysis and artificial neural networks for the classification of Internet users. The classification is based on a partition obtained by cluster analysis. We classify the Internet users based on a data set measured at the access network of a Portuguese ISP. Using cluster analysis performed over half of the users (randomly chosen) we have identified three groups of users with similar behavior. The classification methods were applied to the second half of users and the obtained classification results compared with those of cluster analysis performed over the complete set of users. Our findings indicate both discriminant analysis and neural networks are valuable classification procedures, with the former slightly outperforming the latter, for the specific scenario under analysis.

A Botnet Detection System Based on Neural Networks

A concerted fight against botnets is needed in order to avoid them from becoming a serious threat to global security in the forthcoming years. Zombie detection is currently performed at the host and/or network levels, but these options have important drawbacks: antivirus, firewalls and anti-spyware are not effective against this threat because they are not able to detect hosts that are compromised via new or target specific malicious software and were not designed to protect the network from external attacks or vulnerabilities that are already present inside the local area network. To overcome these limitations, we propose a new botnet detection approach based on the identification of traffic patterns: since each network application, whether it is licit or illicit, has a characteristic traffic pattern that can uniquely identify it, the detection framework will rely on an Artificial Neural Network to identify the licit and illicit patterns. After the identification phase, the system will generate alarms to the system administrator, that can trigger the most appropriate security actions, like blocking the corresponding IP addresses, putting them under a deeper surveillance or acting over some suspicious network segment. A general detection framework was developed in order to incorporate the detection methodology itself, as well as the data collection and storage modules and all the necessary management functions. Some performance tests were already carried out on the proposed system and the results obtained show that the system is stable and fast and the detection approach is efficient, since it provides high detection rates with low computational overhead.

Modeling Self-similar Traffic through Markov Modulated Poisson Processes over Multiple Time Scales

In recent years several studies have reported peculiar types of traffic behavior, such as long-range dependence and self-similarity, which can have significant impact on network performance. In this paper we propose a novel traffic model and parameter fitting procedure, based on Markov Modulated Poisson Processes (MMPPs), which is able to capture variability over many time scales, a characteristic of self-similar traffic. The fitting procedure matches the complete distribution at each time scale, and not only some of its moments as it is the case in related proposals. Our results show that the proposed traffic model and parameter fitting procedure closely matches the main characteristics of measured traces over the time scales present in data.

Markovian modelling of internet traffic

This tutorial discusses the suitability of Markovian models to describe IP network traffic that exhibits peculiar scale invariance properties, such as selfsimilarity and long range dependence. Three Markov Modulated Poisson Processes (MMPP), and their associated parameter fitting procedures, are proposed to describe the packet arrival process by incorporating these peculiar behaviors in their mathematical structure and parameter inference procedures. Since an accurate modeling of certain types of IP traffic requires matching closely not only the packet arrival process but also the packet size distribution, we also discuss a discrete-time batch Markovian arrival process that jointly characterizes the packet arrival process and the packet size distribution. The accuracy of the fitting procedures is evaluated by comparing the long range dependence properties, the probability mass function at each time scale and the queuing behavior corresponding to measured and synthetic traces generated from the inferred models.

Detection of Illicit Network Activities Based on Multivariate Gaussian Fitting of Multi-Scale Traffic Characteristics

Methodologies that are able to accurately identify Internet attacks and intrusions are becoming vital to assure secure on-line communications. Such methodologies must be able to act under strict confidentiality restrictions, such as traffic encryption, which are increasingly used in current communication environments. Proposed approaches must be able to analyze the traffic profiles in order to determine if the network is under a security attack or not. In this paper, we propose an approach that was designed to cope with the previously mentioned restrictions and is able to perform a pseudo real-time identification of illicit traffic: by passively analyzing some statistical properties of captured IP traffic, the methodology calculates and analyses the multi-scale properties of each traffic flow in order to infer multi-dimensional probability distributions for each one of studied protocols, allowing the analysis of the correlation between the values of several dimensions. By doing this, more exact approximations are inferred, enabling the assignment of unknown traffic to the corresponding protocol and the identification of illicit flows. The results obtained prove that the proposed technique can accurately classify Internet traffic and identify illicit flows on a quasi real-time basis. Besides, the fact that the analysis is performed over statistics that were collected for each traffic flow makes it suitable for scenarios where the packet payload is not accessible.

Study on geographical distribution and availability of BitTorrent peers sharing video files

Peer-to-Peer (P2P) file sharing systems are becoming part of the daily life of content consumers. The traffic generated by P2P systems, and particularly BitTorrent, represents the major portion of the global Internet traffic, largely overtaking the traffic share of the World Wide Web. Nowadays, we see P2P systems evolving towards a Video on Demand (VoD) platform. One of the main challenges of these systems is to efficiently share data in the dynamic and continuously evolving Ad-Hoc networks of users. This ability heavily depends on the peerspsila locations and characteristics on this type of networks. In this paper, we use the current BitTorrent system to evaluate the peer-level characteristics of users sharing video contents. These characteristics include geographical location, peers availability and peers rdquodistancerdquo, measured in terms of the round trip time. From the results obtained, we conclude that peers sharing video content present some unexpected locality features that, if conveniently exploited, could greatly benefit VoD P2P systems both in technological and service marketing terms.

Modeling multifractal traffic with stochastic L-systems

This paper proposes a novel multifractal traffic model, and an associated parameter fitting procedure, based on stochastic L-systems, which were introduced by biologist A. Lindenmayer (1968) as a method to model plant growth. We provide a detailed comparison with a related multifractal model based on conservative cascades. Our results, that include applying the fitting procedure to real observed data with multifractal scaling behavior, show that L-system based models can achieve excellent fitting performance in terms of first and second order statistics and queuing behavior.