Darryl Veitch

Wavelet analysis of long-range-dependent traffic

A wavelet-based tool for the analysis of long-range dependence and a related semi-parametric estimator of the Hurst parameter is introduced. The estimator is shown to be unbiased under very general conditions, and efficient under Gaussian assumptions. It can be implemented very efficiently allowing the direct analysis of very large data sets, and is highly robust against the presence of deterministic trends, as well as allowing their detection and identification. Statistical, computational, and numerical comparisons are made against traditional estimators including that of Whittle. The estimator is used to perform a thorough analysis of the long-range dependence in Ethernet traffic traces. New features are found with important implications for the choice of valid models for performance evaluation. A study of mono versus multifractality is also performed, and a preliminary study of the stationarity with respect to the Hurst parameter and deterministic trends.

A wavelet-based joint estimator of the parameters of long-range dependence

A joint estimator is presented for the two parameters that define the long-range dependence phenomenon in the simplest case. The estimator is based on the coefficients of a discrete wavelet decomposition, improving a wavelet-based estimator of the scaling parameter (Abry and Veitch 1998), as well as extending it to include the associated power parameter. An important feature is its conceptual and practical simplicity, consisting essentially in measuring the slope and the intercept of a linear fit after a discrete wavelet transform is performed, a very fast (O(n)) operation. Under well-justified technical idealizations the estimator is shown to be unbiased and of minimum or close to minimum variance for the scale parameter, and asymptotically unbiased and efficient for the second parameter. Through theoretical arguments and numerical simulations it is shown that in practice, even for small data sets, the bias is very small and the variance close to optimal for both parameters. Closed-form expressions are given for the covariance matrix of the estimator as a function of data length, and are shown by simulation to be very accurate even when the technical idealizations are not satisfied. Comparisons are made against two maximum-likelihood estimators. In terms of robustness and computational cost the wavelet estimator is found to be clearly superior and statistically its performance is comparable. We apply the tool to the analysis of Ethernet teletraffic data, completing an earlier study on the scaling parameter alone.

Multiscale nature of network traffic

The complexity and richness of telecommunications traffic is such that one may despair to find any regularity or explanatory principles. Nonetheless, the discovery of scaling behavior in teletraffic has provided hope that parsimonious models can be found. The statistics of scaling behavior present many challenges, especially in nonstationary environments. In this article, we overview the state of the art in this area, focusing on the capabilities of the wavelet transform as a key tool for unraveling the mysteries of traffic statistics and dynamics.

Long‐range Dependence: Revisiting Aggregation with Wavelets

The aggregation procedure is a natural way to analyse signals which exhibit long-range-dependent features and has been used as a basis for estimation of the Hurst parameter, H. In this paper it is shown how aggregation can be naturally rephrased within the wavelet transform framework, being directly related to approximations of the signal in the sense of a Haar multiresolution analysis. A natural wavelet-based generalization to traditional aggregation is then proposed: ‘a-aggregation’. It is shown that a-aggregation cannot lead to good estimators of H, and so a new kind of aggregation, ‘d-aggregation’, is defined, which is related to the details rather than the approximations of a multiresolution analysis. An estimator of H based on d-aggregation has excellent statistical and computational properties, whilst preserving the spirit of aggregation. The estimator is applied to telecommunications network data.

PC based precision timing without GPS

A highly accurate monitoring solution for active network measurement is provided without the need for GPS, based on an alternative software clock for PCs running Unix. With respect to clock rate, its performance exceeds common GPS and NTP synchronized software clock accuracy. It is based on the TSC register counting CPU cycles and offers a resolution of around 1ns, a rate stability of 0.1PPM equal to that of the underlying hardware, and a processing overhead well under 1µs per timestamp. It is scalable and can be run in parallel with the usual clock. It is argued that accurate rate, and not synchronised offset, is the key requirement of a clock for network measurement. The clock requires an accurate estimation of the CPU cycle period. Two calibration methods which do not require a reference clock at the calibration point are given. To the TSC clock we add timestamping optimisations to create two high accuracy monitors, one based on Linux and the other on Real-Time Linux. The TSC-RT-Linux monitor has offset fluctuations of the order of 1µs. The clock is ideally suited for high precision active measurement.

Self-similar traffic and network dynamics

One of the most significant findings of traffic measurement studies over the last decade has been the observed self-similarity in packet network traffic. Subsequent research has focused on the origins of this self-similarity, and the network engineering significance of this phenomenon. This paper reviews what is currently known about network traffic self-similarity and its significance. We then consider a matter of current research, namely, the manner in which network dynamics (specifically, the dynamics of transmission control protocol (TCP), the predominant transport protocol used in todays Internet) can affect the observed self-similarity. To this end, we first discuss some of the pitfalls associated with applying traditional performance evaluation techniques to highly-interacting, large-scale networks such as the Internet. We then present one promising approach based on chaotic maps to capture and model the dynamics of TCP-type feedback control in such networks. Not only can appropriately chosen chaotic map models capture a range of realistic source characteristics, but by coupling these to network state equations, one can study the effects of network dynamics on the observed scaling behavior We consider several aspects of TCP feedback, and illustrate by examples that while TCP-type feedback can modify the self-similar scaling behavior of network traffic, it neither generates it nor eliminates it.

Cluster processes: a natural language for network traffic

We introduce a new approach to the modeling of network traffic, consisting of a semi-experimental methodology combining models with data and a class of point processes (cluster models) to represent the process of packet arrivals in a physically meaningful way. Wavelets are used to examine second-order statistics, and particular attention is paid to the modeling of long-range dependence and to the question of scale invariance at small scales. We analyze in depth the properties of several large traces of packet data and determine unambiguously the influence of network variables such as arrival patterns, durations, and volumes of transport control protocol (TCP) flows and internal flow structure. We show that session-level modeling is not relevant at the packet level. Our findings naturally suggest the use of cluster models. We define a class where TCP flows are directly modeled, and each model parameter has a direct meaning in network terms, allowing the model to be used to predict traffic properties as networks and traffic evolve. The class has the key advantage of being mathematically tractable, in particular, its spectrum is known and can be readily calculated, its wavelet spectrum deduced, interarrival distributions can be obtained, and it can be simulated in a straightforward way. The model reproduces the main second-order features, and results are compared against a simple black box point process alternative. Discrepancies with the model are discussed and explained, and enhancements are outlined. The elephant and mice view of traffic flows is revisited in the light of our findings.

The packet size dependence of packet pair like methods

Packet-pair based link estimation methods allow the estimation of bottleneck bandwidth in Internet routes. In practice, several complicating effects combine which can seriously distort such estimates. We provide a new delay variation based route model which allows the principles of packet-pair to be formalised and extended. This enables the effect of probe size to be evaluated, downstream noise understood, peak detection recognised as superior to mode or minimum based filtering, and new estimation methods to be proposed and evaluated. Using insight from the governing equations and simulation, it is shown how real measurements made over a 12 hop route can be interpreted. Unexpected additional probe size dependencies were found, inspiring an extension of the route model to include lower layer headers. It is shown how the enhanced model accounts very well for the observed dependencies, allowing more accurate estimates and a greater understanding of the role of cross traffic.

Bridging router performance and queuing theory

This paper provides an authoritative knowledge of through-router packet delays and therefore a better understanding of data network performance. Thanks to a unique experimental setup, we capture all packets crossing a router for 13 hours and present detailed statistics of their delays. These measurements allow us to build the following physical model for router performance: each packet experiences a minimum router processing time before entering a fluid output queue. Although simple, this model reproduces the router behaviour with excellent accuracy and avoids two common pitfalls. First we show that in-router packet processing time accounts for a significant portion of the overall packet delay and should not be neglected. Second we point out that one should fully understand both link and physical layer characteristics to use the appropriate bandwidth value.Focusing directly on router performance, we provide insights into system busy periods and show precisely how queues build up inside a router. We explain why current practices for inferring delays based on average utilization have fundamental problems, and propose an alternative solution to directly report router delay information based on busy period statistics.

Heavy traffic analysis of a storage model with long range dependent On/Off sources

We consider a fluid queueing system with infinite storage capacity and constant output rate offered a superposition ofN identical On/Off sources, where the ratio of input to output rate is small. The On and/or Off periods have heavy tailed distributions with infinite variance, giving rise to Long Range Dependence in the arrival process. In the limit of a large number of sources and high load, it is shown that the tail of the stationary queue content distribution is Weibullian, implying much larger queue contents than in the classical case of exponential tails. Noting that similar results were recently found by I. Norros for a storage system input by a Fractional Brownian Motion, we then show how the two models are related, thus providing a further physical motivation for the Fractional Brownian Motion model.