Jin Cao

On the nonstationarity of Internet traffic

Traffic variables on an uncongested Internet wire exhibit a pervasive nonstationarity. As the rate of new TCP connections increases, arrival processes (packet and connection) tend locally toward Poisson, and time series variables (packet sizes, transferred file sizes, and connection round-trip times) tend locally toward independent. The cause of the nonstationarity is superposition: the intermingling of sequences of connections between different source-destination pairs, and the intermingling of sequences of packets from different connections. We show this empirically by extensive study of packet traces for nine links coming from four packet header databases. We show it theoretically by invoking the mathematical theory of point processes and time series. If the connection rate on a link gets sufficiently high, the variables can be quite close to Poisson and independent; if major congestion occurs on the wire before the rate gets sufficiently high, then the progression toward Poisson and independent can be arrested for some variables.

Bandwidth Estimation for Best-Effort Internet Traffic

A fundamental problem of Internet traffic engineering is bandwidth estimation: determining the bandwidth (bits/sec) required to carry traffic with a specific bit rate (bits/sec) offered to an Internet link and satisfy quality-of-service requirements. The traffic is packets of varying sizes arriving for transmission on the link. Packets can queue up and are dropped if the queue size (bits) is bigger than the size of the buffer (bits) for the queue. For the predominant traffic on the Internet, best-effort traffic, quality metrics are the packet loss (fraction of lost packets), a queueing delay (sec), and the delay probability (probability of a packet exceeding the delay). This article presents an introduction to bandwidth estimation and a solution to the problem for best-effort traffic for the case where the quality criteria specify negligible packet loss. The solution is a simple statistical model: (1) a formula for the bandwidth as a function of the delay, the delay probability, the traffic bit rate, and the mean number of active host-pair connections of the traffic, and (2) a random error term. The model is built and validated using queueing theory and extensive empirical study; it is valid for traffic with 64 host pair connections or more, which is about 1 megabit/sec of traffic. The model provides for Internet best-effort traffic what the Erlang delay formula provides for queueing systems with Poisson arrivals and i.i.d. exponential service times.

The S-Net System for Internet Packet Streams: Strategies for Stream Analysis and System Architecture

The traffic on an Internet link is a packet stream: packets of varying sizes arriving for transmission on the link. Each packet has an arrival time, and contained within the packet are headers that carry many critical variables. Packet traces, which consist of captured headers and measurements of the arrival times, convey substantial information about the Internet—security, usage, network performance, and the performance of engineering protocols. This article discusses strategies for the analysis of very large databases of packet traces, and the architecture of a software system that facilitates the use of these strategies. The system has a pipeline: (1) raw packet traces; (2) a database with objects tailored to ensuing analyses; and (3) an environment with tools for data analysis: statistical methods, model fitting, and visualization. The pipeline addresses the full set of tasks in the study of packet streams, from the initial processing of raw packet traces to the final output, often a visual display. S-Net—an extensible, open-source software implementation of this architecture—is based on the R implementation of the S language for graphics and data analysis, and has been developed on Linux.