Tian Bu

Monitoring Time-Varying Network Streams Using State-Space Models

In one embodiment, a statistical model is generated based on observed data, the observed data being associated with a network device, online parameter fitting is performed on parameters of the statistical model, and for each newly observed data value, a forecast value is generated based on the statistical model, the forecast value being a prediction of a next observed data value, a forecasting error is generated based on the forecast value and the newly observed data value, and whether the data of the network stream is abnormal is determined based on a log likelihood ratio test of the forecasting errors and a threshold value.

A Quasi-Likelihood Approach for Accurate Traffic Matrix Estimation in a High Speed Network

Knowing the traffic matrix, i.e., packet/byte counts between pairs of nodes in a network, is important for network management. The main challenges for accurate traffic matrix estimation in a high speed network are the computation and memory limitations. In this paper, we propose a novel algorithm for traffic matrix estimation that can yield accurate estimates whereas uses small memory and per packet update overhead. Our algorithm constructs a compact probabilistic traffic digest at each network node, and derives a Quasi Maximum Likelihood Estimate (Quasi-MLE) of the traffic matrix by correlating the traffic digests received at a central location. Our new approach is highly efficient, requiring no prior knowledge of the exact packet size distributions. We derive accurate approximation of the relative error distribution of our estimate. For an origin- destination (OD) pair (o,d), we show that by using an array of size M for each traffic digest at o and d, the relative estimation standard error is O(M-1/2(sigmao +sigmad)1/2), where sigmao,sigmad are the noise-to-signal ratios, defined as the ratios of non-OD packet/byte counts to OD packet/byte counts at the origin and destination. This is superior to the state-of-the-art algorithms, especially for large sigmao and sigmad, where the estimation is more challenging. We further demonstrate the effectiveness of our approach using both model and real Internet trace-driven simulations.

Tracking Quantiles of Network Data Streams with Dynamic Operations

Quantiles are very useful in characterizing the data distribution of an evolving dataset in the process of data mining or network monitoring. The method of Stochastic Approximation (SA) tracks quantiles online by incrementally deriving and updating local approximations of the underly distribution function at the quantiles of interest. In this paper, we propose a generalization of the SA method for quantile estimation that allows not only data insertions, but also dynamic data operations such as deletions and updates.