Ngai Hang Chan

Data mining meets performance evaluation: fast algorithms for modeling bursty traffic

Network, Web, and disk I/O traffic are usually bursty and self-similar and therefore cannot be modeled adequately with Poisson arrivals. However, we wish to model these types of traffic and generate realistic traces, because of obvious applications for disk scheduling, network management, and Web server design. Previous models (like fractional Brownian motion and FARIMA, etc.) tried to capture the burstiness. However, the proposed models either require too many parameters to fit and/or require prohibitively large (quadratic) time to generate large traces. We propose a simple, parsimonious method, the b-model, which solves both problems: it requires just one parameter, and can easily generate large traces. In addition, it has many more attractive properties: (a) with our proposed estimation algorithm, it requires just a single pass over the actual trace to estimate b. For example, a one-day-long disk trace in milliseconds contains about 86 Mb data points and requires about 3 minutes for model fitting and 5 minutes for generation. (b) The resulting synthetic traces are very realistic: our experiments on real disk and Web traces show that our synthetic traces match the real ones very well in terms of queuing behavior.

Inference for Near-Integrated Time Series with Infinite Variance

Abstract An autoregressive time series is said to be near-integrated (nearly nonstationary) if some of its characteristic roots are close to the unit circle. Statistical inference for the least squares estimators of near-integrated AR(1) models has been under rigorous study recently both in the statistics and econometric literatures. Although classical asymptotics are no longer available, through the study of weak convergence of stochastic processes, one can establish the asymptotic theories in terms of simple diffusion processes or Brownian motions. Such results rely heavily on the finiteness of the variance of the noise. When this finite variance condition fails, whereas many physical and economic phenomena are believed to be generated by an infinite variance noise sequence, the aforementioned asymptotics are not applicable. In this article, a unified theory concerning near-integrated autoregressive time series with infinite variance is developed. In particular, when the noise sequence {e t } belongs to...

Priors for unit root models

Abstract A method of assessing an economists subjective prior for a unit root model is given, and applied. The methods extend previous work by allowing a family of piecewise conjugate prior distribution that permit different opinions when ϱ 1. This larger family is still closed under sampling, so it retains the simplicity that is the principal advantage of conjugate analysis.

Estimation and forecasting of long-memory processes with missing values

This paper addresses the issues of maximum likelihood estimation and forecasting of a long-memory time series with missing values. A state-space representation of the underlying long-memory process is proposed. By incorporating this representation with the Kalman filter, the proposed method allows not only for an efficient estimation of an ARFIMA model but also for the estimation of future values under the presence of missing data. This procedure is illustrated through an analysis of a foreign exchange data set. An investment scheme is developed which demonstrates the usefulness of the proposed approach.

Long memory stochastic volatility : A bayesian approach

We propose a simulation-based Bayesian approach to the analysis of long memory stochastic volatility models, stationary and nonstationary. The main tool used to reduce the likelihood function to a tractable form is an approximate state-space representation of the model, A data set of stock market returns is analyzed with the proposed method. The approach taken here allows a quantitative assessment of the empirical evidence in favor of the stationarity, or nonstationarity, of the instantaneous volatility of the data.

ON THE NONINVERTIBLE MOVING AVERAGE TIME SERIES WITH INFINITE VARIANCE

The limiting distribution of the least squares estimate of the derived process of a noninvertible and nearly noninvertible moving average model with infinite variance innovations is established as a functional of a Levy process. The form of the limiting law depends on the initial value of the innovation and the stable index α. This result enables one to perform asymptotic testing for the presence of a unit root for a noninvertible moving average model through the constructed derived process under the null hypothesis. It provides not only a parallel analog of its autoregressive counterparts, but also a useful alternative to determine “over-differencing” for time series that exhibit heavy-tailed phenomena.

On the Use of Canonical Correlation Analysis in Testing Common Trends

Motivated by the asymptotic uncorrelatedness between the stationary and nonstationary components of a vector time series, a statistic is constructed from the canonical correlations of these components to test for the number of common trends and, hence, the presence of co-integration. For univariate series, such a test statistic possesses direct relationships with the classical Dickey-Fuller test. An iterative testing procedure is then proposed which can handle unit roots of higher multiplicities as well as seasonal co-integrations. In applications, both bootstrap and simulation are used to obtain the empirical critical values of the test statistic. The proposed procedure is illustrated by two real examples.