Anthony Brockwell

Storage device performance prediction with CART models

Storage device performance prediction is a key element of self-managed storage systems. The paper explores the application of a machine learning tool, CART (classification and regression trees) models, to storage device modeling. Our approach predicts a devices performance as a function of input workloads, requiring no knowledge of the device internals. We propose two uses of CART models: one that predicts per-request response times (and then derives aggregate values); one that predicts aggregate values directly from workload characteristics. After being trained on the device in question, both provide accurate black-box models across a range of test traces from real environments. Experiments show that these models predict the average and 90th percentile response time with a relative error as low as 19%, when the training workloads are similar to the testing workloads, and interpolate well across different workloads.

Parallel Markov chain Monte Carlo Simulation by Pre-Fetching

In recent years, parallel processing has become widely available to researchers. It can be applied in an obvious way in the context of Monte Carlo simulation, but techniques for “parallelizing” Markov chain Monte Carlo (MCMC) algorithms are not so obvious, apart from the natural approach of generating multiple chains in parallel. Although generation of parallel chains is generally the easiest approach, in cases where burn-in is a serious problem, it is often desirable to use parallelization to speed up generation of a single chain. This article briefly discusses some existing methods for parallelization of MCMC algorithms, and proposes a new “pre-fetching” algorithm to parallelize generation of a single chain.

Identification of Regeneration Times in MCMC Simulation, With Application to Adaptive Schemes

Regeneration is a useful tool in Markov chain Monte Carlo simulation because it can be used to side-step the burn-in problem and to construct better estimates of the variance of parameter estimates themselves. It also provides a simple way to introduce adaptive behavior into a Markov chain, and to use parallel processors to build a single chain. Regeneration is often difficult to take advantage of because, for most chains, no recurrent proper atom exists, and it is not always easy to use Nummelins splitting method to identify regeneration times. This article describes a constructive method for generating a Markov chain with a specified target distribution and identifying regeneration times. As a special case of the method, an algorithm which can be “wrapped” around an existing Markov transition kernel is given. In addition, a specific rule for adapting the transition kernel at regeneration times is introduced, which gradually replaces the original transition kernel with an independence-sampling Metropolis-Hastings kernel using a mixture normal approximation to the target density as its proposal density. Computational gains for the regenerative adaptive algorithm are demonstrated in examples.

A Gridding Method for Bayesian Sequential Decision Problems

This article introduces a numerical method for finding optimal or approximately optimal decision rules and corresponding expected losses in Bayesian sequential decision problems. The method, based on the classical backward induction method, constructs a grid approximation to the expected loss at each decision time, viewed as a function of certain statistics of the posterior distribution of the parameter of interest. In contrast with most existing techniques, this method has a computation time which is linear in the number of stages in the sequential problem. It can also be applied to problems with insufficient statistics for the parameters of interest. Furthermore, it is well-suited to be implemented using parallel processors.

Adaptive, unsupervised stream mining

Abstract.Sensor devices and embedded processors are becoming widespread, especially in measurement/monitoring applications. Their limited resources (CPU, memory and/or communication bandwidth, and power) pose some interesting challenges. We need concise, expressive models to represent the important features of the data and that lend themselves to efficient estimation. In particular, under these severe constraints, we want models and estimation methods that (a) require little memory and a single pass over the data, (b) can adapt and handle arbitrary periodic components, and (c) can deal with various types of noise. We propose \ensuremath{\mathrm{AWSOM}} (Arbitrary Window Stream mOdeling Method), which allows sensors in remote or hostile environments to efficiently and effectively discover interesting patterns and trends. This can be done automatically, i.e., with no prior inspection of the data or any user intervention and expert tuning before or during data gathering. Our algorithms require limited resources and can thus be incorporated into sensors - possibly alongside a distributed query processing engine [10,6,27]. Updates are performed in constant time with respect to stream size using logarithmic space. Existing forecasting methods (SARIMA, GARCH, etc.) and “traditional” Fourier and wavelet analysis fall short on one or more of these requirements. To the best of our knowledge, \ensuremath{\mathrm{AWSOM}} is the first framework that combines all of the above characteristics. Experiments on real and synthetic datasets demonstrate that \ensuremath{\mathrm{AWSOM}} discovers meaningful patterns over long time periods. Thus, the patterns can also be used to make long-range forecasts, which are notoriously difficult to perform. In fact, \ensuremath{\mathrm{AWSOM}} outperforms manually set up autoregressive models, both in terms of long-term pattern detection and modeling and by at least 10 x in resource consumption.

Modelling the Dynamic Dependence Structure in Multivariate Financial Time Series

The dependence structure in multivariate financial time series is of great importance in portfolio management. By studying daily return histories of 17 exchange-traded index funds, we identify important features of the data, and we propose two new models to capture these features. The first is an extension of the multivariate BEKK (Baba, Engle, Kraft, Kroner) model, which includes a multivariate t-type error distribution with different degrees of freedom. We demonstrate that this error distribution is able to accommodate different levels of heavy-tailed behavior and thus provides a better fit than models based on a multivariate t-with a common degree of freedom. The second model is copula based, and can be regarded as an extension of the standard and the generalized dynamic conditional correlation model to a Student copula. Model comparison is carried out using criteria including the Akaike information criteria and Bayesian information criteria. We also evaluate the two models from an asset-allocation perspective using a three-asset portfolio as an example, constructing optimal portfolios based on the Markowitz theory. Our results indicate that, for our data, the proposed models both outperform the standard BEKK model, with the copula model performing better than the extension of the BEKK model.

A Strong Law of Large Numbers for Strongly Mixing Processes

We prove a strong law of large numbers for a class of strongly mixing processes. Our result rests on recent advances in understanding of concentration of measure. It is simple to apply and gives finite-sample (as opposed to asymptotic) bounds, with readily computable rate constants. In particular, this makes it suitable for analysis of inhomogeneous Markov processes. We demonstrate how it can be applied to establish an almost-sure convergence result for a class of models that includes as a special case a class of adaptive Markov chain Monte Carlo algorithms.

Modeling phase spectra using gaussian mixture models for human face identification

It has been established that information distinguishing one human face from another is contained to a large extent in the Fourier domain phase component of the facial image. However, to date, formal statistical models for this component have not been deployed in face recognition tasks. In this paper we introduce a model-based approach using Gaussian mixture models (GMM) for the phase component for performing human identification. Classification and verification are performed using a MAP estimate and we show that we are able to achieve identification error rates as low as 2% and verification error rates as low as 0.3% on a database with 65 individuals with extreme illumination variations. The proposed method is easily able to deal with other distortions such as expressions and poses, and hence this establishes its robustness to intra-personal variations. A potential use of the method in illumination normalization is also discussed.

A class of models for aggregated traffic volume time series

The development of time series models for traffic volume data constitutes an important step in constructing automated tools for the management of computing infrastructure resources. We analyse two traffic volume time series: one is the volume of hard disc activity, aggregated into half-hour periods, measured on a workstation, and the other is the volume of Internet requests made to a workstation. Both of these time series exhibit features that are typical of network traffic data, namely strong seasonal components and highly non-Gaussian distributions. For these time series, a particular class of non-linear state space models is proposed, and practical techniques for model fitting and forecasting are demonstrated. Copyright 2003 Royal Statistical Society.

A regulator for a class of unknown continuous-time nonlinear systems

Abstract We introduce a control law for a class of unknown nonlinear continuous-time systems in which full state measurements are available. We show that as long as a certain feedback gain parameter is sufficiently large, the closed-loop system is stable. Furthermore, the magnitude of the control is bounded in the limit. As a corollary to the main result, we show that first-order nonlinear systems in the class we consider can be stabilized using linear proportional integral controllers, and we give explicit tuning rules for these controllers.