Idris A. Eckley

Optimal Detection of Changepoints With a Linear Computational Cost

We consider the problem of detecting multiple changepoints in large data sets. Our focus is on applications where the number of changepoints will increase as we collect more data: for example in genetics as we analyse larger regions of the genome, or in finance as we observe time-series over longer periods. We consider the common approach of detecting changepoints through minimising a cost function over possible numbers and locations of changepoints. This includes several established procedures for detecting changing points, such as penalised likelihood and minimum description length. We introduce a new ∗R. Killick is Senior Research Associate, Department of Mathematics & Statistics, Lancaster University, Lancaster, UK (E-mail: r.killick@lancs.ac.uk). P. Fearnhead is Professor, Department of Mathematics & Statistics, Lancaster University, Lancaster, UK (E-mail: p.fearnhead@lancs.ac.uk). I.A. Eckley is Senior Lecturer, Department of Mathematics & Statistics, Lancaster University, Lancaster, UK (E-mail: i.eckley@lancs.ac.uk). The authors are grateful to Richard Davis and Alice Cleynen for providing the Auto-PARM and PDPA software respectively. Part of this research was conducted whilst R. Killick was a jointly funded Engineering and Physical Sciences Research Council (EPSRC) / Shell Research Ltd graduate student at Lancaster University. Both I.A. Eckley and R. Killick also gratefully acknowledge the financial support of the EPSRC grant number EP/I016368/1. 1 ar X iv :1 10 1. 14 38 v3 [ st at .M E ] 9 O ct 2 01 2 method for finding the minimum of such cost functions and hence the optimal number and location of changepoints that has a computational cost which, under mild conditions, is linear in the number of observations. This compares favourably with existing methods for the same problem whose computational cost can be quadratic or even cubic. In simulation studies we show that our new method can be orders of magnitude faster than these alternative exact methods. We also compare with the Binary Segmentation algorithm for identifying changepoints, showing that the exactness of our approach can lead to substantial improvements in the accuracy of the inferred segmentation of the data.

An empirical comparison of three boosting algorithms on real data sets with artificial class noise

Boosting algorithms are a means of building a strong ensemble classifier by aggregating a sequence of weak hypotheses. In this paper we consider three of the best-known boosting algorithms: Adaboost [9], Logitboost [11] and Brownboost [8]. These algorithms are adaptive, and work by maintaining a set of example and class weights which focus the attention of a base learner on the examples that are hardest to classify. We conduct an empirical study to compare the performance of these algorithms, measured in terms of overall test error rate, on five real data sets. The tests consist of a series of cross-validatory samples. At each validation, we set aside one third of the data chosen at random as a test set, and fit the boosting algorithm to the remaining two thirds, using binary stumps as a base learner. At each stage we record the final training and test error rates, and report the average errors within a 95% confidence interval. We then add artificial class noise to our data sets by randomly reassigning 20% of class labels, and repeat our experiment. We find that Brownboost and Logitboost prove less likely than Adaboost to overfit in this circumstance.

Estimating Time-Evolving Partial Coherence Between Signals via Multivariate Locally Stationary Wavelet Processes

We consider the problem of estimating time-localized cross-dependence in a collection of nonstationary signals. To this end, we develop the multivariate locally stationary wavelet framework, which provides a time-scale decomposition of the signals and, thus, naturally captures the time-evolving scale-specific cross-dependence between components of the signals. Under the proposed model, we rigorously define and estimate two forms of cross-dependence measures: wavelet coherence and wavelet partial coherence. These dependence measures differ in a subtle but important way. The former is a broad measure of dependence, which may include indirect associations, i.e., dependence between a pair of signals that is driven by another signal. Conversely, wavelet partial coherence measures direct linear association between a pair of signals, i.e., it removes the linear effect of other observed signals. Our time-scale wavelet partial coherence estimation scheme thus provides a mechanism for identifying hidden dynamic relationships within a network of nonstationary signals, as we demonstrate on electroencephalograms recorded in a visual-motor experiment.

A wavelet-based approach for detecting changes in second order structure within nonstationary time series

This article proposes a test to detect changes in general autocovariance structure in nonstationary time series. Our approach is founded on the locally stationary wavelet (LSW) process model for time series which has previously been used for classification and segmentation of time series. Using this framework we form a likelihood-based hypothesis test and demonstrate its performance against existing methods on various simulated examples as well as applying it to a problem arising from ocean engineering.

Efficient computation of the discrete autocorrelation wavelet inner product matrix

Discrete autocorrelation (a.c.) wavelets have recently been applied in the statistical analysis of locally stationary time series for local spectral modelling and estimation. This article proposes a fast recursive construction of the inner product matrix of discrete a.c. wavelets which is required by the statistical analysis. The recursion connects neighbouring elements on diagonals of the inner product matrix using a two-scale property of the a.c. wavelets. The recursive method is an ↻(log (N)3) operation which compares favourably with the ↻(N log N) operations required by the brute force approach. We conclude by describing an efficient construction of the inner product matrix in the (separable) two-dimensional case.

A Note on the Effect of Wavelet Choice on the Estimation of the Evolutionary Wavelet Spectrum

The locally stationary wavelet process model assumes some underlying wavelet family in order to generate the process. Analyses of such processes also assume that the same wavelet family is used to obtain unbiased estimates of the wavelet spectrum. In practice this would not typically be possible since, a priori, the underlying wavelet family is not known. This article considers the effect of wavelet choice within this setting. A particular focus is given to the estimation of the evolutionary wavelet spectrum due to its importance in many reported applications.

The effect of recovery algorithms on compressive sensing background subtraction

Background subtraction is a key method required to aid processing surveillance videos. Current methods require storing each pixel of every video frame, which can be wasteful as most of this information refers to the uninteresting background. Compressive sensing can offer an efficient solution by using the fact that foreground is often sparse in the spatial domain. By making this assumption and applying a specific recovery algorithm to a trained background, it is possible to reconstruct the foreground, using only a low dimensional representation of the difference between the current frame and the estimated background scene. Although new compressive sensing background subtraction algorithms are being created, no study has been made of the effect of recovery algorithms on performance of background subtraction. This is considered by applying both Basis Pursuit and Orthogonal Matching Pursuit (OMP) to a standard test video, and comparing their accuracy.

Computationally efficient changepoint detection for a range of penalties

ABSTRACT In the multiple changepoint setting, various search methods have been proposed, which involve optimizing either a constrained or penalized cost function over possible numbers and locations of changepoints using dynamic programming. Recent work in the penalized optimization setting has focused on developing an exact pruning-based approach that, under certain conditions, is linear in the number of data points. Such an approach naturally requires the specification of a penalty to avoid under/over-fitting. Work has been undertaken to identify the appropriate penalty choice for data-generating processes with known distributional form, but in many applications the model assumed for the data is not correct and these penalty choices are not always appropriate. To this end, we present a method that enables us to find the solution path for all choices of penalty values across a continuous range. This permits an evaluation of the various segmentations to identify a suitable penalty choice. The computational complexity of this approach can be linear in the number of data points and linear in the difference between the number of changepoints in the optimal segmentations for the smallest and largest penalty values. Supplementary materials for this article are available online.

Spectral correction for locally stationary Shannon wavelet processes

It is well-known that if a time series is not sampled at a fast enough rate to capture all the high frequencies then aliasing may occur. Aliasing is a distortion of the spectrum of a series which can cause severe problems for time series modelling and forecasting. The situation is more complex and more interesting for nonstationary series as aliasing can be intermittent. Recent work has shown that it is possible to test for the absence of aliasing in nonstationary time series and this article demonstrates that additional benefits can be obtained by modelling a series using a Shannon locally stationary wavelet (LSW) process. We show that for Shannon LSW processes the effects of dyadic-sampling-induced aliasing can be reversed. We illustrate our method by simulation on Shannon LSW processes and also a time-varying autoregressive process where aliasing is detected. We present an analysis of a wind power time series and show that it can be adequately modelled by a Shannon LSW process, the absence of aliasing can not be inferred and present a dealiased estimate of the series.

A MULTICLASS EXTENSION TO THE BROWNBOOST ALGORITHM

Brownboost is an adaptive, continuous time boosting algorithm based on the Boost-by-Majority (BBM) algorithm. Though it has been little studied at the time of writing, it is believed that it should prove especially robust with respect to noisy data sets. This would make it a very useful boosting algorithm for real-world applications. More familiar algorithms such as Adaboost, or its successor Logitboost, are known to be especially susceptible to overfitting the training data examples. This can lead to a poor generalization error in the presence of class noise, since weak hypotheses induced at later iterations to fit the noisy examples will tend to be given undue influence in the final combined hypothesis. Brownboost allows us to specify an expected base-line error rate in advance, corresponding to our prior beliefs about the proportion of noise in the training data, and thus avoid overfitting. The original derivation of Brownboost is restricted to binary classification problems. In this paper we propose a natural multiclass extension to the basic algorithm, incorporating error-correcting output codes and a multiclass gain measure. We test two-class and multiclass versions of the algorithm on a number of real and simulated data sets with artificial class noise, and show that Brownboost consistently outperforms Adaboost in these situations.