Arne Kovac

Extending the Scope of Wavelet Regression Methods by Coefficient-Dependent Thresholding

Abstract Various aspects of the wavelet approach to nonparametric regression are considered, with the overall aim of extending the scope of wavelet techniques to irregularly spaced data, to regularly spaced datasets of arbitrary size, to heteroscedastic and correlated data, and to data that contain outliers. The core of the methodology is an algorithm for finding ail of the variances and within-level covariances in the wavelet table of a sequence with given covariance structure. If the original covariance matrix is band-limited, then the algorithm is linear in the length of the sequence. The variance calculation algorithm allows data on any set of independent variable values to be treated, by first interpolating to a fine regular grid of suitable length, and then constructing a wavelet expansion of the gridded data. Various thresholding methods are discussed and investigated. Exact risk formulas for the mean square error of the methodology for given design are derived. Good performance is obtained by nois...

Extensions of smoothing via taut strings

Suppose that we observe independent, identically distributed random pairs (X1; Y1), (X2; Y2), . . . , (Xn; Yn). Our goal is to estimate regression functions such as the conditional mean or nquantile of Y given X, where 0 0 is some tuning parameter. This framework is extended further in order to include binary or Poisson regression, and to include local variation penalties. The latter are needed in order to construct estimators adapting to inhomogenous smoothness of f . For the general framework we develop noniterative algorithms for the solution of the minimization problems which are closely related to the taut string algorithm (cf. Davies and Kovac 2001).

Nonparametric regression, confidence regions and regularization

In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and nonasymptotic confidence region A n which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically center on certain simplest functions in A n where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the confidence region can be used to provide honest nonasymptotic confidence bounds which are less informative but conceptually much simpler.

Nonparametric Regression on a Graph

The ‘Signal plus Noise’ model for nonparametric regression can be extended to the case of observations taken at the vertices of a graph. This model includes many familiar regression problems. This article discusses the use of the edges of a graph to measure roughness in penalized regression. Distance between estimate and observation is measured at every vertex in the L2 norm, and roughness is penalized on every edge in the L1 norm. Thus the ideas of total variation penalization can be extended to a graph. The resulting minimization problem presents special computational challenges, so we describe a new and fast algorithm and demonstrate its use with examples. The examples include image analysis, a simulation applicable to discrete spatial variation, and classification. In our examples, penalized regression improves upon kernel smoothing in terms of identifying local extreme values on planar graphs. In all examples we use fully automatic procedures for setting the smoothing parameters. Supplemental materials are available online.

Smooth functions and local extreme values

The problem of specifying a smooth and simple function that approximates noisy data is considered. A new automatic method is described that is based on solving a constrained optimisation problem. The target functional to be minimised is the sum of the squared residuals penalised by the curve length of the approximation. Multiresolution and monotonicity constraints provide a good approximation to the data with a small number of local extreme values. The new method can also be applied to density estimation.

Bivariate density estimation using BV regularisation

The problem of bivariate density estimation is studied with the aim of finding the density function with the smallest number of local extreme values which is adequate with the given data. Adequacy is defined via Kuiper metrics. The concept of the taut-string algorithm which provides adequate approximations with a small number of local extrema is generalised for analysing two- and higher dimensional data, using Delaunay triangulation and diffusion filtering. Results are based on equivalence relations in one dimension between the taut-string algorithm and the method of solving the discrete total variation flow equation. The generalisation and some modifications are developed and the performance for density estimation is shown.

Robust Nonparametric Regression and Modality

The paper considers the problem of nonparametric regression with emphasis on controlling the number of local extremes and on resistance against patches of outliers. The robust taut string method is introduced and robustness properties are discussed. An automatic procedure is described.

Quantifying the cost of simultaneous non-parametric approximation of several samples

We consider the standard non-parametric regression model with Gaussian errors but where the data consist of different samples. The question to be answered is whether the samples can be adequately represented by the same regression function. To do this we define for each sample a universal, honest and non-asymptotic confidence region for the regression function. Any subset of the samples can be represented by the same function if and only if the intersection of the corresponding confidence regions is non-empty. If the empirical supports of the samples are disjoint then the intersection of the confidence regions is always non--empty and a negative answer can only be obtained by placing shape or quantitative smoothness conditions on the joint approximation. Alternatively a simplest joint approximation function can be calculated which gives a measure of the cost of the joint approximation, for example, the number of extra peaks required.

Modality, Runs, Strings and Wavelets

The paper considers the problem of non-parametric regression with emphasis on controlling the number of local extrema. Two methods, the run method and the taut string-wavelet method, are introduced and analysed on standard test beds. It is shown that the number and location of local extreme values are consistently estimated. Rates of convergence are proved for both methods. The run method has a slow rate but can withstand blocks as well as a high proportion of isolated outliers. The rate of convergence of the taut string-wavelet method is almost optimal and the method is extremely sensitive being able to detect very low power peaks. Section 1 contains a short introduction with special reference to modality. The run method is described in Section 2 and the taut string-wavelet method in Section 3. Low power peaks are considered in Section 4. Section 5 contains a short conclusion and the proofs are given in Section 6.