Eduard Belitser

Oracle convergence rate of posterior under projection prior and Bayesian model selection

We apply the Bayes approach to the problem of projection estimation of a signal observed in the Gaussian white noise model and we study the rate at which the posterior distribution concentrates about the true signal from the space ℓ2 as the information in observations tends to infinity. A benchmark is the rate of a so-called oracle projection risk, i.e., the smallest risk of an unknown true signal over all projection estimators. Under an appropriate hierarchical prior, we study the performance of the resulting (appropriately adjusted by the empirical Bayes approach) posterior distribution and establish that the posterior concentrates about the true signal with the oracle projection convergence rate. We also construct a Bayes estimator based on the posterior and show that it satisfies an oracle inequality. The results are nonasymptotic and uniform over ℓ2. Another important feature of our approach is that our results on the oracle projection posterior rate are always stronger than any result about posterior convergence with the minimax rate over all nonparametric classes for which the corresponding projection oracle estimator is minimax over this class. We also study implications for the model selection problem, namely, we propose a Bayes model selector and assess its quality in terms of the so-called false selection probability.

On coverage and local radial rates of credible sets

In the mildly ill-posed inverse signal-in-white-noise model, we construct confidence sets as credible balls with respect to the empirical Bayes posterior resulting from a certain two-level hierarchical prior. The quality of the posterior is characterized by the contraction rate which we allow to be local, that is, depending on the parameter. The issue of optimality of the constructed confidence sets is addressed via a trade-off between its “size” (the local radial rate) and its coverage probability. We introduce excessive bias restriction (EBR), more general than self-similarity and polished tail condition recently studied in the literature. Under EBR, we establish the confidence optimality of our credible set with some local (oracle) radial rate. We also derive the oracle estimation inequality and the oracle posterior contraction rate. The obtained local results are more powerful than global: adaptive minimax results for a number of smoothness scales follow as consequence, in particular, the ones considered by Szabo et al. [Ann. Statist. 43 (2015) 1391–1428].

On Robust Recursive Nonparametric Curve Estimation

Suppose we observe X k = θ(x k ) + ξ k . The function θ: [0,1] → ℝ, is assumed to belong a priori to a given nonparametric smoothness class, the ξ k ’s are independent identically distributed random variables with zero medians. The only prior information about the distribution of the noise is that it belongs to a rather wide class. The assumptions describing this class include cases in which no moments of the noises exist, so that linear estimation methods (for example, kernel methods) can not be applied. We propose a robust estimator based on a stochastic approximation procedure and derive its rate of convergence, as the frequency of observations n tends to infinity, in almost sure as well as in mean square sense, uniformly over the smoothness class Finally, we discuss a multivariate formulation of the problem, a robust nonparametric M-estimator (the least deviations estimator), the so called penalized estimator, and the case when the noises are not necessarily identically distributed.

Local minimax pointwise estimation of a multivariate density

We consider the problem of the nonparametric minimax estimation of a multivariate density at a given point. A concept of smoothness classes in nonparametric minimax estimation problems is proposed. The smoothness of a function is characterized by the approximability of the function at a point by an integral of the product of this function with an approximate identity. We propose a singular integral estimator, an integral of this approximate identity with respect to the empirical distribution function. Under some assumptions on the approximate identity, the bias of the estimator is shown to be of smaller order asymptotically than the variance, and the estimator itself is shown to be asymptotically locally minimax with respect to the quadratic risk in a proper topology.

On properties of the algorithm for pursuing a drifting quantile

The recurrent algorithm for pursuing a time-varying (“drifting”) quantile is suggested. The common nonasymptotic upper bound of the algorithm quality is established, which is then used in a few examples of the conditions for the quantile drift function. Estimates of the degree (rate) of convergence of the algorithm for the considered examples are obtained.

Recursive Tracking Algorithm for a Predictable Time-Varying Parameter of a Time Series

We propose a recursive algorithm for tracking a multi-dimensional time-varying parameter of a time series, which is also allowed to be a predictable process with respect to the underlying time series. The algorithm is driven by a gain function. For an arbitrary time series model and a gain function satisfying some conditions, we derive a general uniform non-asymptotic accuracy bound for the tracking algorithm in terms of chosen step size for the algorithm and the oscillations of the parameter of interest. We outline how appropriate gain functions can be constructed and give several examples of different variability settings for the parameter process for which our general result can be applied, leading to different convergence rates in different asymptotic regimes. The proposed approach covers many frameworks and models where stochastic approximation algorithms comprise the main inference tool for the data analysis.We treat in some detail a couple of specificmodels.

Recursive Estimation of Conditional Spatial Medians and Conditional Quantiles

Abstract We consider the problem of constructing an on-line (recursive) algorithm for tracking a conditional spatial median, a center of a multivariate distribution. In the one-dimensional case we also track conditional quantiles of arbitrary level. We establish a nonasymptotic upper bound for the L p -risk of the algorithm, which is then minimized under different assumptions on the magnitude of the variation of the spatial median or quantile. We derive convergence rates for the examples we consider.

Adaptive filtering of a random signal in Gaussian white noise

We consider the problem of estimating an infinite-dimensional vector θ observed in Gaussian white noise. Under the condition that components of the vector have a Gaussian prior distribution that depends on an unknown parameter β, we construct an adaptive estimator with respect to β. The proposed method of estimation is based on the empirical Bayes approach.

Minimax recovery of blurred signal from discrete noisy data

The problem of the minimax estimation of a nonparametric signal blurred by some known function and observed with additive noise is considered. The unknown function is assumed to belong to a hyperrectangle in L 2-([0,l]). Under some conditions, we find the exact asymptotic behavior of the quadratic minimax risk. We propose an estimator and show that its maximal risk attains asymptotically the minimax risk. The results are illustrated by examples.

CONSISTENCY IN NONPARAMETRIC MINIMAX REGRESSION ESTIMATION

ABSTRACT The problem of the minimax estimation of an additive nonparametric regression is considered. The regression function is assumed to belong to an ellipsoid-type subset of L 2([0,1]). We find the necessary and sufficient condition for the quadratic minimax risk to converge to zero.