Mathematics

We establish that Laplace transforms of the posterior Dirichlet process converge to those of the limiting Brownian bridge process in a neighbourhood about zero, uniformly over Glivenko-Cantelli function classes. For real-valued random variables and functions of bounded variation, we strengthen this result to hold for all real numbers. This last result is proved via an explicit strong approximation coupling inequality.

Recent years have witnessed the success of adaptive (or unified) approaches in estimating symmetric properties of discrete distributions, where one first obtains a distribution estimator independent of the target property, and then plugs the estimator into the target property as the final estimator. Several such approaches have been proposed and proved to be adaptively optimal, i.e. they achieve the optimal sample complexity for a large class of properties within a low accuracy, especially for a large estimation error ε≫ n −1/3 where n is the sample size. In this paper, we characterize the high accuracy limitation, or the penalty for adaptation, for all such approaches. Specifically, we show that under a mild assumption that the distribution estimator is close to the true sorted distribution in expectation, any adaptive approach cannot achieve the optimal sample complexity for every 1 -Lipschitz property within accuracy ε≪ n −1/3 . In particular, this result disproves a conjecture in [Acharya et al. 2017] that the profile maximum likelihood (PML) plug-in approach is optimal in property estimation for all ranges of ε , and confirms a conjecture in [Han and Shiragur, 2021] that their competitive analysis of the PML is tight.

Latent class models have recently become popular for multiple-systems estimation in human rights applications. However, it is currently unknown when a given family of latent class models is identifiable in this context. We provide necessary and sufficient conditions on the number of latent classes needed for a family of latent class models to be identifiable. Along the way we provide a mechanism for verifying identifiability in a class of multiple-systems estimation models that allow for individual heterogeneity.

The purpose of this review is to present a comprehensive overview of the theory of ensemble Kalman-Bucy filtering for linear-Gaussian signal models. We present a system of equations that describe the flow of individual particles and the flow of the sample covariance and the sample mean in continuous-time ensemble filtering. We consider these equations and their characteristics in a number of popular ensemble Kalman filtering variants. Given these equations, we study their asymptotic convergence to the optimal Bayesian filter. We also study in detail some non-asymptotic time-uniform fluctuation, stability, and contraction results on the sample covariance and sample mean (or sample error track). We focus on testable signal/observation model conditions, and we accommodate fully unstable (latent) signal models. We discuss the relevance and importance of these results in characterising the filter's behaviour, e.g. it's signal tracking performance, and we contrast these results with those in classical studies of stability in Kalman-Bucy filtering. We provide intuition for how these results extend to nonlinear signal models and comment on their consequence on some typical filter behaviours seen in practice, e.g. catastrophic divergence.

We study the minimal error of the Empirical Risk Minimization (ERM) procedure in the task of regression, both in the random and the fixed design settings. Our sharp lower bounds shed light on the possibility (or impossibility) of adapting to simplicity of the model generating the data. In the fixed design setting, we show that the error is governed by the global complexity of the entire class. In contrast, in random design, ERM may only adapt to simpler models if the local neighborhoods around the regression function are nearly as complex as the class itself, a somewhat counter-intuitive conclusion. We provide sharp lower bounds for performance of ERM for both Donsker and non-Donsker classes. We also discuss our results through the lens of recent studies on interpolation in overparameterized models.

This paper focuses on the non-asymptotic concentration of the heteroskedastic Wishart-type matrices. Suppose Z is a p 1 -by- p 2 random matrix and Z ij ∼N(0, σ 2 ij ) independently, we prove that \begin{equation*} \bbE \left\|ZZ^\top - \bbE ZZ^\top\right\| \leq (1+\epsilon)\left\{2\sigma_C\sigma_R + \sigma_C^2 + C\sigma_R\sigma_*\sqrt{\log(p_1 \wedge p_2)} + C\sigma_*^2\log(p_1 \wedge p_2)\right\}, \end{equation*} where σ 2 C := max j ∑ p 1 i=1 σ 2 ij , σ 2 R := max i ∑ p 2 j=1 σ 2 ij and σ 2 ∗ := max i,j σ 2 ij . A minimax lower bound is developed that matches this upper bound. Then, we derive the concentration inequalities, moments, and tail bounds for the heteroskedastic Wishart-type matrix under more general distributions, such as sub-Gaussian and heavy-tailed distributions. Next, we consider the cases where Z has homoskedastic columns or rows (i.e., σ ij ≈ σ i or σ ij ≈ σ j ) and derive the rate-optimal Wishart-type concentration bounds. Finally, we apply the developed tools to identify the sharp signal-to-noise ratio threshold for consistent clustering in the heteroskedastic clustering problem.

Wilk's theorem, which offers universal chi-squared approximations for likelihood ratio tests, is widely used in many scientific hypothesis testing problems. For modern datasets with increasing dimension, researchers have found that the conventional Wilk's phenomenon of the likelihood ratio test statistic often fails. Although new approximations have been proposed in high dimensional settings, there still lacks a clear statistical guideline regarding how to choose between the conventional and newly proposed approximations, especially for moderate-dimensional data. To address this issue, we develop the necessary and sufficient phase transition conditions for Wilk's phenomenon under popular tests on multivariate mean and covariance structures. Moreover, we provide an in-depth analysis of the accuracy of chi-squared approximations by deriving their asymptotic biases. These results may provide helpful insights into the use of chi-squared approximations in scientific practices.

We revisit the problem of the estimation of the differential entropy H(f) of a random vector X in R d with density f , assuming that H(f) exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate H n of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent ( E{| H n ?�H(f)|}?? as n?��? ) if and only if E{log(?�X??1)}<??. Furthermore, if X has compact support, then H n ?�H(f) almost surely.

Some 40 years ago Khmaladze introduced a transform which greatly facilitated the distribution free goodness of fit testing of statistical hypotheses. In the last decade, he has published a related transform, broadly offering an alternative means to the same end. The aim of this paper is to derive these transforms using relatively elementary means, making some simplifications, but losing little in the way of generality. In this way it is hoped to make these transforms more accessible and more widely used in statistical practice. We also propose a change of name of the second transform to the Khmaladze rotation, in order to better reflect its nature.

In this paper we analyse the behaviour of adaptive filters or detectors when they are trained with t -distributed samples rather than Gaussian distributed samples. More precisely we investigate the impact on the distribution of some relevant statistics including the signal to noise ratio loss and the Gaussian generalized likelihood ratio test. Some properties of partitioned complex F distributed matrices are derived which enable to obtain statistical representations in terms of independent chi-square distributed random variables. These representations are compared with their Gaussian counterparts and numerical simulations illustrate and quantify the induced degradation.