Mathematics

We derive an asymptotic expansion for the log likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal-to-noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation-Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone toward the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest.

We consider a general linear program in standard form whose right-hand side constraint vector is subject to random perturbations. This defines a stochastic linear program for which, under general conditions, we characterize the fluctuations of the corresponding empirical optimal solution by a central limit-type theorem. Our approach relies on the combinatorial nature and the concept of degeneracy inherent in linear programming, in strong contrast to well-known results for smooth stochastic optimization programs. In particular, if the corresponding dual linear program is degenerate the asymptotic limit law might not be unique and is determined from the way the empirical optimal solution is chosen. Furthermore, we establish consistency and convergence rates of the Hausdorff distance between the empirical and the true optimality sets. As a consequence, we deduce a limit law for the empirical optimal value characterized by the set of all dual optimal solutions which turns out to be a simple consequence of our general proof techniques. Our analysis is motivated from recent findings in statistical optimal transport that will be of special focus here. In addition to the asymptotic limit laws for optimal transport solutions, we obtain results linking degeneracy of the dual transport problem to geometric properties of the underlying ground space, and prove almost sure uniqueness statements that may be of independent interest.

Consider the p×p matrix that is the product of a population covariance matrix and the inverse of another population covariance matrix. Suppose that their difference has a divergent rank with respect to p , when two samples of sizes n and T from the two populations are available, we construct its corresponding sample version. In the regime of high dimension where both n and T are proportional to p , we investigate the limiting laws for extreme (spiked) eigenvalues of the sample (spiked) Fisher matrix when the number of spikes is divergent and these spikes are unbounded.

This note investigates functions from R d to R?�{±?�} that satisfy axioms of linearity wherever allowed by extended-value arithmetic. They have a nontrivial structure defined inductively on d , and unlike finite linear functions, they require Ω( d 2 ) parameters to uniquely identify. In particular they can capture vertical tangent planes to epigraphs: a function (never ?��? ) is convex if and only if it has an extended-valued subgradient at every point in its effective domain, if and only if it is the supremum of a family of "affine extended" functions. These results are applied to the well-known characterization of proper scoring rules, for the finite-dimensional case: it is carefully and rigorously extended here to a more constructive form. In particular it is investigated when proper scoring rules can be constructed from a given convex function.

We establish a nonasymptotic lower bound on the L 2 minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).

In this paper we are concerned with a sequence of univariate random variables with piecewise polynomial means and independent sub-Gaussian noise. The underlying polynomials are allowed to be of arbitrary but fixed degrees. All the other model parameters are allowed to vary depending on the sample size. We propose a two-step estimation procedure based on the ℓ 0 -penalisation and provide upper bounds on the localisation error. We complement these results by deriving a global information-theoretic lower bounds, which show that our two-step estimators are nearly minimax rate-optimal. We also show that our estimator enjoys near optimally adaptive performance by attaining individual localisation errors depending on the level of smoothness at individual change points of the underlying signal. In addition, under a special smoothness constraint, we provide a minimax lower bound on the localisation errors. This lower bound is independent of the polynomial orders and is sharper than the global minimax lower bound.

We study Voronoi cells in the statistical setting by considering preimages of the maximum likelihood estimator that tessellate an open probability simplex. In general, logarithmic Voronoi cells are convex sets. However, for certain algebraic models, namely finite models, models with ML degree 1, linear models, and log-linear (or toric) models, we show that logarithmic Voronoi cells are polytopes. As a corollary, the algebraic moment map has polytopes for both its fibres and its image, when restricted to the simplex. We also compute non-polytopal logarithmic Voronoi cells using numerical algebraic geometry. Finally, we determine logarithmic Voronoi polytopes for the finite model consisting of all empirical distributions of a fixed sample size. These polytopes are dual to the logarithmic root polytopes of Lie type A, and we characterize their faces.

Majorisation, also called rearrangement inequalities, yields a type of stochastic ordering in which two or more distributions can be compared. In this paper we argue that majorisation is a good candidate as a theory for uncertainty. We present operations that can be applied to study uncertainty in a range of settings and demonstrate our approach to assessing uncertainty with examples from well known distributions and from applications of climate projections and energy systems.

We consider a testing problem for cross-sectional dependence for high-dimensional panel data, where the number of cross-sectional units is potentially much larger than the number of observations. The cross-sectional dependence is described through a linear regression model. We study three tests named the sum test, the max test and the max-sum test, where the latter two are new. The sum test is initially proposed by Breusch and Pagan (1980). We design the max and sum tests for sparse and non-sparse residuals in the linear regressions, respectively.And the max-sum test is devised to compromise both situations on the residuals. Indeed, our simulation shows that the max-sum test outperforms the previous two tests. This makes the max-sum test very useful in practice where sparsity or not for a set of data is usually vague. Towards the theoretical analysis of the three tests, we have settled two conjectures regarding the sum of squares of sample correlation coefficients asked by Pesaran (2004 and 2008). In addition, we establish the asymptotic theory for maxima of sample correlations coefficients appeared in the linear regression model for panel data, which is also the first successful attempt to our knowledge. To study the max-sum test, we create a novel method to show asymptotic independence between maxima and sums of dependent random variables. We expect the method itself is useful for other problems of this nature. Finally, an extensive simulation study as well as a case study are carried out. They demonstrate advantages of our proposed methods in terms of both empirical powers and robustness for residuals regardless of sparsity or not.

This paper concerns the parameter estimation problem for the quadratic potential energy in interacting particle systems from continuous-time and single-trajectory data. Even though such dynamical systems are high-dimensional, we show that the vanilla maximum likelihood estimator (without regularization) is able to estimate the interaction potential parameter with optimal rate of convergence simultaneously in mean-field limit and in long-time dynamics. This to some extend avoids the curse-of-dimensionality for estimating large dynamical systems under symmetry of the particle interaction.