I. Vajda

Asymptotic divergence of estimates of discrete distributions

Abstract o-divergence D o ( P , Q ) of two estimates P and Q of a discrete distribution Pθ is shown to play an important role in mathematical statistics and information theory. If both P and Q are based on the same sample then special attention is paid to nonparameteric estimates P which are √n-consistent and parametric estimates Q = P gq defined by means of minimum o∗-divergence point estimates gq where o∗ need not be the same as o. Under a standard regularity these point estimates are shown to be efficient and the corresponding distribution estimates Q √n-consistent. But the asymptotics of D o ( P , Q ) is evaluated for arbitrary cn-consistent distribution stimates P and Q with cn → ∞. If the two distribution estimates are based on different samples then the asymptotics of D o ( P , Q ) is evaluated only for the above-mentioned special P and Q .

Two approaches to grouping of data and related disparity statistics

Csiszars φ-divergences of discrete distributions are extended to a more general class of disparity measures by restricting the convexity of functions φ(t), t > 0, to the local convexity at t = 1 and monotonicity on intervals (0, 1) and (l,∞). Goodness-of-fit estimation and testing procedures based on the (^-disparity statistics are introduced. Robustness of the estimation procedure is discussed and the asymptotic distributions for the testing procedure are established in statistical models with data grouped according to their values or orders

Rényi Statistics in Directed Families of Exponential Experiments

Rényi statistics are considered in a directed family of general exponential models. These statistics are defined as Rényi distances between estimated and hypothetical model. An asymptotically quadratic approximation to the Rényi statistics is established, leading to similar asymptotic distribution results as established in the literature for the likelihood ratio statistics. Some arguments in favour of the Rényi statistics are discussed, and a numerical comparison of the Rényi goodness-of-fit tests with the likelihood ratio test is presented.

Asymptotic distributions of φ‐divergences of hypothetical and observed frequencies on refined partitions

For a wide class of goodness-of-fit statistics based on φ-divergences between hypothetical cell probabilities and observed relative frequencies, the asymptotic normality is established under the assumption n/mnγ∈(0,∞), where n denotes sample size and mn the number of cells. Related problems of asymptotic distributions of φ-divergence errors, and of φ-divergence deviations of histogram estimators from their expected values, are considered too.

Minimum divergence estimators based on grouped data

The paper considers statistical models with real-valued observations i.i.d. by F(x, θ0) from a family of distribution functions (F(x, θ); θ ε Θ), Θ ⊂ Rs, s ≥ 1. For random quantizations defined by sample quantiles (Fn−1 (λ1),θ, Fn−1 (λm−1)) of arbitrary fixed orders 0 < λ1 θ < λm-1 < 1, there are studied estimators θφ,n of θ0 which minimize φ-divergences of the theoretical and empirical probabilities. Under an appropriate regularity, all these estimators are shown to be as efficient (first order, in the sense of Rao) as the MLE in the model quantified nonrandomly by (F−1 (λ1,θ0),θ, F−1 (λm−1, θ0)). Moreover, the Fisher information matrix Im (θ0, λ) of the latter model with the equidistant orders λ = (λj = j/m : 1 ≤ j ≤ m − 1) arbitrarily closely approximates the Fisher information J(θ0) of the original model when m is appropriately large. Thus the random binning by a large number of quantiles of equidistant orders leads to appropriate estimates of the above considered type.

About divergence-based goodness-of-fit tests in the Dirichlet-multinomial model

The Pearson X2 statistic, the loglikelihood ratio statistic, the Freeman-Tukey statistic, and others well known statistics are considered as particular cases of appropriately normed (φ-divergences Dφ(FNF) between empirical and hypothetical distributions. All these statistics are asymptotically x2-distributed. For the Dirichlet-multinomial F, conditions on J are found under which the first two moments of the test statistics match the moments of x2, up to terms of order N--1. By calculating the exact sizes and powers of goodness-of-fit tests for 14 selected φ-divergence statistics it is shown that these conditions are of practical importance even for small sample sizes N.

Minimum Disparity Estimators for Discrete and Continuous Models

Disparities of discrete distributions are introduced as a natural and useful extension of the information-theoretic divergences. The minimum disparity point estimators are studied in regular discrete models with i.i.d. observations and their asymptotic efficiency of the first order, in the sense of Rao, is proved. These estimators are applied to continuous models with i.i.d. observations when the observation space is quantized by fixed points, or at random, by the sample quantiles of fixed orders. It is shown that the random quantization leads to estimators which are robust in the sense of Lindsay [9], and which can achieve the efficiency in the underlying continuous models provided these are regular enough.

APPROXIMATIONS TO POWERS OF φ-DISPARITY GOODNESS-OF-FIT TESTS*

The paper studies a class of tests based on disparities between the real-valued data and theoretical models resulting either from fixed partitions of the observation space, or from the partitions by the sample quantiles of fixed orders. In both cases there are considered the goodness-of-fit tests of simple and composite hypotheses. All tests are shown to be consistent, and their power is evaluated at the nonlocal as well as local alternatives.

Asymptotic laws for disparity statistics in product multinomial models

The paper presents asymptotic distributions of φ-disparity goodness-of-fit statistics in product multinomial models, under hypotheses and alternatives assuming sparse and nonsparse cell frequencies. The φ-disparity statistics include the power divergences of Read and Cressie (Goodness-of-fit Statistics for Discrete Multivariate Data, Springer, New York, 1988), the φ-divergences of Ciszar (Studia Sci. Math. Hungar. 2 (1967) 299) and the robust goodness of fit statistics of Lindsay (Ann. Statist. 22 (1994) 1081).

On convergence of fisher informations in continuous models with quantized observations

Continuous location models with real observations and well defined Fisher information are considered and reduction of the Fisher information due to quantizations of the observation space intom intervals is studied. In fact, generalized Fisher informations of orders α≥1 are considered where α=2 corresponds to the classical Fisher information. By an example it is argued that in some models the information of order α=2 is infinite while the informations of some orders α↮2 are finite. Among the studied problems is the existence of optimal quantizations which maximize the reduced information for fixedm and α≥1 and the construction of simple and practically applicable quantizations for which the reduction converges to zero whenm→∞, uniformly for all α≥1. The rate of this convergence is estimated for all α≥1 and directly evaluated for α=1 and α=2. For special models the reductions are directly evaluated form=1.2,… either analytically or numerically.