Charles Stein

Estimation with Quadratic Loss

It has long been customary to measure the adequacy of an estimator by the smallness of its mean squared error. The least squares estimators were studied by Gauss and by other authors later in the nineteenth century. A proof that the best unbiased estimator of a linear function of the means of a set of observed random variables is the least squares estimator was given by Markov [12], a modified version of whose proof is given by David and Neyman [4]. A slightly more general theorem is given by Aitken [1]. Fisher [5] indicated that for large samples the maximum likelihood estimator approximately minimizes the mean squared error when compared with other reasonable estimators. This paper will be concerned with optimum properties or failure of optimum properties of the natural estimator in certain special problems with the risk usually measured by the mean squared error or, in the case of several parameters, by a quadratic function of the estimators. We shall first mention some recent papers on this subject and then give some results, mostly unpublished, in greater detail.

On the Theory of Some Non-Parametric Hypotheses

For two types of non-parametric hypotheses optimum tests are derived against certain classes of alternatives. The two kinds of hypotheses are related and may be illustrated by the following example: (1) The joint distribution of the variables Xi, • • •, Xm, Yi, • • •, Yn is invariant under all permutations of the variables; (2) the variables are independently and identically distributed. It is shown that the theory of optimum tests for hypotheses of the first kind is the same as that of optimum similar tests for hypotheses of the second kind. Most powerful tests are obtained against arbitrary simple alternatives, and in a number of important cases most stringent tests are derived against certain composite alternatives. For the example (1), if the distributions are restricted to probability densities, Pitman’s test based on ȳ — x is most powerful against the alternatives that the X’s and F’s are independently normally distributed with common variance, and that E(Xi) = ξ, E(Yi) =η where η > ξ If η — ξ may be positive or negative the test based on | ȳ — x| is most stringent. The definitions are sufficiently general that the theory applies to both continuous and discrete problems, and that tied observations present no difficulties. It is shown that continuous and discrete problems may be combined. Pitman’s test for example, when applied to certain discrete problems, coincides with Fisher’s exact test, and when m = n the test based on | ȳ — x | is most stringent for hypothesis (1) against a broad class of alternatives which includes both discrete and absolutely continuous distributions.

A Normal Approximation for the Number of Local Maxima of a Random Function on a Graph

Publisher Summary This chapter discusses the normal approximation for the number of local maxima of a random function on a graph. It discusses the conditions for the approximate normality of the distribution of the number of local maxima of a random function on the set of vertices of a graph when the values of the random function are independently identically distributed with a continuous distribution function. For a regular graph, the distribution of the number of local maxima is approximately normal if its variance is large. The basic idea of a normal approximation theorem is to exploit a sum of indicator random variables. The chapter discusses a basic lemma on normal approximation for sums of indicator random variables.

Most Powerful Tests of Composite Hypotheses. I. Normal Distributions

For testing a composite hypothesis, critical regions are determined which are most powerful against a particular alternative at a given level of significance. Here a region is said to have level of significance e if the probability of the region under the hypothesis tested is bounded above by e. These problems have been considered by Neyman, Pearson and others, subject to the condition that the critical region be similar. In testing the hypothesis specify-ing the value of the variance of a normal distribution with unknown mean against an alternative with larger variance, and in some other problems, the best similar region is also most powerful in the sense of this paper. However, in the analo-gous problem when the variance under the alternative hypothesis is less than that under the hypothesis tested, in the case of Student’s hypothesis when the level of significance is less than and in some other cases, the best similar region is not most powerful in the sense of this paper. There exist most powerful tests which are quite good against certain alternatives in some cases where no proper similar region exists. These results indicate that in some practical cases the standard test is not best if the class of alternatives is sufficiently restricted.

Completeness In The Sequential Case

Recently, in a series of papers, Girshick, Mosteller, Savage and Wolfowitz have considered the uniqueness of unbiased estimates depending only on an appropriate sufficient statistic for sequential sampling schemes of binomial variables. A complete solution was obtained under the restriction to bounded estimates. This work, which has immediate consequences with respect to the existence of unbiased estimates with uniformly minimum variance, is extended here in two directions. A general necessary condition for uniqueness is found, and this is applied to obtain a complete solution of the uniqueness problem when the random variables have a Poisson or rectangular distribution. Necessary and sufficient conditions are also found in the binomial case without the restriction to bounded estimates. This permits the statement of a somewhat stronger optimum property for the estimates, and is applicable to the estimation of unbounded functions of the unknown probability.

The Admissibility Of Certain Invariant Statistical Tests Involving A Translation Parameter

Introduction. The notion of invariance (or symmetry) has such strong intuitive appeal that many current statistical procedures have the invariance property and are in fact the best invariant procedures although they were proposed long before a general discussion of invariance was available. Hotelling [1], [2] and Pitman [3], [4] emphasized the invariant nature of certain tests and esti-mates. A general definition of the notion for the problem of testing hypotheses was given by Hunt and Stein who showed that in this case under severe restric-tions on the group of transformations an optimum invariant test is most stringent or more generally minimax with respect to an invariant loss function (see [5]). This result has been extended to more general decision problems and more general groups by Peisakoff [6]. However, these results do not prove admissibility of the procedures in question unless the group of transformations is compact.