Thomas Sellke

Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence

Abstract The problem of testing a point null hypothesis (or a “small interval” null hypothesis) is considered. Of interest is the relationship between the P value (or observed significance level) and conditional and Bayesian measures of evidence against the null hypothesis. Although one might presume that a small P value indicates the presence of strong evidence against the null, such is not necessarily the case. Expanding on earlier work [especially Edwards, Lindman, and Savage (1963) and Dickey (1977)], it is shown that actual evidence against a null (as measured, say, by posterior probability or comparative likelihood) can differ by an order of magnitude from the P value. For instance, data that yield a P value of .05, when testing a normal mean, result in a posterior probability of the null of at least .30 for any objective prior distribution. (“Objective” here means that equal prior weight is given the two hypotheses and that the prior is symmetric and nonincreasing away from the null; other definiti...

Calibration of p Values for Testing Precise Null Hypotheses

P values are the most commonly used tool to measure evidence against a hypothesis or hypothesized model. Unfortunately, they are often incorrectly viewed as an error probability for rejection of the hypothesis or, even worse, as the posterior probability that the hypothesis is true. The fact that these interpretations can be completely misleading when testing precise hypotheses is first reviewed, through consideration of two revealing simulations. Then two calibrations of a ρ value are developed, the first being interpretable as odds and the second as either a (conditional) frequentist error probability or as the posterior probability of the hypothesis.

Redefine Statistical Significance

We propose to change the default P-value threshold for statistical significance from 0.05 to 0.005 for claims of new discoveries.

Rejection Odds and Rejection Ratios: A Proposal for Statistical Practice in Testing Hypotheses

Much of science is (rightly or wrongly) driven by hypothesis testing. Even in situations where the hypothesis testing paradigm is correct, the common practice of basing inferences solely on p-values has been under intense criticism for over 50 years. We propose, as an alternative, the use of the odds of a correct rejection of the null hypothesis to incorrect rejection. Both pre-experimental versions (involving the power and Type I error) and post-experimental versions (depending on the actual data) are considered. Implementations are provided that range from depending only on the p-value to consideration of full Bayesian analysis. A surprise is that all implementations – even the full Bayesian analysis – have complete frequentist justification. Versions of our proposal can be implemented that require only minor modifications to existing practices yet overcome some of their most severe shortcomings.

Generalized gauss-chebyshev inequalities for unimodal distributions

Letg be an even function on ℝ which is nondecreasing in |x|. Letk be a positive constant. Sharp inequalities relatingP(|X|≥k) toEg(X) are obtained for random variablesX which are unimodal with mode 0, and for random variablesX which are unimodal with unspecified mode. The bounds in the mode 0 case generalize an inequality due to Gauss (1823), whereg(x)=x2. The bounds in the second case generalize inequalities of Vysochanskiĭ and Petunin (1980, 1983) and Dharmadhikari and Joag-dev (1985).

Chebyshev inequalities for unimodal distributions

Abstract Let g be an even function on ℝ that is nondecreasing on [0, ∞), and let k be a positive constant. For random variables X that are unimodal with mode 0, and for random variables X that are unimodal with an unspecified mode, we derive sharp upper bounds on P(|X| ≥ k) in terms of Eg(X). The proofs consist largely of drawing a chord and a few tangent lines on graphs of cdfs.

An extension of Hawkes theorem on the Hausdorff dimension of a Galton–Watson tree

Abstract. Let ? be the genealogical tree of a supercritical multitype Galton–Watson process, and let Λ be the limit set of ?, i.e., the set of all infinite self-avoiding paths (called ends) through ? that begin at a vertex of the first generation. The limit set Λ is endowed with the metric d(ζ, ξ) = 2−n where n = n(ζ, ξ) is the index of the first generation where ζ and ξ differ. To each end ζ is associated the infinite sequence Φ(ζ) of types of the vertices of ζ. Let Ω be the space of all such sequences. For any ergodic, shift-invariant probability measure μ on Ω, define Ωμ to be the set of all μ-generic sequences, i.e., the set of all sequences ω such that each finite sequence v occurs in ω with limiting frequency μ(Ω(v)), where Ω(v) is the set of all ω′?Ω that begin with the word v. Then the Hausdorff dimension of Λ∩Φ−1 (Ωμ) in the metric d is almost surely on the event of nonextinction, where h(μ) is the entropy of the measure μ and q(i, j) is the mean number of type-j offspring of a type-i individual. This extends a theorem of HAWKES [5], which shows that the Hausdorff dimension of the entire boundary at infinity is log2 α, where α is the Malthusian parameter.

Anisotropic contact processes on homogeneous trees

Sufficient conditions are given for the existence of a weak survival phase in a homogeneous but not necessarily isotropic contact process on a homogeneous tree. These require that the contact process be homogeneous, that is, for any two vertices x,y of the tree there is an automorphism mapping x to y leaving the infection rates invariant; and that the contact process be weakly symmetric, that is, for each vertex there should be at least two incident edges with the same infection rate.

How Many Geometric (p) Samples does it take to See all the Balls in a Box

Reach into a box containing m balls and pull out a geometric (p) - sized sample. Then put the balls back into the box and sample again. Let X be the number of samples needed to see all m balls. We derive nonrecursive approximation formulas for the mean and standard deviation of X.