Bruno Lecoutre

Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests

We investigated the way experienced users interpret Null Hypothesis Significance Testing (NHST) outcomes. An empirical study was designed to compare the reactions of two populations of NHST users, psychological researchers and professional applied statisticians, when faced with contradictory situations. The subjects were presented with the results of an experiment designed to test the efficacy of a drug by comparing two groups (treatment/placebo). Four situations were constructed by combining the outcome of the t test (significant vs. nonsignificant) and the observed difference between the two means D (large vs. small). Two of these situations appeared as conflicting (t significant/D small and t nonsignificant/D large). Three fundamental aspects of statistical inference of statistical inference were investigated by means of open questions: drawing inductive conclusions about the magnitude of the true difference from the data in hand, making predictions for future data, and making decisions about stopping ...

Two useful distributions for Bayesian predictive procedures under normal models

The K-prime and K-square distributions, involved in the Bayesian predictive distributions of standard t and F tests are investigated. They generalize the classical noncentral t and noncentral F distributions and can receive different characterizations. Their moments and their probability density and distribution functions are made explicit.

Frequentist performance of Bayesian inference with response‐adaptive designs

In controlled clinical trials, where minimizing treatment failures is crucial, response-adaptive designs are attractive competitors to 1:1 randomized designs for comparing the success rates φ(1) and φ(2) of two treatments. In these designs each new treatment assignment depends on previous outcomes through some predefined rule. Here Play-The-Winner (PW), Randomized Play-The-Winner (RPW), Drop-The-Loser, Generalized Drop-the-Loser and Doubly adaptive Biased Coin Designs are considered for new treatment assignments. As frequentist inference relies on complex sampling distributions in those designs, we investigate how Bayesian inference, based on two independent Beta prior distributions, performs from a frequentist point-of-view. Performance is assessed through coverage probabilities of interval estimation procedures, power and minimization of failure count. It is shown that Bayesian inference can be favorably compared to frequentist procedures where the latter are available. The power of response-adaptive designs is generally very close to the power of 1:1 randomized design. However, failure count savings are generally small, except for the PW and Doubly adaptive Biased Coin designs in particular ranges of the true success rates. The RPW assignment rule has the worst performance, while PW, Generalized Drop-the-Loser or Doubly adaptive Biased Coin Designs may outperform other designs depending on different particular ranges of the true success rates.

Implementing Bayesian predictive procedures: The K-prime and K-square distributions

The implementation of Bayesian predictive procedures under standard normal models is considered. Two distributions are of particular interest, the K-prime and K-square distributions. They also give exact inferences for simple and multiple correlation coefficients. Their cumulative distribution functions can be expressed in terms of infinite series of multiples of incomplete beta function ratios, thus adequate for recursive calculations. Efficient algorithms are provided. To deal with special cases where possible underflows may prevent a recurrence to work properly, a simple solution is proposed which results in a procedure which is intermediate between two classes of algorithm. Some examples of applications are given.

Adaptative Designs for Multi-Arm Clinical Trials: The Play-the-Winner Rule Revisited

Adaptative designs for clinical trials that are based on a generalization of the “play-the-winner” rule are considered as an alternative to previously developed models. Theoretical and numerical results show that these designs perform better for the usual criteria. Bayesian methods are proposed for the statistical analysis of these designs.

The Significance Test Controversy Revisited

This chapter revisits the significance test controversy in the light of Jeffreys’ views about the role of statistical inference in experimental investigations. These views have been clearly expressed in the third edition of his Theory of Probability. The relevant passage is quoted and commented. The elementary inference about the difference between two means is considered, but the conclusions are applicable to most of the usual situations encountered in experimental data analysis.

And if you were a Bayesian without knowing it

The literature is full of Bayesian interpretations of frequentist p‐values and confidence levels. All the attempts to rectify these interpretations have been a loosing battle. In fact such interpretations suggest that most users are likely to be Bayesian “without knowing it” and really want to make a different kind of inference.

Replication is not coincidence: Reply to Iverson, Lee, and Wagenmakers (2009)

Iverson, Lee, and Wagenmakers (2009) claimed that Killeen’s (2005) statistic prep overestimates the “true probability of replication.” We show that Iverson et al. confused the probability of replication of an observed direction of effect with a probability of coincidence—the probability that two future experiments will return the same sign. The theoretical analysis is punctuated with a simulation of the predictions of prep for a realistic random effects world of representative parameters, when those are unknown a priori. We emphasize throughout that prep is intended to evaluate the probability of a replication outcome after observations, not to estimate a parameter. Hence, the usual conventional criteria (unbiasedness, minimum variance estimator) for judging estimators are not appropriate for probabilities such as p and prep.

Failure to Construct and Transfer Correct Representations across Probability Problems

Previous studies carried out on “purely random” situations (with dice or poker chips) show the difficulties encountered by people in such situations, however simple they may be. In fact, in this type of situation, prior knowledge guides spontaneous representations, and the “errors” observed could be explained by the activation of “implicit models” which form the basis of erroneous representations. 42 statistically naïve undergraduates were given several variants of a probability problem on which errors are common. In a learning phase, subjects were given four problems involving geometric figures which were pairwise related by complementarity and equivalence relations. In a subsequent transfer phase, they were given a fifth problem involving poker chips, which was structurally isomorphic to the fourth geometric-figures problem. The findings show that people do not realize the relations between problems, and that transfer occurred only for the subset of subjects who performed correctly on the training problems of the learning phase. These results appear to have some significant implications in teaching mathematical concepts.

New Results for Computing Blaker’s Exact Confidence Interval for One Parameter Discrete Distributions

The authors state new general results for computing Blaker’s exact confidence interval limits for usual one-parameter discrete distributions. Specific results for implementing an accurate and fast algorithm are made explicit for the binomial, negative binomial, Poisson and hypergeometric model.