Jeffrey N. Rouder

Bayesian Analysis of Factorial Designs.

This article provides a Bayes factor approach to multiway analysis of variance (ANOVA) that allows researchers to state graded evidence for effects or invariances as determined by the data. ANOVA is conceptualized as a hierarchical model where levels are clustered within factors. The development is comprehensive in that it includes Bayes factors for fixed and random effects and for within-subjects, between-subjects, and mixed designs. Different model construction and comparison strategies are discussed, and an example is provided. We show how Bayes factors may be computed with BayesFactor package in R and with the JASP statistical package.

A multinomial modeling analysis of the mnemonic benefits of bizarre imagery

A series of experiments was conducted to explore the cognitive processes that mediate the bizarreness effect, that is, the finding that bizarre or unusual imagery is recalled better than common imagery. In all experiments, subjects were presented with noun pairs that were embedded within bizarre or common sentences ina mixed-list design. None of the experiments produced a bizarrenesB effect for cued recall; however, for two of the experiments, the bizarre noun pairs were remembered significantly better than the common pairs for free recall. To determine if these differences were due to the storage or retrieval of the items, a multinomial model for the analyis of imagery mediation in paired-associate learning was developed and applied to the data from the experiments. The model revealed that bizarre sentences benefited the retrieval of the noun pairs but not their storage within memory. The empirical and modeling results are discussed relative to previous findings and theories on thebizarreness effect.

Overstated evidence for short-term effects of violent games on affect and behavior: A reanalysis of Anderson et al. (2010).

Violent video games are theorized to be a significant cause of aggressive thoughts, feelings, and behaviors. Important evidence for this claim comes from a large meta-analysis by Anderson and colleagues (2010), who found effects of violent games in experimental, cross-sectional, and longitudinal research. In that meta-analysis, the authors argued that there is little publication or analytic bias in the literature, an argument supported by their use of the trim-and-fill procedure. In the present manuscript, we reexamine their meta-analysis using a wider array of techniques for detecting bias and adjusting effect sizes. Our conclusions differ from those of Anderson and colleagues in 3 salient ways. First, we detect substantial publication bias in experimental research on the effects of violent games on aggressive affect and aggressive behavior. Second, after adjustment for bias, the effects of violent games on aggressive behavior in experimental research are estimated as being very small, and estimates of effects on aggressive affect are much reduced. In contrast, the cross-sectional literature finds correlations that appear largely unbiased. Third, experiments meeting the original authors’ criteria for methodological quality do not yield larger adjusted effects than other experiments, but instead yield larger indications of bias, indicating that perhaps they were selected for significance. We outline future directions for stronger experimental research. The results indicate the need for an open, transparent, and preregistered research process to test the existence of the basic phenomenon.

Attaching meaning to the number words: contributions of the object tracking and approximate number systems.

Childrens understanding of the quantities represented by number words (i.e., cardinality) is a surprisingly protracted but foundational step in their learning of formal mathematics. The development of cardinal knowledge is related to one or two core, inherent systems - the approximate number system (ANS) and the object tracking system (OTS) - but whether these systems act alone, in concert, or antagonistically is debated. Longitudinal assessments of 198 preschool children on OTS, ANS, and cardinality tasks enabled testing of two single-mechanism (ANS-only and OTS-only) and two dual-mechanism models, controlling for intelligence, executive functions, preliteracy skills, and demographic factors. Measures of both OTS and ANS predicted cardinal knowledge in concert early in the school year, inconsistent with single-mechanism models. The ANS but not the OTS predicted cardinal knowledge later in the school year as well the acquisition of the cardinal principle, a critical shift in cardinal understanding. The results support a Merge model, whereby both systems initially contribute to childrens early mapping of number words to cardinal value, but the role of the OTS diminishes over time while that of the ANS continues to support cardinal knowledge as children come to understand the counting principles.

Continued misinterpretation of confidence intervals: response to Miller and Ulrich

Miller and Ulrich (2015) critique our claim (Hoekstra et al., Psychonomic Bulletin & Review, 21(5), 1157–1164, 2014), based on a survey given to researchers and students, of widespread misunderstanding of confidence intervals (CIs). They suggest that survey respondents may have interpreted the statements in the survey that we deemed incorrect in an idiosyncratic, but correct, way, thus calling into question the conclusion that the results indicate that respondents could not properly interpret CIs. Their alternative interpretations, while correct, cannot be deemed acceptable renderings of the questions in the survey due to the well-known reference class problem. Moreover, there is no support in the data for their contention that participants may have had their alternative interpretations in mind. Finally, their alternative interpretations are merely trivial restatements of the definition of a confidence interval, and have no implications for the location of a parameter.

Developing constraint in Bayesian mixed models.

Model comparison in Bayesian mixed models is becoming popular in psychological science. Here we develop a set of nested models that account for order restrictions across individuals in psychological tasks. An order-restricted model addresses the question “Does everybody,” as in “Does everybody show the usual Stroop effect,” or “Does everybody respond more quickly to intense noises than subtle ones?” The crux of the modeling is the instantiation of 10s or 100s of order restrictions simultaneously, one for each participant. To our knowledge, the problem is intractable in frequentist contexts but relatively straightforward in Bayesian ones. We develop a Bayes factor model-comparison strategy using Zellner and Siow’s default g-priors appropriate for assessing whether effects obey equality and order restrictions. We apply the methodology to seven data sets from Stroop, Simon, and Eriksen interference tasks. Not too surprisingly, we find that everybody Stroops—that is, for all people congruent colors are truly named more quickly than incongruent ones. But, perhaps surprisingly, we find these order constraints are violated for some people in the Simon task, that is, for these people spatially incongruent responses occur truly more quickly than congruent ones! Implications of the modeling and conjectures about the task-related differences are discussed.

Power, Dominance, and Constraint: A Note on the Appeal of Different Design Traditions

The recent field-wide emphasis on power has brought the number of participants used in psychological experiments into focus. Social psychology typically follows a tradition in which many participants perform a small number of trials each; in psychophysics, the tradition is to include only a few participants, who perform many trials each; and the tradition in cognitive psychology falls in between, balancing the number of participants and trials. We ask whether it is better to add trials or to add participants if one wishes to increase power. The answer is straightforward—greatest power is achieved by using more people, and the gain from adding people is greater than the gain from adding trials. In light of these results, the design parameters in the social psychology tradition seem ideal. Yet there are conditions in which one may trade people for trials with only a minor decrement in power. Under these conditions, the limiting factor is the trial-to-trial variability rather than the variability across people in the population. These conditions are highly plausible, and we present a theoretical argument as to why. We think that most cognitive effects are characterized by stochastic dominance; that is, everyone’s true effect is in the same direction. For example, it is plausible that when performing the Stroop task, all people truly identify congruent colors faster than incongruent ones. When dominance holds, small mean effects imply a small degree of variability across the population. It is this degree of homogeneity, the consequence of dominance, that licenses the design parameters of the cognitive psychology and psychophysics traditions.

How much evidence is p > .05? Stimulus pre-testing and null primary outcomes in violent video games research.

Research on the effects of violent video games frequently relies on arguments for the null hypothesis. Proponents of the effects argue that there are no meaningful differences save violent content between the violent and nonviolent games played, while critics of the effects argue that their nonsignificant study results constitute evidence for the null hypothesis of no difference. However, neither argument can be supported through the use of traditional null-hypothesis significance testing, as such tests can only ever reject or retain the null, never rejecting the alternative hypothesis in favor of the null. Therefore, to evaluate these claims, we apply a more appropriate Bayesian analysis to measure evidence for or against the null hypothesis relative to reasonable alternative hypotheses. We conclude that current methodological standards cannot rule out substantial confounds between violent and nonviolent video games. Furthermore, we find that studies that claim to find an absence of violent video game effects vary substantially in the strength of evidence, with some strongly supporting the null, others weakly supporting the null, and some others finding evidence of differences between conditions. We recommend the use of Bayesian analyses, larger sample sizes, and the creation of custom-designed games for experimental research.

Children’s early understanding of number predicts their later problem-solving sophistication in addition

Previous studies suggest that the sophistication of the strategies children use to solve arithmetic problems is related to a more basic understanding of number, but they have not examined the relation between number knowledge in preschool and strategy choices at school entry. Accordingly, the symbolic and nonsymbolic quantitative knowledge of 134 children (65 boys) was assessed at the beginning of preschool and in kindergarten, and the sophistication of the strategies they used to solve addition problems was assessed at the beginning of first grade. Using a combination of Bayes and standard regression models, we found that childrens understanding of the cardinal value of number words at the beginning of preschool predicted the sophistication of their strategy choices 3u202fyears later, controlling for other factors. The relation between childrens early understanding of cardinality and their strategy choices was mediated by their symbolic and nonsymbolic quantitative knowledge in kindergarten. The results suggest that sophisticated strategy choices emerge from childrens developing understanding of the relations among numbers, in keeping with the overlapping waves model.

Teaching Bayes' Theorem: Strength of Evidence As Predictive Accuracy

ABSTRACT Although teaching Bayes’ theorem is popular, the standard approach—targeting posterior distributions of parameters—may be improved. We advocate teaching Bayes’ theorem in a ratio form where the posterior beliefs relative to the prior beliefs equals the conditional probability of data relative to the marginal probability of data. This form leads to an interpretation that the strength of evidence is relative predictive accuracy. With this approach, students are encouraged to view Bayes’ theorem as an updating mechanism, to obtain a deeper appreciation of the role of the prior and of marginal data, and to view estimation and model comparison from a unified perspective.