Catherine O. Fritz

Effect size estimates: Current use, calculations, and interpretation.

The Publication Manual of the American Psychological Association (American Psychological Association, 2001, American Psychological Association, 2010) calls for the reporting of effect sizes and their confidence intervals. Estimates of effect size are useful for determining the practical or theoretical importance of an effect, the relative contributions of factors, and the power of an analysis. We surveyed articles published in 2009 and 2010 in the Journal of Experimental Psychology: General, noting the statistical analyses reported and the associated reporting of effect size estimates. Effect sizes were reported for fewer than half of the analyses; no article reported a confidence interval for an effect size. The most often reported analysis was analysis of variance, and almost half of these reports were not accompanied by effect sizes. Partial η2 was the most commonly reported effect size estimate for analysis of variance. For t tests, 2/3 of the articles did not report an associated effect size estimate; Cohens d was the most often reported. We provide a straightforward guide to understanding, selecting, calculating, and interpreting effect sizes for many types of data and to methods for calculating effect size confidence intervals and power analysis.

Expanding retrieval practice: An effective aid to preschool children's learning.

The benefits of expanding retrieval practice for preschool children were explored in two experiments. In Experiment 1, three groups learned names for six plush toy pigs using expanding retrieval practice, a reward incentive, or a control condition. Reward did not significantly improve learning but retrieval practice doubled recall. In Experiment 2, three groups learned names to soft toys, comparing recall following massed elaborative study with either expanding retrieval practice or expanding re-presentation. Recall was tested after 1 minute, 1 day, and 2 days. A very large effect size (d = 1.9) indicated the very considerable benefit from expanding retrieval practice over the elaboration condition. Comparison with the re-presentation condition suggested that half of the benefit of expanding retrieval practice came from spaced scheduling and half from retrieval practice. Expanding retrieval practice provides an effective method to improve learning by young children.

Facial distinctiveness: Its measurement, distribution and influence on immediate and delayed recognition

It is conventionally assumed that many faces are relatively typical and few are distinctive (e.g. Valentine, 1991), producing a highly skewed distribution. However, Burton and Vokey (1998) argue that the distribution will be normal, and our review of previous research suggested this is true. In three studies we explored the distributions using different techniques to estimate distinctiveness. Both traditional ratings and pairwise selection produced normal distributions. However, ratings emphasizing the degree of deviation from a typical face were skewed towards the distinctive end of the scale. The instructions given when distinctiveness is rated may not necessarily oppose typicality with distinctiveness: a face that is relatively typical might also stand out in a crowd because of some particular feature, familiarity or a host of other reasons. In our fourth study, recognition memory was predicted by all of the distinctiveness measures, with the relationship being stronger after a 5-week delay than in the immediate test.

The improved name game: Better use of expanding retrieval practice

Morris and Fritz (2000) demonstrated the effectiveness of the name game as a retrieval-practice based technique for learning the names of group members. We hypothesised that a reversed version of the name game would be even more effective. Performance was contrasted with a no-retrieval condition that mimicked the name game in every way except for the retrieval of the names, allowing an estimate of the specific contribution of retrieval practice. The benefit of a few refresher rounds of the game after 2 weeks was also examined. The reversed name game was superior to the original name game and the refresher rounds benefited all groups. The very considerable superiority of the name game over the no-retrieval condition demonstrated the magnitude of the benefit of expanding retrieval practice.

Effect sizes in memory research

Effect sizes are omitted from many research articles and are rarely discussed. To help researchers evaluate effect sizes we collected values for the more commonly reported effect size measures (partial eta squared and d) from papers reporting memory research published in 2010. Cohens small, medium, and large generic guideline values for d mapped neatly onto the observed distributions, but his values for partial eta squared were considerably lower than those observed in current memory research. We recommend interpreting effect sizes in the context of either domain-specific guideline values agreed for an area of research or the distribution of effect size estimates from published research in the domain. We provide cumulative frequency tables for both partial eta squared and d enabling authors to report and consider not only the absolute size of observed effects but also the percentage of reported effects that are larger or smaller than those observed.

Part-set cuing of texts, scenes, and matrices

In four experiments, we extended the study of part-set cuing to expository texts and pictorial scenes. In Experiment 1, recall of expository text was tested with and without part-set cues in the same order as the original text; cues strongly impaired recall. Experiment 2 repeated Experiment 1 but used cues in random order and found significant but reduced impairment with cuing. Experiments 3 and 4 examined the part-set cuing of objects presented in a scene or matrix and found virtually no effect of cuing. More objects were recalled from the scene than from the matrix, indicating that the scenes organization aided memory, but the cues did not assist recall. These results extend the domains in which part-set cues have either impaired or failed to improve recall. Implications for education and eyewitness accounts are briefly considered.

Meeting the challenge of the Psychonomic Society’s 2012 Guidelines on Statistical Issues: Some success and some room for improvement

The Psychonomic Society (PS) adopted New Statistical Guidelines for Journals of the Psychonomic Society in November 2012. To evaluate changes in statistical reporting within and outside PS journals, we examined all empirical papers published in PS journals and in the Experimental Psychology Society journal, The Quarterly Journal of Experimental Psychology (QJEP), in 2013 and 2015, to describe these populations before and after effects of the Guidelines. Comparisons of the 2013 and 2015 PS papers reveal differences associated with the Guidelines, and QJEP provides a baseline of papers to reflect changes in reporting that are not directly influenced by the Guidelines. A priori power analyses increased from 5% to 11% in PS papers, but not in QJEP papers (2%). The reporting of effect sizes in PS papers increased from 61% to 70%, similar to the increase for QJEP from 58% to 71%. Only 18% of papers reported confidence intervals (CIs) for means; only two PS papers in 2015 reported CIs for effect sizes. Although variability statistics are important to understanding data, and to further analysis, they were only reported as numbers in just over half of the PS journal papers. Almost all PS and QJEP papers relied exclusively on null hypothesis significance testing to guide interpretation of the data. Changes associated with the Guidelines are in the desired direction with respect to reporting effect sizes and power analyses but are not yet reflected in researchers’ practices in describing their data, addressing data assumptions, and thinking beyond the p value when interpreting their data.

Failure of further learning: activities, structure and meaning

Previous research has shown that little benefit is achieved through spaced study and recall of text passages after the first recall attempt, an effect that we term the failure-of-further-learning. We hypothesized that the effect occurs because a situation model of the texts gist is formed when the text is first comprehended and is consolidated when recalled; it dominates later recall after verbatim memories of more recent study episodes have been lost. Experiments 1 and 2 attempted to circumvent the effect by varying the activities of participants and requiring interactive exploration. In both experiments, recall after four, weekly sessions showed little benefit beyond performance on the first recall. Experiment 3 interfered with the formation of an immediate situation model by introducing passages that were hard to comprehend without a title. Performance improved substantially across four sessions when titles were not supplied, but the standard effect was replicated when titles were given. Experiment 4 made verbatim memories available by incorporating all re-presentations and tests into one session; as predicted, recall improved over successive tests.