Mary A. Pyc

Why Testing Improves Memory: Mediator Effectiveness Hypothesis

Testing not only evaluates the state of memory, but also improves memory more than restudy. A wealth of research has established that practice tests improve memory for the tested material. Although the benefits of practice tests are well documented, the mechanisms underlying testing effects are not well understood. We propose the mediator effectiveness hypothesis, which states that more-effective mediators (that is, information linking cues to targets) are generated during practice involving tests with restudy versus during restudy only. Effective mediators must be retrievable at time of test and must elicit the target response. We evaluated these two components of mediator effectiveness for learning foreign language translations during practice involving either test-restudy or restudy only. Supporting the mediator effectiveness hypothesis, test-restudy practice resulted in mediators that were more likely to be retrieved and more likely to elicit targets on a final test.

Why is test-restudy practice beneficial for memory? An evaluation of the mediator shift hypothesis.

Although the memorial benefits of testing are well established empirically, the mechanisms underlying this benefit are not well understood. The authors evaluated the mediator shift hypothesis, which states that test-restudy practice is beneficial for memory because retrieval failures during practice allow individuals to evaluate the effectiveness of mediators and to shift from less effective to more effective mediators. Across a series of experiments, participants used a keyword encoding strategy to learn word pairs with test-restudy practice or restudy only. Robust testing effects were obtained in all experiments, and results supported predictions of the mediator shift hypothesis. First, a greater proportion of keyword shifts occurred during test-restudy practice versus restudy practice. Second, a greater proportion of keyword shifts occurred after retrieval failure trials versus retrieval success trials during test-restudy practice. Third, a greater proportion of keywords were recalled on a final keyword recall test after test-restudy versus restudy practice.

Examining the efficiency of schedules of distributed retrieval practice

Given that students typically have a sizeable amount of course material to learn but a finite amount of study time, evaluating the efficiency of study schedules is important. We explored the efficiency of various schedules of distributed retrieval plus restudy. Across two experiments, 227 undergraduates were asked to learn Swahili—English vocabulary word pairs. Inconventional schedule groups, all items were presented for 3 practice trials after initial study (as in most previous research). Indropout schedule groups, the number of practice trials allocated to each item varied, in that practice with a given item was discontinued after criterion performance had been reached. A dropout schedule led to levels of performance similar to those for conventional schedules (but in fewer trials), and it was particularly effective for learning initially incorrect items. However, the efficiency of the various schedules depended critically on the interval between presentations of an item. Results suggest that dropout can be a more efficient learning schedule for students than can conventional schedules of practice.

The interim test effect: testing prior material can facilitate the learning of new material.

A wealth of prior research has shown that testing can improve subsequent learning of the initially tested material. In contrast, only one recent study has shown that an interim test over prior material can improve learning of subsequent new material (i.e., an interim-test effect). Five experiments replicated and extended this initial work by exploring the extent to which interim test effects generalize to complex text material. Participants were prompted to recall each section of an expository text before moving on to study the next section, or were only prompted to recall after the final section. In all experiments, recall of the final, target section was greater when prior sections had received interim tests versus no interim tests. Experiment 3 established that the effect was due to interim testing in particular rather than to intervening activity in general. Experiment 4 established that the effect was not due to test expectancy differences. In contrast to prior research, Experiment 4 also provided evidence that the effect is not due to release from proactive interference. We discuss other possible mechanisms underlying interim-test effects with text, including shifting to more effective encoding strategies.

Toward an understanding of students' allocation of study time: Why do they decide to mass or space their practice?

Two recent studies have led to discrepant findings regarding students’ decisions to mass or space practice of to-be-learned items: Son (2004) reported that participants chose to mass practice of difficult items more than that of easy items, whereas Benjamin and Bird (2006) reported the opposite. To resolve this apparent discrepancy, we had participants study items during an initial trial and then decide whether the next study trial would be massed or spaced. Across three experiments, we systematically varied factors that differed across these studies. In general, the participants more often chose to space than to mass practice, but they did favor massed practice when items were very difficult to learn. Moreover, although previous hypotheses implicated metamemory causes for these effects, the present results indicate that nonmetamemory causes are also responsible, such as using spaced practice as a default and deciding to mass practice for items that had not been fully perceived during an initial study trial.

Establishing and explaining the testing effect in free recall for young children.

Research has consistently shown memory is enhanced when learning combines test trials with study trials compared with study trials alone (i.e., testing effect). However, the majority of work on testing effects has involved undergraduate students. In the current experiment, the authors examined testing effects in 2 groups of elementary school children. First and 3rd graders learned 1 list of words in a test-plus-restudy condition and a 2nd list of words in a restudy condition. In addition to examining the benefits of testing in children, the authors examined possible explanations for the memorial benefits of testing that have received initial support in adult populations (Zaromb & Roediger, 2010). Results showed that the benefits of testing extend to elementary school children and that testing enhanced item-specific processing but not relational processing. Third graders were also aware of the memorial benefits of testing, whereas 1st graders were not. Most important, even 1st graders benefit from testing, in part because testing allows them to capitalize on their ability to process item-specific information.

The Production Effect: Costs and Benefits in Free Recall

The production effect, the memorial benefit for information read aloud versus silently, has been touted as a simple memory improvement tool. The current experiments were designed to evaluate the relative costs and benefits of production using a free recall paradigm. Results extend beyond prior work showing a production effect only when production is manipulated within subject, not between, using a free recall paradigm. Furthermore, the results also indicate that the production effect is primarily driven by decreased memory for items read silently, not increased memory for items read aloud.

Normative multitrial recall performance, metacognitive judgments, and retrieval latencies for Lithuanian-English paired associates.

Memory researchers using paired associates have benefited greatly from the Swahili—English norms reported by Nelson and Dunlosky (1994). Given recent increases in the amount and kinds of research using paired associates, however, researchers would now benefit from an expanded set of normative measures for foreign language vocabulary words. We report data for 120 Lithuanian—English word pairs collected from 236 undergraduates. Participants completed three study-test trials and were asked to make metacognitive judgments for each item. We report normative recall performance, recall latencies, and error types for each item across trials, as well as the perceived difficulty of each item on the basis of metacognitive judgments.

Repeated retrieval practice and item difficulty: Does criterion learning eliminate item difficulty effects?

A wealth of previous research has established that retrieval practice promotes memory, particularly when retrieval is successful. Although successful retrieval promotes memory, it remains unclear whether successful retrieval promotes memory equally well for items of varying difficulty. Will easy items still outperform difficult items on a final test if all items have been correctly recalled equal numbers of times during practice? In two experiments, normatively difficult and easy Lithuanian–English word pairs were learned via test–restudy practice until each item had been correctly recalled a preassigned number of times (from 1 to 11 correct recalls). Despite equating the numbers of successful recalls during practice, performance on a delayed final cued-recall test was lower for difficult than for easy items. Experiment 2 was designed to diagnose whether the disadvantage for difficult items was due to deficits in cue memory, target memory, and/or associative memory. The results revealed a disadvantage for the difficult versus the easy items only on the associative recognition test, with no differences on cue recognition, and even an advantage on target recognition. Although successful retrieval enhanced memory for both difficult and easy items, equating retrieval success during practice did not eliminate normative item difficulty differences.

The mediator effectiveness hypothesis revisited

To the Editor — Pyc and Rawson1 proposed that test practice (versus restudy) enhances long-term memory because testing enhances the effectiveness of mediators generated during encoding. In that study, we evaluated two components of mediator effectiveness: mediator retrieval and decoding. Camerer et al.2 sought to replicate the mediator retrieval component of this mediator effectiveness hypothesis. We provided our program and spoke numerous times with the authors regarding the method. Based on responses we received from the project team, the replication was a close methodological match to the original method1. They concluded2 that they replicated our findings in the second stage of data collection but not the first. Below, we discuss potential reasons why our results were not replicated in the first stage of data collection. The mediator effectiveness hypothesis provides a directional prediction about the relationship between learning schedule (study versus test) and the effectiveness of mediators generated during encoding. The first stage yielded a trend in the predicted direction that did not meet standard significance levels with a two-tailed test. Stage 2 provided results that met standard significance levels. An important measure that was collected but not reported by Camerer et al.2 is target recall on the final test. The mediator effectiveness hypothesis states that differences in mediator recall contribute to differences in target recall. If differences in target recall were not obtained, there would be no testing effect to be explained and therefore no expectation for differences in mediator recall. Likewise, if a weaker testing effect were observed than in the original study, one would expect weaker effects in mediator recall than in the original study. Without knowing target recall, the size of the expected mediator recall effect is not clear. Another factor that may have influenced the results is prior exposure to Swahili– English word pairs. These normed items from Nelson and Dunlosky3 have been widely used in the field. We ensured1 that participants had no prior exposure, but this information was not provided by Camerer et al.2. Prior exposure to the information could influence the effectiveness of test versus study practice. One final difference is in relation to the population sampled. We used undergraduate students only, and Camerer et al. used both undergraduate and graduate students. Their participants also received greater compensation. The possibility that testing effects of interest may be moderated by differences in participant characteristics is consistent with outcomes recently reported4. More generally, variability in effect size estimates across studies is expected. Instead of focusing solely on P values to determine replicability, Braver, Thoemmes and Rosenthal5 argue that a meta-analytic approach provides more appropriate criteria. A continuously cumulating meta-analysis (CCMA) of outcomes from the original study and from stage 1 data reported by Camerer et al. yielded a pooled d of 0.43 (95% CI = 0.12–0.74), P = 0.006, consistent with Braver and co-workers’ argument that “a CCMA analysis on a replication attempt that does not reach significance might nonetheless provide more, not less, evidence that the effect is real” (p. 333)5. Taking this approach into account, results of both stages 1 and 2 of Camerer et al. replicate the mediator retrieval assumption of the mediator effectiveness hypothesis. ❐