Diego Blum

Response Time Reduction Due to Retesting in Mental Speed Tests: A Meta-Analysis

As retest effects in cognitive ability tests have been investigated by various primary and meta-analytic studies, most studies from this area focus on score gains as a result of retesting. To the best of our knowledge, no meta-analytic study has been reported that provides sizable estimates of response time (RT) reductions due to retesting. This multilevel meta-analysis focuses on mental speed tasks, for which outcome measures often consist of RTs. The size of RT reduction due to retesting in mental speed tasks for up to four test administrations was analyzed based on 36 studies including 49 samples and 212 outcomes for a total sample size of 21,810. Significant RT reductions were found, which increased with the number of test administrations, without reaching a plateau. Larger RT reductions were observed in more complex mental speed tasks compared to simple ones, whereas age and test-retest interval mostly did not moderate the size of the effect. Although a high heterogeneity of effects exists, retest effects were shown to occur for mental speed tasks regarding RT outcomes and should thus be more thoroughly accounted for in applied and research settings.

Automatic generation of figural analogies with the IMak package

Automatic Item Generation (AIG) techniques are offering innovative ways to produce test items as they overcome many disadvantages involving standard item writing, such as time-consuming work and resource-intensive demands. Although this field is relatively new, it is progressing at a high speed, and several contributions have been accomplished. Nevertheless, a scarce amount of AIG software evidencing favorable psychometric properties of the generated items has been made accessible to the broad scientific community. This research had two goals: first, to present an empirical study of items produced with the aid of the Item Maker (IMak) package available online and, second, to present IMak itself for the automatic generation of figural analogies. We were particularly interested in assessing whether automatically created figural analogy rules could predict item psychometric difficulty. A total of 23 items were generated and administered to 307 participants, 49.51% from Germany. The mean age was 28.61 (SD = 10.19) and 57.65% of the participants were female. Results reveal adequate psychometric properties including convergent validity, that most of the manipulated rules contribute to item difficulty, and that rule-based difficulty prediction is possible to some extent. In other words, psychometric quality of the generated items is supported, which reveals the utility of the IMak package in assessment settings. Finally, the package is presented and its functions for figural analogy item generation are further described.