Alexander Robitzsch

The multilevel latent covariate model: a new, more reliable approach to group-level effects in contextual studies.

In multilevel modeling (MLM), group-level (L2) characteristics are often measured by aggregating individual-level (L1) characteristics within each group so as to assess contextual effects (e.g., group-average effects of socioeconomic status, achievement, climate). Most previous applications have used a multilevel manifest covariate (MMC) approach, in which the observed (manifest) group mean is assumed to be perfectly reliable. This article demonstrates mathematically and with simulation results that this MMC approach can result in substantially biased estimates of contextual effects and can substantially underestimate the associated standard errors, depending on the number of L1 individuals per group, the number of groups, the intraclass correlation, the sampling ratio (the percentage of cases within each group sampled), and the nature of the data. To address this pervasive problem, the authors introduce a new multilevel latent covariate (MLC) approach that corrects for unreliability at L2 and results in unbiased estimates of L2 constructs under appropriate conditions. However, under some circumstances when the sampling ratio approaches 100%, the MMC approach provides more accurate estimates. Based on 3 simulations and 2 real-data applications, the authors evaluate the MMC and MLC approaches and suggest when researchers should most appropriately use one, the other, or a combination of both approaches.

Impact of Missing Data on the Detection of Differential Item Functioning: The Case of Mantel-Haenszel and Logistic Regression Analysis

This article describes the results of a simulation study to investigate the impact of missing data on the detection of differential item functioning (DIF). Specifically, it investigates how four methods for dealing with missing data (listwise deletion, zero imputation, two-way imputation, response function imputation) interact with two methods of DIF detection (Mantel-Haenszel statistic, logistic regression analysis) under three mechanisms of missingness (data missing completely at random, data missing at random, and data missing not at random) to produce over- or underestimates of the DIF effect sizes and detection rates. Results show that the interaction effects between missingness mechanism, treatment, and rate are most influential for explaining variation in bias, root mean square errors, and rejection rates. An incorrect treatment of missing data can thus lead to severe increases of Type I and Type II error rates. However, the choice between the two DIF detection methods investigated in this study is not important.

Have Cognitive Diagnostic Models Delivered Their Goods? Some Substantial and Methodological Concerns

Falmagne, J.-C., et al. (1990), Introduction to knowledge spaces: How to build, test, and search them. Psychological Review, 97, 201–224. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26(4), 301–323. Jaeger, J., Tatsuoka, C., Berns, S., & Varadi, F. (2006), Distinguishing neurocognitive functions using partially ordered classification models. Schizophrenia Bulletin, 32, 679–691. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272. Rupp, A. A. & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6(4), 219–262. Tatsuoka, C. (1996), Sequential classification on partially ordered sets, Doctoral dissertation, Cornell University. Tatsuoka, C. (2002). Data-analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society (Series C, Applied Statistics), 51, 337–350. Tatsuoka, C. (2005), Corrigendum to “Data analytic methods for latent partially ordered classification models.” Applied Statistics, 54, 465–467. Tatsuoka, C., & Ferguson, T. (2003), Sequential classification on partially ordered sets. Journal of the Royal Statistical Society, Series B, 143–157. Tatsuoka, K. (1991), Boolean algebra applied to determination of universal set of knowledge states. ETS Technical Report. Tatsuoka, K. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 327–360). Hillsdale, NJ: Erlbaum.

Effects of Mini-Games for Enhancing Multiplicative Abilities: A First Exploration

Developing knowledge and understanding of multiplicative relations is a main goal of primary school mathematics education. It is important that students consolidate basic multiplication table facts as well as learn how to flexibly apply this knowledge in more complex multiplicative problems [e.g., 1, 2]. Mathematical computer games are considered to contribute to attaining both these learning goals [e.g., 3]. However, as recent review articles have pointed out, clear empirical evidence of the effects of educational computer games is sparse, and in-class longitudinal studies are needed [e.g., 4, 5]. In the BRXXX study we use a large-scale longitudinal design to provide evidence for the domain of multiplication and division. Moreover, our study goes beyond the use of computer games in class and also includes playing games at home. We investigate the effects of multiplication and division mini-games from the popular website RekenWeb (www.rekenweb.nl).