Timo M. Bechger

Using Classical Test Theory in Combination with Item Response Theory

This study is about relations between classical test theory (CTT) and item response theory (IRT). It is shown that CTT is based on the assumption that measures are exchangeable, whereas IRT is based on conditional independence. Thus, IRT is presented as an extension of CTT, and concepts from both theories are related to one another. Furthermore, it is demonstrated that IRT can be used to provide CTT statistics in situations where CTT fails. Reliability, for instance, can be determined even though a test was not administered to the intended population.

Equivalent Linear Logistic Test Models

This paper is about the Linear Logistic Test Model (LLTM). We demonstrate that there are infinitely many equivalent ways to specify a model. An implication is that there may well be many ways to change the specification of a given LLTM and achieve the same improvement in model fit. To illustrate this phenomenon, we analyze a real data set using a Lagrange multiplier test for the specification of the model. This Lagrange multiplier test is similar to the modification index used in structural equation modeling.

On Interpreting the Model Parameters for the Three Parameter Logistic Model.

This paper addresses two problems relating to the interpretability of the model parameters in the three parameter logistic model. First, it is shown that if the values of the discrimination parameters are all the same, the remaining parameters are nonidentifiable in a nontrivial way that involves not only ability and item difficulty, but also the guessing parameters. Second, a situation is considered where different researchers analyze the same test with different instances of the three parameter logistic model. One researcher reaches the conclusion that students guess, whereas the other one concludes that students do not guess. Both examples illustrate the many-one relation between statistical models and the probability distributions they imply, which is the overarching topic of this paper.

Bayesian inference for low-rank Ising networks

Estimating the structure of Ising networks is a notoriously difficult problem. We demonstrate that using a latent variable representation of the Ising network, we can employ a full-data-information approach to uncover the network structure. Thereby, only ignoring information encoded in the prior distribution (of the latent variables). The full-data-information approach avoids having to compute the partition function and is thus computationally feasible, even for networks with many nodes. We illustrate the full-data-information approach with the estimation of dense networks.

Detecting Halo Effects in Performance-Based Examinations

The main purpose of this article is to demonstrate how halo effects may be detected and quantified using two independent ratings of the same person. A practical illustration is given to show how halo effects can be avoided.

Equivalent Diagnostic Classification Models.

Levy, R., & Mislevy, R. J (2004). Specifying and refining a measurement model for a computer-based interactive assessment. International Journal of Testing, 4, 333–369. Levy, R., Mislevy, R. J., & Sinharay, S. (in press). Posterior predictive model checking for multidimensionality in item response theory. Applied Psychological Measurement. Mislevy, R. J., & Levy, R. (2007). Bayesian psychometric modeling from an evidence-centered design perspective. In C. R. Rao and S. Sinharay (Eds.), Handbook of statistics, Volume 26 (pp. 839–865). North-Holland: Elsevier. Mislevy, R. J., Levy, R., Kroopnick, M., & Rutstein, D. (2008). Evidentiary foundations of mixture item response theory models. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in latent variable mixture models (pp. 149–175). Charlotte, NC: Information Age Publishing. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–62. National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press. Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art, Measurement 6(4), 219–262. Sijtsma, K. (2006). Psychometrics in psychological research: Role model or partner in science? Psychometrika, 71, 451–455. Sinharay, S., & Almond, R. G. (2007). Assessing fit of cognitively diagnostic models—a case study. Educational and Psychological Measurement, 67, 239–257. Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.

What can we learn from Plausible Values

In this paper, we show that the marginal distribution of plausible values is a consistent estimator of the true latent variable distribution, and, furthermore, that convergence is monotone in an embedding in which the number of items tends to infinity. We use this result to clarify some of the misconceptions that exist about plausible values, and also show how they can be used in the analyses of educational surveys.

Identifiability of nonlinear logistic test models

The linear logistic test model (LLTM) specifies the item parameters as a weighted sum of basic parameters. The LLTM is a special case of a more general nonlinear logistic test model (NLTM) where the weights are partially unknown. This paper is about the identifiability of the NLTM. Sufficient and necessary conditions for global identifiability are presented for a NLTM where the weights are linear functions, while conditions for local identifiability are shown to require a model with less restrictions. It is also discussed how these conditions are checked using an algorithm due to Bekker, Merckens, and Wansbeek (1994). Several illustrations are given.

Using Structural Equation Modeling To Fit Models Incorporating Principal Components

The aim of this article is to consider models incorporating principal components from the perspective of structural equation modeling. These models include the principal component analysis of patterned matrices, multiple analysis of variance based on principal components, and multigroup principal component analysis. We demonstrate that these models can be fit readily using the programs LISREL 8 and Mx. The models and certain extensions are discussed, and several illustrations are given.

Equivalent MIRID models

It is shown that in the context of the Model with Internal Restrictions on the Item Difficulties (MIRID), different componential theories about an item set may lead to equivalent models. Furthermore, we provide conditions for the identifiability of the MIRID model parameters, and it will be shown how the MIRID model relates to the Linear Logistic Test Model (LLTM). While it is known that the LLTM is a special case of the MIRID, we show that it is possible to construct an LLTM that encompasses the MIRID. The MIRID model places a bilinear restriction on the item parameters of the Rasch model. It is explained how this fact is used to simplify the results of Bechger, Verhelst, and Verstralen (2001) and Bechger, Verstralen, and Verhelst (2002), and extend their scope to a wider class of models.