Gunter Maris

Cognitive psychology meets psychometric theory: on the relation between process models for decision making and latent variable models for individual differences.

This article analyzes latent variable models from a cognitive psychology perspective. We start by discussing work by Tuerlinckx and De Boeck (2005), who proved that a diffusion model for 2-choice response processes entails a 2-parameter logistic item response theory (IRT) model for individual differences in the response data. Following this line of reasoning, we discuss the appropriateness of IRT for measuring abilities and bipolar traits, such as pro versus contra attitudes. Surprisingly, if a diffusion model underlies the response processes, IRT models are appropriate for bipolar traits but not for ability tests. A reconsideration of the concept of ability that is appropriate for such situations leads to a new item response model for accuracy and speed based on the idea that ability has a natural zero point. The model implies fundamentally new ways to think about guessing, response speed, and person fit in IRT. We discuss the relation between this model and existing models as well as implications for psychology and psychometrics.

Using Classical Test Theory in Combination with Item Response Theory

This study is about relations between classical test theory (CTT) and item response theory (IRT). It is shown that CTT is based on the assumption that measures are exchangeable, whereas IRT is based on conditional independence. Thus, IRT is presented as an extension of CTT, and concepts from both theories are related to one another. Furthermore, it is demonstrated that IRT can be used to provide CTT statistics in situations where CTT fails. Reliability, for instance, can be determined even though a test was not administered to the intended population.

On Interpreting the Model Parameters for the Three Parameter Logistic Model.

This paper addresses two problems relating to the interpretability of the model parameters in the three parameter logistic model. First, it is shown that if the values of the discrimination parameters are all the same, the remaining parameters are nonidentifiable in a nontrivial way that involves not only ability and item difficulty, but also the guessing parameters. Second, a situation is considered where different researchers analyze the same test with different instances of the three parameter logistic model. One researcher reaches the conclusion that students guess, whereas the other one concludes that students do not guess. Both examples illustrate the many-one relation between statistical models and the probability distributions they imply, which is the overarching topic of this paper.

Bayesian inference for low-rank Ising networks

Estimating the structure of Ising networks is a notoriously difficult problem. We demonstrate that using a latent variable representation of the Ising network, we can employ a full-data-information approach to uncover the network structure. Thereby, only ignoring information encoded in the prior distribution (of the latent variables). The full-data-information approach avoids having to compute the partition function and is thus computationally feasible, even for networks with many nodes. We illustrate the full-data-information approach with the estimation of dense networks.

Detecting Halo Effects in Performance-Based Examinations

The main purpose of this article is to demonstrate how halo effects may be detected and quantified using two independent ratings of the same person. A practical illustration is given to show how halo effects can be avoided.

Testing the race model inequality: A nonparametric approach

Abstract This paper introduces a nonparametric procedure for testing the race model explanation of the redundant signals effect. The null hypothesis is the race model inequality derived from the race model by Miller (Cognitive Psychol. 14 (1982) 247). The construction of a nonparametric test is made possible by a small change in the usual experimental procedure. This change involves that whenever only a single stimulus is presented, its modality is determined independently from the previous trials. It is shown that the test procedure is consistent against every violation of the null hypothesis. The test procedure is developed for data from a single participant, but it can easily be extended to the testing of the null hypothesis across participants, and this is also shown in the paper.

Equivalent Diagnostic Classification Models.

Levy, R., & Mislevy, R. J (2004). Specifying and refining a measurement model for a computer-based interactive assessment. International Journal of Testing, 4, 333–369. Levy, R., Mislevy, R. J., & Sinharay, S. (in press). Posterior predictive model checking for multidimensionality in item response theory. Applied Psychological Measurement. Mislevy, R. J., & Levy, R. (2007). Bayesian psychometric modeling from an evidence-centered design perspective. In C. R. Rao and S. Sinharay (Eds.), Handbook of statistics, Volume 26 (pp. 839–865). North-Holland: Elsevier. Mislevy, R. J., Levy, R., Kroopnick, M., & Rutstein, D. (2008). Evidentiary foundations of mixture item response theory models. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in latent variable mixture models (pp. 149–175). Charlotte, NC: Information Age Publishing. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–62. National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press. Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art, Measurement 6(4), 219–262. Sijtsma, K. (2006). Psychometrics in psychological research: Role model or partner in science? Psychometrika, 71, 451–455. Sinharay, S., & Almond, R. G. (2007). Assessing fit of cognitively diagnostic models—a case study. Educational and Psychological Measurement, 67, 239–257. Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.

What can we learn from Plausible Values

In this paper, we show that the marginal distribution of plausible values is a consistent estimator of the true latent variable distribution, and, furthermore, that convergence is monotone in an embedding in which the number of items tends to infinity. We use this result to clarify some of the misconceptions that exist about plausible values, and also show how they can be used in the analyses of educational surveys.

Time-varying boundaries for diffusion models of decision making and response time

Diffusion models are widely-used and successful accounts of the time course of two-choice decision making. Most diffusion models assume constant boundaries, which are the threshold levels of evidence that must be sampled from a stimulus to reach a decision. We summarize theoretical results from statistics that relate distributions of decisions and response times to diffusion models with time-varying boundaries. We then develop a computational method for finding time-varying boundaries from empirical data, and apply our new method to two problems. The first problem involves finding the time-varying boundaries that make diffusion models equivalent to the alternative sequential sampling class of accumulator models. The second problem involves finding the time-varying boundaries, at the individual level, that best fit empirical data for perceptual stimuli that provide equal evidence for both decision alternatives. We discuss the theoretical and modeling implications of using time-varying boundaries in diffusion models, as well as the limitations and potential of our approach to their inference.

Three representations of the Ising model

Statistical models that analyse (pairwise) relations between variables encompass assumptions about the underlying mechanism that generated the associations in the observed data. In the present paper we demonstrate that three Ising model representations exist that, although each proposes a distinct theoretical explanation for the observed associations, are mathematically equivalent. This equivalence allows the researcher to interpret the results of one model in three different ways. We illustrate the ramifications of this by discussing concepts that are conceived as problematic in their traditional explanation, yet when interpreted in the context of another explanation make immediate sense.