Anton Beguin

MCMC estimation and some model-fit analysis of multidimensional IRT models

A Bayesian procedure to estimate the three-parameter normal ogive model and a generalization of the procedure to a model with multidimensional ability parameters are presented. The procedure is a generalization of a procedure by Albert (1992) for estimating the two-parameter normal ogive model. The procedure supports analyzing data from multiple populations and incomplete designs. It is shown that restrictions can be imposed on the factor matrix for testing specific hypotheses about the ability structure. The technique is illustrated using simulated and real data.

Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design

Item response theory item parameters can be estimated using data from a common-item equating design either separately for each form or concurrently across forms. This paper reports the results of a simulation study of separate versus concurrent item parameter estimation. Using simulated data from a test with 60 dichotomous items, four factors were considered: (a) estimation program (MULTILOG versus BILOG-MG), (b) sample size per form (3,000 versus 1,000), (c) number of common items (20 versus 10), and (d) equivalent versus nonequivalent groups taking the two forms (no mean difference versus a mean difference of 1 SD). In addition, four methods of item parameter scaling were used in the separate estimation condition: two item characteristic curve methods (Stocking-Lord and Haebara) and two moment methods (Mean/Mean and Mean/Sigma). Concurrent estimation generally resulted in lower error than separate estimation, although not universally so. The results suggest that one factor accounting for the lower error when using concurrent estimation may be that the parameter estimates for the common item parameters are based on larger samples. It is argued that the results of this study, together with other research on this topic, are not sufficient to recommend completely avoiding separate estimation in favor of concurrent estimation.

Using Classical Test Theory in Combination with Item Response Theory

This study is about relations between classical test theory (CTT) and item response theory (IRT). It is shown that CTT is based on the assumption that measures are exchangeable, whereas IRT is based on conditional independence. Thus, IRT is presented as an extension of CTT, and concepts from both theories are related to one another. Furthermore, it is demonstrated that IRT can be used to provide CTT statistics in situations where CTT fails. Reliability, for instance, can be determined even though a test was not administered to the intended population.

Robustness of IRT Observed-Score Equating

One of the most important aspects of equating, scaling, and linking is whether the models used are appropriate. In this chapter, some heuristic methods and a more formal model test for the evaluation of the robustness of the procedures used for equating, scaling, and linking are presented. The methods are outlined in the framework of item response theory (IRT) observed score (OS) equating of number-correct (NC) scores (Kolen & Brennan, 1995; Zeng & Kolen, 1995). The methods for the evaluation of IRT-OS-NC equating will be demonstrated using concurrent estimation of the parameters of the one-parameter logistic (1PL) model and the three-parameter logistic (3PL) model. In this procedure the parameters are estimated on a common scale by using all available data simultaneously. The data used are from the national school-leaving examinations at the end of secondary education in the Netherlands. To put the presentation into perspective, the application will be presented first.