R. Darrell Bock

Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm

Maximum likelihood estimation of item parameters in the marginal distribution, integrating over the distribution of ability, becomes practical when computing procedures based on an EM algorithm are used. By characterizing the ability distribution empirically, arbitrary assumptions about its form are avoided. The Em procedure is shown to apply to general item-response models lacking simple sufficient statistics for ability. This includes models with more than one latent dimension.

Estimating item parameters and latent ability when responses are scored in two or more nominal categories

A multivariate logistic latent trait model for items scored in two or more nominal categories is proposed. Statistical methods based on the model provide 1) estimation of two item parameters for each response alternative of each multiple choice item and 2) recovery of information from “wrong” responses when estimating latent ability. An application to a large sample of data for twenty vocabulary items shows excellent fit of the model according to a chi-square criterion. Item and test information curves are compared for estimation of ability assuming multiple category and dichotomous scoring of these items. Multiple scoring proves substantially more precise for subjects of less than median ability, and about equally precise for subjects above the median.

Fitting a response model forn dichotomously scored items

A method of estimating the parameters of the normal ogive model for dichotomously scored item-responses by maximum likelihood is demonstrated. Although the procedure requires numerical integration in order to evaluate the likelihood equations, a computer implemented Newton-Raphson solution is shown to be straightforward in other respects. Empirical tests of the procedure show that the resulting estimates are very similar to those based on a conventional analysis of item “difficulties” and first factor loadings obtained from the matrix of tetrachoric correlation coefficients. Problems of testing the fit of the model, and of obtaining invariant parameters are discussed.

Full-Information Item Factor Analysis

A method of item factor analysis based on Thur stones multiple-factor model and implemented by marginal maximum likelihood estimation and the EM algorithm is described. Statistical significance of suc cessive factors added to the model is tested by the likelihood ratio criterion. Provisions for effects of guessing on multiple-choice items, and for omitted and not-reached items, are included. Bayes constraints on the factor loadings are found to be necessary to suppress Heywood cases. Numerous applications to simulated and real data are presented to substantiate the accuracy and practical utility of the method. Index terms: Armed Services Vocational Aptitude Bat tery, Beta prior, EM algorithm, Item factor analysis, TESTFACT, Tetrachoric correlation.

Adaptive EAP Estimation of Ability in a Microcomputer Environment

Expected a posteriori (EAP) estimation of ability, based on numerical evaluation of the mean and variance of the posterior distribution, is shown to have unusually good properties for computerized adaptive testing. The calculations are not complex, precede noniteratively by simple summation of log likelihoods as items are added, and require only values of the response function obtainable from precalculated tables at a limited number of quadrature points. Simulation studies are reported showing the near equivalence of the posterior standard deviation and the standard error of measurement. When the adaptive testings terminate at a fixed posterior standard deviation criterion of .90 or better, the regression of the EAP estimator on true ability is virtually linear with slope equal to the reliability, and the measurement error homogeneous, in the range ± 2.5 standard deviations.

Multiple Group IRT

The extension of item response theory to data from more than one group of persons offers a unified approach to such problems as differential item functioning, item parameter drift, nonequivalent groups equating, vertical equating, two-stage testing, and matrix-sampled educational assessment. The common element in these problems is the existence of persons from different populations responding to the same test or to tests containing common items. In differential item functioning, the populations typically correspond to sex or demographic groups; in item parameter drift, to annual cohorts of students; in vertical equating, to children grouped by age or grade; in nonequivalent groups equating, to normative samples from different places or times; in two-stage testing, to examinees classified by levels of performance on a pretest; and in matrix-sampled educational assessment, to students from different schools or programs administered matrix-sampled assessment instruments. In all these settings, the objective of the multiplegroup analysis is to estimate jointly the item parameters and the latent distribution of a common attribute or ability of the persons in each of the populations.

Statistical Problems of Fitting Individual Growth Curves

A thorough-going longitudinal study of a child’s growth can produce upward of forty observations spaced over the years from birth to maturity. Such a data record is too long and inevitably too noisy (because of measurement error and short-run growth variation) to be interpreted without some sort of condensation and smoothing. The length of the record forces attention to certain critical regions or features of the curve, but the noisiness of the data makes it risky to characterize these regions or features by a few isolated measurements. The only safe approach to interpretation of individual growth data is via a statistical method capable of revealing the essential trend and concisely describing its main features.

Trend in correlated proportions

A random effects probit model is developed for the case in which the same units are sampled repeatedly at each level of an independent variable. Because the observed proportions may be correlated under these conditions, estimating their trend with respect to the independent variable is no longer a standard problem for probit, logit or loglinear analysis. Using a qualitative analogue of a random regressions model, we employ instead marginal maximum likelihood to estimate the average latent trend line. Likelihood ratio tests of the hypothesis of no trend in the average line, and the hypothesis of no differences in average trend lines between experimental treatments, are proposed. We illustrate the model both with simulated data and with observed data from a clinical experiment in which psychiatric patients on two drug therapies are rated on five occasions for the presence or absence of symptoms.

Individual growth in stature: A comparison of four growth studies in the U.S.A.

Patterns of growth in stature of subjects from the four major U.S. longitudinal growth studies are compared by means of a two-component model for individual growth. Problems inherent in the comparison of data from independent growth studies, such as those arising from different methods of sampling subjects and scheduling measurements and the occurrence of atypical subjects, are considered and solutions are offered. Statistical tests of the individual growth parameters revealed significant, but small, differences among the samples in the magnitude of the contributions of prepubertal and adolescent growth to mature stature and in the velocity of growth. No differences between samples in the timing of the adolescent component were detected.

The Next Stage in Educational Assessment

of maturity. It shows every sign of fulfilling its early promise and becoming a permanent and vital part of American education. Here we will review some of the events that shaped assessment as it is presently practiced and will suggest some logical next steps for the national and state programs. Our discussion of national assessment refers to the National Assessment of Educational Progress (NAEP), presently a function of the National Institute of Education (NIE) and administered by the Education Commission for the States. As a model of state assessment, we will take the California Assess-