Sun-Joo Cho

Model Selection Methods for Mixture Dichotomous IRT Models.

This study examines model selection indices for use with dichotomous mixture item response theory (IRT) models. Five indices are considered: Akaikes information coefficient (AIC), Bayesian information coefficient (BIC), deviance information coefficient (DIC), pseudo-Bayes factor (PsBF), and posterior predictive model checks (PPMC). The five indices provide somewhat different recommendations for a set of real data. Results from a simulation study indicate that BIC selects the correct (i.e., the generating) model well under most conditions simulated and for all three of the dichotomous mixture IRT models considered. PsBF is almost as effective. AIC and PPMC tend to select the more complex model under some conditions. DIC is least effective for this use.

A Multilevel Mixture IRT Model With an Application to DIF

Mixture item response theory models have been suggested as a potentially useful methodology for identifying latent groups formed along secondary, possibly nuisance dimensions. In this article, we describe a multilevel mixture item response theory (IRT) model (MMixIRTM) that allows for the possibility that this nuisance dimensionality may function differently at different levels. A MMixIRT model is described that enables simultaneous detection of differences in latent class composition at both examinee and school levels. The MMixIRTM can be viewed as a combination of an IRT model, an unrestricted latent class model, and a multilevel model. A Bayesian estimation of the MMixIRTM is described including analysis of label switching, use of priors, and model selection strategies. Results of a simulation study indicated that the generated parameters were recovered very well for the conditions considered. Use of MMixIRTM also was illustrated with the standardized mathematics test.

Alternating imputation posterior estimation of models with crossed random effects

Generalized linear mixed models or latent variable models for categorical data are difficult to estimate if the random effects or latent variables vary at non-nested levels, such as persons and test items. Clayton and Rasbash (1999) suggested an Alternating Imputation Posterior (AIP) algorithm for approximate maximum likelihood estimation. For item response models with random item effects, the algorithm iterates between an item wing in which the item mean and variance are estimated for given person effects and a person wing in which the person mean and variance are estimated for given item effects. The person effects used for the item wing are sampled from the conditional posterior distribution estimated in the person wing and vice versa. Clayton and Rasbash (1999) used marginal quasi-likelihood (MQL) and penalized quasi-likelihood (PQL) estimation within the AIP algorithm, but this method has been shown to produce biased estimates in many situations, so we use maximum likelihood estimation with adaptive quadrature. We apply the proposed algorithm to the famous salamander mating data, comparing the estimates with many other methods, and to an educational testing dataset. We also present a simulation study to assess performance of the AIP algorithm and the Laplace approximation with different numbers of items and persons and a range of item and person variances.

Accuracy of the Parallel Analysis Procedure With Polychoric Correlations

The purpose of this study was to investigate the application of the parallel analysis (PA) method for choosing the number of factors in component analysis for situations in which data are dichotomous or ordinal. Although polychoric correlations are sometimes used as input for component analyses, the random data matrices generated for use in PA typically consist of Pearson correlations. In this study, the authors matched the type of random data matrix to the type of input matrix. Analyses were conducted on both polychoric and Pearson correlation matrices, and random matrices of the same type (polychoric or Pearson) were generated for the PA procedure. PA based on random Pearson correlations was found to perform at least as well as PA based on random polychoric correlations, for nearly all of the conditions studied.

Markov chain Monte Carlo estimation of a mixture item response theory model

Markov chain Monte Carlo (MCMC) algorithms have been shown to be useful for estimation of complex item response theory (IRT) models. Although an MCMC algorithm can be very useful, it also requires care in use and interpretation of results. In particular, MCMC algorithms generally make extensive use of priors on model parameters. In this paper, MCMC estimation is illustrated using a simple mixture IRT model, a mixture Rasch model (MRM), to demonstrate how the algorithm operates and how results may be affected by some commonly used priors. Priors on the probabilities of mixtures, label switching, model selection, metric anchoring, and implementation of the MCMC algorithm using WinBUGS are described, and their effects illustrated on parameter recovery in practical testing situations. In addition, an example is presented in which an MRM is fitted to a set of educational test data using the MCMC algorithm and a comparison is illustrated with results from three existing maximum likelihood estimation methods.

Explanatory secondary dimension modeling of latent differential item functioning

The models used in this article are secondary dimension mixture models with the potential to explain differential item functioning (DIF) between latent classes, called latent DIF. The focus is on models with a secondary dimension that is at the same time specific to the DIF latent class and linked to an item property. A description of the models is provided along with a means of estimating model parameters using easily available software and a description of how the models behave in two applications. One application concerns a test that is sensitive to speededness and the other is based on an arithmetic operations test where the division items show latent DIF.

Effects of Blended Instructional Models on Math Performance

A pretest-posttest cluster-randomized trial involving 31 middle schools and 335 students with disabilities tested the effects of combining explicit and anchored instruction on fraction computation and problem solving. Results of standardized and researcher-developed tests showed that students who were taught with the blended units outscored students in Business As Usual classes. Students made the largest gains in computing with fractions and on problems related to ratios, proportions, and geometry. The findings suggest important implications for the way curriculum is designed for middle school students with disabilities who exhibit low performance in math.

Item Response Theory Analyses of the Cambridge Face Memory Test (CFMT)

We evaluated the psychometric properties of the Cambridge Face Memory Test (CFMT; Duchaine & Nakayama, 2006). First, we assessed the dimensionality of the test with a bifactor exploratory factor analysis (EFA). This EFA analysis revealed a general factor and 3 specific factors clustered by targets of CFMT. However, the 3 specific factors appeared to be minor factors that can be ignored. Second, we fit a unidimensional item response model. This item response model showed that the CFMT items could discriminate individuals at different ability levels and covered a wide range of the ability continuum. We found the CFMT to be particularly precise for a wide range of ability levels. Third, we implemented item response theory (IRT) differential item functioning (DIF) analyses for each gender group and 2 age groups (age ≤ 20 vs. age > 21). This DIF analysis suggested little evidence of consequential differential functioning on the CFMT for these groups, supporting the use of the test to compare older to younger, or male to female, individuals. Fourth, we tested for a gender difference on the latent facial recognition ability with an explanatory item response model. We found a significant but small gender difference on the latent ability for face recognition, which was higher for women than men by 0.184, at age mean 23.2, controlling for linear and quadratic age effects. Finally, we discuss the practical considerations of the use of total scores versus IRT scale scores in applications of the CFMT.

Latent Transition Analysis With a Mixture Item Response Theory Measurement Model

A latent transition analysis (LTA) model was described with a mixture Rasch model (MRM) as the measurement model. Unlike the LTA, which was developed with a latent class measurement model, the LTA-MRM permits within-class variability on the latent variable, making it more useful for measuring treatment effects within latent classes. A simulation study indicated that model recovery using the LTA-MRM was good except for small sample size—short test conditions. A real data application of a mathematics intervention with middle school students indicated that the LTA-MRM clearly detected the intervention effect and also provided a means of helping to better understand the effects compared to a standard multiwave analysis of variance.

Are Increased Weight and Appetite Useful Indicators of Depression in Children and Adolescents

During childhood and adolescence, physiological, psychological, and behavioral processes strongly promote weight gain and increased appetite while also inhibiting weight loss and decreased appetite. The Diagnostic and Statistical Manual-IV (DSM-IV) treats both weight-gain/increased-appetite and weight-loss/decreased-appetite as symptoms of major depression during these developmental periods, despite the fact that one complements typical development and the other opposes it. To disentangle the developmental versus pathological correlates of weight and appetite disturbance in younger age groups, the current study examined symptoms of depression in an aggregated sample of 2307 children and adolescents, 47.25% of whom met criteria for major depressive disorder. A multigroup, multidimensional item response theory model generated three key results. First, weight loss and decreased appetite loaded strongly onto a general depression dimension; in contrast, weight gain and increased appetite did not. Instead, weight gain and increased appetite loaded onto a separate dimension that did not correlate strongly with general depression. Second, inclusion or exclusion of weight gain and increased appetite affected neither the nature of the general depression dimension nor the fidelity of major depressive disorder diagnosis. Third, the general depression dimension and the weight-gain/increased-appetite dimension showed different patterns across age and gender. In child and adolescent populations, these results call into question the utility of weight gain and increased appetite as indicators of depression. This has serious implications for the diagnostic criteria of depression in children and adolescents. These findings inform a revision of the DSM, with implications for the diagnosis of depression in this age group and for research on depression.