Xiangdong Yang

Qualitative or Quantitative Differences?: Latent Class Analysis of Mathematical Ability for Special Education Students

The current article investigates whether the mathematics achievement of students in special education can be used to identify those who share common cognitive skills that may not be in concordance with their disability labels. Latent class analysis of a comprehensive test of mathematics taken by fourth-grade students with various disabilities reveals that a model with 2 latent classes is adequate to characterize the latent structure of the data. A parallel relationship of response profiles across the 2 classes suggests differences in the levels of mathematical ability (quantitative), rather than differences in the type of mathematical ability (qualitative), between the 2 latent classes in terms of generic mathematical proficiency. Cross-validation on a separate data set with careful matching of content areas within the math test verified this conclusion. Although a significant relationship exists between the identified latent classes and various disabilities, the analysis also found common mathematical problem-solving behaviors across disability categories. Implications for intervention and limitations of the current study are discussed.

A MULTICOMPONENT LATENT TRAIT MODEL FOR DIAGNOSIS

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479–494, 1980; Embretson in Psychometrika, 49:175–186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.

Improving Assessment Validity for Students With Disabilities in Large-Scale Assessment Programs

A test designed with built-in modifications and covering the same grade-level mathematics content provided more precise measurement of mathematics achievement for lower performing students with disabilities. Fourth-grade students with disabilities took a test based on modified state curricular standards for their mandated statewide mathematics assessment. To link the modified test with the general test, a block of items was administered to students with and without disabilities who took the general mathematics assessment. Item difficulty and student mathematics ability parameters were estimated using item response theory (IRT) methodology. Results support the conclusion that a modified test, based on the same curricular objectives but providing a more targeted measurement of expected outcomes for lower achieving students, could be developed for this special population.

23 Automatic Item Generation and Cognitive Psychology

Publisher Summary This chapter discusses item generation and the role of item response theory (IRT) models that permit cognitive variables to predict item parameters. It presents an overview of the methods of item generation and the research requirements for application. It reviews both the item model approach and the cognitive design system approach to item generation. The item model approach has the advantage of being applicable to item generation relatively quickly as it requires a lesser cognitive foundation. The cognitive design approach has the advantages of explicating construct validity at the item level because the level and the source of cognitive complexity in an item are quantified. The chapter also describes psychometric models that are based on IRT. The models reviewed included the linear logistic test model (LLTM), the 2PL-constrained model, and the hierarchical IRT model. The latter has been shown to produce a broad family of models appropriate for item generation with certain constraints applied. The chapter illustrates some estimation procedures for the psychometric models and presents an example of automatic item generation to spatial ability.

Effects of Estimation Bias on Multiple-Category Classification With an IRT-Based Adaptive Classification Procedure

The effects of five ability estimators, that is, maximum likelihood estimator, weighted likelihood estimator, maximum a posteriori, expected a posteriori, and Owens sequential estimator, on the performances of the item response theory–based adaptive classification procedure on multiple categories were studied via simulations. The following results were found. (a) The Bayesian estimators were more likely to misclassify examinees into an inward category because of their inward biases, when a fixed start value of zero was assigned to every examinee. (b) When moderately accurate start values were available, however, Bayesian estimators produced classifications that were slightly more accurate than was the maximum likelihood estimator or weighted likelihood estimator. Expected a posteriori was the procedure that produced the most accurate results among the three Bayesian methods. (c) All five estimators produced equivalent efficiencies in terms of number of items required, which was 50 or more items except for abilities that were less than -2.00 or greater than 2.00.

Measuring Attentional Ability in Older Adults Development and Psychometric Evaluation of DriverScan

Although deficits in visual attention are often postulated as an important component of many declines in cognitive processing and functional outcomes in older adults, surprisingly little emphasis has been placed on evaluating psychometric instruments with which individual differences in visual attention ability can be assessed. This article reports the development and beginning psychometric evaluation of DriverScan, a change detection measure of attentional search for older adults. A constrained graded response model is used to approximate response speed and accuracy with categories of immediate, delayed, or no response. DriverScan items are shown to have excellent reliability over the studied sample, and the distribution of items is shown to adequately cover the difficulty continuum and to be maximally sensitive at distinguishing individuals with lower than average abilities (i.e., individuals with attention deficits). Item design features representing goaldirected and stimulus-driven attentional processing significantly predict item difficulty as hypothesized.