Douglas R. Glasnapp

The Impact of Language Characteristics in Mathematics Test Items on the Performance of English Language Learners and Students with Disabilities.

The impact of language characteristics in mathematics test items on student performance was evaluated for students with disabilities (SWD) and English language learners (ELL) as well as a large general student sample. Relationships were examined for test items and students at 4th, 7th, and 10th grades. The individual test item was the unit of analysis. Student performance was represented by item difficulty, or the probability of answering the item correctly. Regression analyses were conducted to examine relationships between item linguistic characteristics as independent variables with item difficulty as the dependent variable. Language characteristics had moderate effects on item difficulty at 4th grade, dropping to small-to-medium effects at 10th grade. ELL and SWD groups were not disproportionately affected by language characteristics in these test items. Difficult mathematics vocabulary had a consistent effect on performance for all students at all grades. Ambiguous or multiple-meaning words increased item difficulty at 4th grade.

Qualitative or Quantitative Differences?: Latent Class Analysis of Mathematical Ability for Special Education Students

The current article investigates whether the mathematics achievement of students in special education can be used to identify those who share common cognitive skills that may not be in concordance with their disability labels. Latent class analysis of a comprehensive test of mathematics taken by fourth-grade students with various disabilities reveals that a model with 2 latent classes is adequate to characterize the latent structure of the data. A parallel relationship of response profiles across the 2 classes suggests differences in the levels of mathematical ability (quantitative), rather than differences in the type of mathematical ability (qualitative), between the 2 latent classes in terms of generic mathematical proficiency. Cross-validation on a separate data set with careful matching of content areas within the math test verified this conclusion. Although a significant relationship exists between the identified latent classes and various disabilities, the analysis also found common mathematical problem-solving behaviors across disability categories. Implications for intervention and limitations of the current study are discussed.

Comparative Reliabilities and Difficulties of the Multiple-Choice and True-False Formats

The present study was initiated to investigate the comparability of multiple-choice and true-false item formats when the time necessary to respond to each type of item was equated empirically. Also investigated was the relative difficulty of multiple-choice (MC), true true-false (Tf), and false true-false (tF) items mea suring the same content. Results indicated that true-false items result in a less reliable test than one using a four-option MC format, even when empirically determined differences in time needed to answer the respective formats were taken into account. When scores were corrected for guessing, the MC items were significantly easier than the true-false format.

Improving Assessment Validity for Students With Disabilities in Large-Scale Assessment Programs

A test designed with built-in modifications and covering the same grade-level mathematics content provided more precise measurement of mathematics achievement for lower performing students with disabilities. Fourth-grade students with disabilities took a test based on modified state curricular standards for their mandated statewide mathematics assessment. To link the modified test with the general test, a block of items was administered to students with and without disabilities who took the general mathematics assessment. Item difficulty and student mathematics ability parameters were estimated using item response theory (IRT) methodology. Results support the conclusion that a modified test, based on the same curricular objectives but providing a more targeted measurement of expected outcomes for lower achieving students, could be developed for this special population.

Change Scores and Regression Suppressor Conditions

The concept of change, growth or discrepancy scores continues to persist in spite of documented statistical and measurement deficiencies. Recent writings support the continued use of change scores by identifying specific conditions where raw change scores have high predictive validity potential. The present paper expands the discussion of change score methodology by relating the concept of change to suppressor variable conditions in a least square regression model. The domain of conditions necessary for a weighted changed score composite to emerge as an underlying construct is mapped and the information loss through arbitrary assignment of weights to a change composite is explored.

Effects of Estimation Bias on Multiple-Category Classification With an IRT-Based Adaptive Classification Procedure

The effects of five ability estimators, that is, maximum likelihood estimator, weighted likelihood estimator, maximum a posteriori, expected a posteriori, and Owens sequential estimator, on the performances of the item response theory–based adaptive classification procedure on multiple categories were studied via simulations. The following results were found. (a) The Bayesian estimators were more likely to misclassify examinees into an inward category because of their inward biases, when a fixed start value of zero was assigned to every examinee. (b) When moderately accurate start values were available, however, Bayesian estimators produced classifications that were slightly more accurate than was the maximum likelihood estimator or weighted likelihood estimator. Expected a posteriori was the procedure that produced the most accurate results among the three Bayesian methods. (c) All five estimators produced equivalent efficiencies in terms of number of items required, which was 50 or more items except for abilities that were less than -2.00 or greater than 2.00.

End-of-course and long-term retention outcomes for mastery and nonmastery learning paradigms

Using both end-of-course achievement outcomes and long-term cognitive retention as criteria, the present study provides comparative data on the effectiveness of a mastery and nonmastery approach to instruction. Differential effects across taxonomic levels were assessed for both criteria. The results indicated that mastery students performed significantly higher than non-mastery students for end-of-course outcomes at the highest taxonomic level and equally well for knowledge, comprehension, and application level outcomes. Retention differences were found for knowledge level outcomes only, with mastery students demonstrating significantly greater retention performance. The feasibility and desirability of implementing a learning for mastery paradigm in a single course at the college level are discussed relative to the magnitude of the present results.

The Test of Word-Finding in Discourse: Diagnostic Utility Evidence:

The Test of Word Finding in Discourse (TWF-D) was developed as a diagnostic instrument to identify children who potentially have word finding disorders defined by atypically high frequencies of word finding behaviors exhibited in verbal discourse. Validity evidence is provided by the present study demonstrating that the TWF-D efficiently and with a high degree of accuracy differentiates between children with clinically diagnosed word-finding disorders and normal learning children. A global index of word finding behaviors resulted in correct classification of over 90% of the children in both groups. Subsequent application of discriminant analysis results to a normative sample indicated that children identified using the TWF-D global index exhibited one of two primary word finding behavioral patterns: (a) atypically high substitution responses or atypically high reformulation, and (b) repetition responses in the presence of low substitution responses.

Effects of Item Characteristics on Psychometric Properties of Forced Choice Scales

Two studies examined the effects of indeterminate frequency modifiers, item length, and statement direction on response to forced choice scale items. A pilot study established relative intensities of nine adverbial modifiers. Study 1 (N = 320) showed no effect on item variances due to the treatments. Differences were found between mean responses of short and long items as well as between positively and negatively stated items. Study 2 (N = 110) investigated effects on scale reliability and validity related to different intensity levels of modifiers. It is concluded that differential responses to items may not reflect differences in the behavior referenced by the items but rather observed differences in individual responses may be due to the mechanical, grammatical or delimitive characteristics of the items.

Content-Sampling as an Evaluation and Research Technique

ITEM SAMPLING (matrix sampling) is defined as a procedure whereby a set of j items is randomly divided into k subsets of items. A population of subjects is also identified and randomly separated into k samples. Each of the k samples of subjects then receives one of the k random subsets of items. This process was first introduced by Lord (1962) as a viable procedure for the estimation of test norms. Subsequent research by other investigators (Cahen, Romber, and Zwirner, 1970; Cook and Stufflebeam, 1967; Lord and Novick, 1968; Plumlee, 1964; Shoemaker, 1970a, 1970b) has provided a wealth of supportive data regarding the utility of this model for estimating such norms. Item-sampling research also has been conducted to investigate the implications for context effects (Sirotnik, 1970), methods for estimating reliability and standard errors of item-sampled tests (Shoemaker, 1970b, 1972; Zimmerman, 1969), and its feasibility in the collection of data when measuring attitudes (Peterson and Anderson, 1971; Shoemaker, 1971). Cronbach (1963), and, more recently, Wiley (1970) have suggested that item-sampling could be a useful technique for classroom evaluation. Even though items are randomly sampled and assigned to individuals at random in this context, a more nearly appropriate description of this procedure would be contentsampling. Within this framework a wider range of course con-