Anthony D. Albano

Statistical equating with measures of oral reading fluency

Recent research on curriculum-based measurement of oral reading fluency has revealed important issues in current passage development procedures, highlighting how dissimilar passages are problematic for monitoring student progress. The purpose of this paper is to describe statistical equating as an option for achieving equivalent scores across non-parallel reading passages. The psychometric and design properties of words-correct scores are examined, and the requirements of traditional equating methods are discussed. Simulated and empirical words-correct scores are used to demonstrate the steps in the equating process and the situations in which each method is most appropriate.

Redefining Individual Growth and Development Indicators: Oral language

Language skills developed during preschool contribute strongly to later reading and academic achievement. Effective preschool assessment and intervention should focus on core components of language development, specifically oral language skills. The Early Language and Literacy Individual Growth and Development Indicators (IGDIs) are a set of measures that have demonstrated utility in language and early literacy assessment, evaluation, and intervention studies. The purpose of this article is to describe two studies conducted to build on the existing oral language IGDI, Picture Naming, to increase utility for instructional decision making, particularly within an early childhood Response to Intervention framework. Study 1 outlines the initial development and piloting of potential measures and Study 2 describes the revision and continued analysis of the most promising measures. Results of Study 1 and Study 2 indicate that Picture Naming has continued utility as a measure of oral language skill development and that one newly developed oral language IGDI, Definitional Vocabulary, has potential utility.

Nominal Weights Mean Equating A Method for Very Small Samples

The authors introduced nominal weights mean equating, a simplified version of Tucker equating, as an alternative for dealing with very small samples. The authors then conducted three simulation studies to compare nominal weights mean equating to six other equating methods under the nonequivalent groups anchor test design with sample sizes of 20, 50, and 80 examinees. Results showed that nominal weights mean equating was generally the most effective. Nominal weights mean equating was, furthermore, never among the least effective methods in any condition, indicating its utility across a wide variety of contexts. Circle-arc equating, another recently developed method, also showed a great deal of promise. The identity function (i.e., no equating) was adequate only when test forms were nearly equivalent in difficulty.

Rasch Scale Stability in the Presence of Item Parameter and Trait Drift

Testing programs often rely on common-item equating to maintain a single measurement scale across multiple test administrations and multiple years. Changes over time, in the item parameters and the latent trait underlying the scale, can lead to inaccurate score comparisons and misclassifications of examinees. This study examined how instability in a scale and the items composing a scale affects item parameter recovery and classification accuracy. Results showed that a Rasch item response theory scale can maintain near baseline recovery properties if the changes in the latent trait over time are small. The Rasch scale also maintained good recovery of item and person parameters if there was equal item drift in both directions. Under conditions of relatively little item drift and small to moderate periodic changes in the latent trait, a Rasch scale may remain stable for 15 years, ±3. Substantial item drift or large changes in the latent trait can dramatically reduce the longevity of the scale.

Examining Differential Math Performance by Gender and Opportunity to Learn

Although a substantial amount of research has been conducted on differential item functioning in testing, studies have focused on detecting differential item functioning rather than on explaining how or why it may occur. Some recent work has explored sources of differential functioning using explanatory and multilevel item response models. This study uses hierarchical generalized linear modeling to examine differential performance due to gender and opportunity to learn, two variables that have been examined in the literature primarily in isolation, or in terms of mean performance as opposed to item performance. The relationships between item difficulty, gender, and opportunity to learn are explored using data for three countries from an international survey of preservice mathematics teachers.

Connecting Teacher Professional Development and Student Mathematics Achievement: A 4-Year Study of an Elementary Mathematics Specialist Program

The present study investigated the effects of Primarily Math, an inservice elementary mathematics specialist program. Primarily Math sought to augment the mathematical knowledge for teaching of kindergarten through third-grade teachers using a longitudinal multiple cohort design. Two sets of analyses were conducted. The first examined impact on teachers’ mathematical knowledge for teaching, attitudes toward learning mathematics, and beliefs about teaching and learning relative to a matched comparison group. Primarily Math teachers demonstrated greater knowledge for teaching Numbers and Operations and more positive attitudes toward learning mathematics, and more often endorsed student-centered beliefs about teaching and learning. The second set of analyses examined the extent to which students of three cohorts of Primarily Math teachers demonstrated more fall–spring growth in a measure of mathematics achievement relative to students of comparison-group teachers. There was a small but positive effect of participation in Primarily Math on student mathematics achievement.

Considering the Use of General and Modified Assessment Items in Computerized Adaptive Testing.

This article used several data sets from a large-scale state testing program to examine the feasibility of combining general and modified assessment items in computerized adaptive testing (CAT) for different groups of students. Results suggested that several of the assumptions made when employing this type of mixed-item CAT may not be met for students with disabilities that have typically taken alternate assessments based on modified achievement standards (AA-MAS). A simulation study indicated that the abilities of AA-MAS students can be underestimated or overestimated by the mixed-item CAT, depending on students’ location on the underlying ability scale. These findings held across grade levels and test lengths. The mixed-item CAT appeared to function well for non-AA-MAS students.

equate: Observed-Score Linking and Equating in R

Linking and equating are statistical procedures used to convert scores from one measurement scale to another. These procedures are most often used in testing programs that involve multiple test forms, where adjustments are made for form difficulty differences when creating a measurement scale that is common across forms. Linking and equating methods are traditionally distinguished by the type of scores they are applied to, whether observed scores or scores from an item response theory model. Methods are also distinguished by the study design under which measurements are taken. The R package equate (Albano, 2014) is free, open-source software for conducting observed-score linking and equating under single-group, equivalentgroups, and nonequivalent-groups designs with one or more anchor tests. The equate package estimates identity, mean, linear, and equipercentile equating functions (Kolen & Brennan, 2014); general linear functions (Albano, 2015); circle-arc functions (Livingston & Kim, 2009); and composites of these. Equating methods include nominal-weights (Babcock, Albano, & Raymond, 2012), Tucker, Levine observed-score, Levine truescore, Braun/Holland, frequency estimation, and chained equating. Plotting and summary methods are also provided, along with methods for multivariate loglinear presmoothing and for (parametric) bootstrap error estimation. The equate package is available for Linux, Mac, and Windows within the statistical environment R (R Core Team, 2015). Source and binary versions of the package and user manual are distributed free via the comprehensive R archival network, online at https://cran.r-project.org/. The latest development version of the package can be obtained online at https://github.com/

Examining Sources of Gender DIF in Mathematics Knowledge of Future Teachers Using Cross-Classified IRT Models

Research on differential item functioning (DIF) has focused traditionally on the detection of effects. However, recent studies have investigated potential sources of DIF, in an attempt to determine how or why it may occur. This study examines variability in item difficulty in math performance that is accounted for by gender, referred to as gender DIF, and the extent to which gender DIF is explained by both person predictors (opportunity to learn [OTL]) and item characteristics (item format). Cross-classified multilevel IRT models are used to examine the relationships among item difficulty, gender, OTL, and item format. Data come from the U.S. cohort of an international study of future math teachers, the Teacher Education and Development Study in Mathematics.

Item Development Research and Practice

Recent legislation and federal regulations in education have increased our attention to issues of inclusion, fairness, equity, and access in achievement testing. This has resulted in a growing literature on item writing for accessibility. This chapter provides an overview of the fundamentals of item development, especially as they pertain to accessible assessments. Constructed-response and selected-response items are first introduced and compared, with examples. Next, the item development process and guidelines for effective item writing are presented. Empirical research examining the item development process is then reviewed for general education items and items modified for accessibility. Methods for evaluating item quality, in regard to accessibility, are summarized. Finally, recent innovations and technological enhancements in item development, administration, and scoring are discussed.