Michael C. Rodriguez

Meta-Analysis of Coefficient Alpha.

The meta-analysis of coefficient alpha across many studies is becoming more common in psychology by a methodology labeled reliability generalization. Existing reliability generalization studies have not used the sampling distribution of coefficient alpha for precision weighting and other common meta-analytic procedures. A framework is provided for a statistically grounded meta-analysis of coefficient alpha using its sampling distribution. Two empirical examples are offered to illustrate these methods, and limitations of reliability generalization are described.

The Role of Classroom Assessment in Student Performance on TIMSS

This project evaluated the relationship between assessment practices and achievement and the mediating roles of student self-efficacy and effort. In part, this was based on a framework proposed by Brookhart (1997). The United States portion of the Third International Math and Science Study was used to estimate these relationships. Several student level characteristics were important explanatory variables regarding variation in mathematics achievement, including mathematics self-efficacy, effort, and level of uncontrollable attributions. At the classroom level, teacher assessment practices had significant relationships to classroom performance. In addition, cross-level interactions (between student characteristics and teacher practices) suggested that classroom assessment practices might uniquely interact with student characteristics in their role of motivating student effort and performance.

A Review of Latino Youth Development Research and a Call for an Asset Orientation

A focus on youth development is the strongest means of prevention of problems faced by Latino youths. Latino youths are the fastest growing segment of the U.S. population and have high rates of developmental deficits. However, youth development researchers have not attended to the inclusion and reporting of results for Latino youths. This study presents a challenge to youth development researchers, service providers, and policy makers to adopt a youth development orientation in their work. A comprehensive review of six youth development journals and two Latino-focused journals allowed for the evaluation of the inclusion of Latino youths. Of 1,010 empirical articles, 30% included Latino youths, 6% reported results for Latino youths, and less than3% focused on Latino youths. The researchers reporting results for Latino youths were largely unguided by specific theoretical frameworks and heavily deficit oriented. A framework to increase attention to Latino youths in developmental research is suggested.

Effects of Using Modified Items to Test Students with Persistent Academic Difficulties

This study investigated the effects of using modified items in achievement tests to enhance accessibility. An experiment determined whether tests composed of modified items would reduce the performance gap between students eligible for an alternate assessment based on modified achievement standards (AA-MAS) and students not eligible, and the impact on student proficiency levels. Three groups of eighth-grade students (N = 755) from four states took original and modified versions of reading and mathematics tests. Findings indicate modified item conditions were significantly easier for all students and modifications would result in more AA-MAS eligible students meeting proficiency status. Study limitations and follow-up research on item modifications and the performance of students with disabilities are discussed.

Modified Multiple-Choice Items for Alternate Assessments: Reliability, Difficulty, and Differential Boost.

Federal policy on alternate assessment based on modified academic achievement standards (AA-MAS) inspired this research. Specifically, an experimental study was conducted to determine whether tests composed of modified items would have the same level of reliability as tests composed of original items, and whether these modified items helped reduce the performance gap between AA-MAS eligible and ineligible students. Three groups of eighth-grade students (N = 755) defined by eligibility and disability status took original and modified versions of reading and mathematics tests. In a third condition, the students were provided limited reading support along with the modified items. Changes in reliability across groups and conditions for both the reading and mathematics tests were determined to be minimal. Mean item difficulties within the Rasch model were shown to decrease more for students who would be eligible for the AA-MAS than for non-eligible groups, revealing evidence of differential boost. Exploratory analyses indicated that shortening the question stem may be a highly effective modification, and that adding graphics to reading items may be a poor modification.

Impact of small-group tutoring interventions on the mathematical problem solving and achievement of third-grade students with mathematics difficulties

This intervention study compared the efficacy of small-group tutoring on the mathematics learning of third-grade students at risk for mathematics difficulty using either a school-provided standards-based curriculum (SBC) or a schema-based instruction (SBI) curriculum. The SBI curriculum placed particular emphasis on the underlying mathematical structure of additive problems to represent and solve word problems. At-risk students (N = 136) from 35 classrooms scoring below a proficiency level on their district accountability assessment were assigned randomly to treatment groups. Results indicated interaction effects on the word problem-solving (WPS) posttest and retention tests such that SBI students with higher incoming (pretest) WPS scores outperformed SBC students with higher pretest scores, whereas SBC students with lower pretest scores outperformed SBI students with lower pretest scores. No effects were found on number combinations automaticity, and mathematics and reading achievement. Implications to improve the problem-solving performance of at-risk students are discussed.

Designing a Measurement Framework for Response to Intervention in Early Childhood Programs

The overall architecture and major components of a measurement system designed and evaluated to support Response to Intervention (RTI) in the areas of language and literacy in early childhood programs are described. Efficient and reliable measurement is essential for implementing any viable RTI system, and implementing such a system in early childhood programs presents several special challenges. This article describes (a) the general approach taken to developing and evaluating this measurement system, including its conceptual and methodological foundations, methods, and results for developing items and scales for second-generation Individual Growth and Development Indicators (IGDIs 2.0), and (b) use of these scales in a larger decision-making framework for differentiating language and literacy instruction to match the needs of preschool children at risk of academic difficulties. The authors conclude with a review of some challenges associated with developing measurement for differentiated instruction in early childhood settings.

Statistical equating with measures of oral reading fluency

Recent research on curriculum-based measurement of oral reading fluency has revealed important issues in current passage development procedures, highlighting how dissimilar passages are problematic for monitoring student progress. The purpose of this paper is to describe statistical equating as an option for achieving equivalent scores across non-parallel reading passages. The psychometric and design properties of words-correct scores are examined, and the requirements of traditional equating methods are discussed. Simulated and empirical words-correct scores are used to demonstrate the steps in the equating process and the situations in which each method is most appropriate.

Effectiveness of Schema-Based Instruction for Improving Seventh-Grade Students’ Proportional Reasoning: A Randomized Experiment

Abstract This study examined the effect of schema-based instruction (SBI) on 7th-grade students’ mathematical problem-solving performance. SBI is an instructional intervention that emphasizes the role of mathematical structure in word problems and also provides students with a heuristic to self-monitor and aid problem solving. Using a pretest-intervention–posttest-retention test design, the study compared the learning outcomes for 1,163 students in 42 classrooms who were randomly assigned to treatment (SBI) or control condition. After 6 weeks of instruction, results of multilevel modeling indicated significant differences favoring the SBI condition in proportion problem solving involving ratios/rates and percents on an immediate posttest (g = 1.24) and on a 6-week retention test (g = 1.27). No significant difference between conditions was found for a test of transfer. These results demonstrate that SBI was more effective than students’ regular mathematics instruction.

Redefining Individual Growth and Development Indicators: Oral language

Language skills developed during preschool contribute strongly to later reading and academic achievement. Effective preschool assessment and intervention should focus on core components of language development, specifically oral language skills. The Early Language and Literacy Individual Growth and Development Indicators (IGDIs) are a set of measures that have demonstrated utility in language and early literacy assessment, evaluation, and intervention studies. The purpose of this article is to describe two studies conducted to build on the existing oral language IGDI, Picture Naming, to increase utility for instructional decision making, particularly within an early childhood Response to Intervention framework. Study 1 outlines the initial development and piloting of potential measures and Study 2 describes the revision and continued analysis of the most promising measures. Results of Study 1 and Study 2 indicate that Picture Naming has continued utility as a measure of oral language skill development and that one newly developed oral language IGDI, Definitional Vocabulary, has potential utility.