Joseph F. T. Nese

Applied Curriculum-Based Measurement as a Predictor of High-Stakes Assessment: Implications for Researchers and Teachers

As part of a Response to Intervention (RTI) model, curriculum-based measurement (CBM) is an evidence-based assessment system that uses screening and formative assessments to help guide instruction. The two primary purposes of this study are to examine the relations between the easyCBM benchmark measures and a statewide large-scale assessment and to establish the optimal cut scores for the CBM measures. The easyCBM benchmark measures are used in 13 states through an online assessment system designed for use in an RTI context. Utility of the assessment system is examined through evidence related to predictive and systemic validity evidence. Included are a regression and a receiver operating characteristics (ROC) analysis of the relation between the benchmark assessments and a statewide large-scale reading test using a sample of approximately 3,600 students in grades 4 and 5. Results indicate strong concurrent validity and show the vocabulary measure as a robust predictor across grades.

In search of average growth: Describing within-year oral reading fluency growth across Grades 1–8☆☆☆

Measures of oral reading fluency (ORF) are perhaps the most often used assessment to monitor student progress as part of a response to intervention (RTI) model. Rates of growth in research and aim lines in practice are used to characterize student growth; in either case, growth is generally defined as linear, increasing at a constant rate. Recent research suggests ORF growth follows a nonlinear trajectory, but limitations related to the datasets used in such studies, composed of only three testing occasions, curtails their ability to examine the true functional form of ORF growth. The purpose of this study was to model within-year ORF growth using up to eight testing occasions for 1448 students in Grades 1 to 8 to assess (a) the average growth trajectory for within-year ORF growth, (b) whether students vary significantly in within-year ORF growth, and (c) the extent to which findings are consistent across grades. Results demonstrated that for Grades 1 to 7, a quadratic growth model fit better than either linear or cubic growth models, and for Grade 8, there was no substantial, stable growth. Findings suggest that the expectation for linear growth currently used in practice may be unrealistic.

Modeling Nonlinear Growth with Three Data Points: Illustration with Benchmarking Data.

The purpose of this article is to demonstrate ways to model nonlinear growth using three testing occasions. We demonstrate our growth models in the context of curriculum-based measurement using the fall, winter, and spring passage reading fluency benchmark assessments. We present a brief technical overview that includes the limitations of a growth model with three time points, and how nonlinear growth can be modeled and the associated limitations. We present results for a piecewise growth mixture modeling approach to model nonlinear growth for 1 to 3 classes, as well as to further explain individual differences and to capture heterogeneity of growth patterns. We discuss our interpretation of these results, as well as the implications of different methods for modeling nonlinear growth with three occasions.

Growth and gaps in mathematics achievement of students with and without disabilities on a statewide achievement test.

This study estimated mathematics achievement growth trajectories in a statewide sample of 92,045 students with and without disabilities over Grades 3 to 7. Students with disabilities (SWDs) were identified in seven exceptionality categories. Students without disabilities (SWoDs) were categorized as General Education (GE) or Academically/Intellectually Gifted (AIG). Students in all groups showed significant growth that decelerated over grades as well as significant variability in achievement by student group, both at the initial assessment in Grade 3 and in rates of growth over time. Race/ethnicity, gender, parental education, free/reduced lunch status, and English language proficiency were also significant predictors of achievement. Effect size estimates showed substantial year-to-year growth that decreased over grades. Sizeable achievement gaps that were relatively stable over grades were observed between SWoDs and students in specific exceptionality categories. Our study also demonstrated the importance of statistically controlling for variation related to student demographic characteristics. Additional research is needed that expands on these results with the same and additional exceptionality groups.

Classification Accuracy of EasyCBM First-Grade Mathematics Measures: Findings and Implications for the Field.

The results from a year-long study to investigate the technical features of easyCBM mathematics screening measures for first grade are presented. Measures were designed based on the National Council of Teachers of Mathematics Curriculum Focal Point standards, which for Grade 1 include (a) Number and Operations and Algebra, (b) Number and Operations, and (c) Geometry (2006). A sample of 145 students was assessed at the beginning, middle, and end of the school year. Reliability of the measures was adequate, and concurrent and predictive validity results—with the TerraNova 3—were moderate. Receiver operating curve analyses, based on mathematics difficulty below the 25th and 40th percentile, indicated adequate classification accuracy. We examine results from the study in the context of linking assessment to instructional decision making.

Growth on Oral Reading Fluency Measures as a Function of Special Education and Measurement Sufficiency.

For 30 years, researchers have investigated oral reading fluency as a measure of growth in reading proficiency. Yet, little research has been done with these measures in the context of progress monitoring in Tier 2 systems. First, we document teachers’ progress-monitoring decisions on type of passage (on-grade or off-grade) and how often to administer them. Then, we use a two-level hierarchical linear model to document the effects on both intercept and slope as a function of student special education status and measurement sufficiency. Across Grades 3 to 5, teachers diagnostically document student performance with different grade-level measures and also target a group of Tier 2 students to monitor early and systematically throughout the year. This latter group starts out much lower but has a significantly different slope than those for whom progress monitoring is more diagnostic and infrequent.

Predicting Abandonment of School-wide Positive Behavioral Interventions and Supports

This study examines predictors of abandonment of evidence-based practices through descriptive analyses of extant state-level training data, fidelity of implementation data, and nationally reported school demographic data across 915 schools in three states implementing school-wide positive behavioral interventions and supports (SWPBIS). Schools included in this study were tracked for a 5-year period after initial training, yet some elected to abandon SWPBIS at various times during implementation. Results showed that a small proportion of schools in the sample abandoned SWPBIS (7%). Logistic regression analysis identified school locale as the only statistically significant predictor of SWPBIS abandonment with schools located in cities being more likely to abandon. Results are discussed in terms of addressing types of schools at greater risk for abandonment and the importance of state-level training and coaching support.

Patterns of Statewide Test Participation for Students With Significant Cognitive Disabilities

Students with significant cognitive disabilities are eligible to participate in two statewide testing options for accountability: alternate assessments or general assessments with appropriate accommodations. Participation guidelines are generally quite vague, leading to students “switching” test participation between years. In this study, we tracked test participation for two cohorts of students with a documented disability over 3 years. Results suggested approximately 25% of students who initially took the alternate assessment switched test type at least once, although patterns of switching were not consistent across disabilities. Students on the performance “bubble” were more likely to switch test participation. Test switching poses challenges for monitoring students’ academic growth within accountability frameworks.

Documenting Reading Achievement and Growth for Students Taking Alternate Assessments

Students with disabilities have been included in state accountability systems for more than a decade; however, only in the past few years have alternate assessments of alternate achievement standards (AA-AAS) become stable enough to allow examination of these students’ achievement growth. Using data from Oregon’s AA-AAS in Reading during the period 2008–2009 to 2010–2011, we examined the achievement growth for a sample of 1,061 elementary students using two growth models: a transition matrix and a multilevel linear growth model. The authors found with the transition matrix model that a majority of students remained at the same performance level from one year to the next, whereas with the multilevel linear growth model, students’ scores revealed small, but statistically meaningful, growth year to year. The article concludes by noting advantages and disadvantages of these models to characterize growth and their implications for policy and practice.

Applications of curriculum-based measures in making decisions with multiple reference points

We write this chapter using a historical discourse, both in the chronology of research and in the development that has occurred over the years with curriculum-based measurement (CBM). More practically, however, we depict the chronology in terms of the sequence of decisions that educators make as they provide special services to students with disabilities. In the first part of the chapter, we begin with a pair of seminal documents that were written in the late 1970s to begin the story of CBM. In the second part of the chapter, we begin with the first decision an educator needs to make in providing special services and then we continue through the chronology of decisions to affect change in learning for individual students. In the end, we conclude with the need to integrate these decisions with multiple references for interpreting data: normative to allocate resources, criterion to diagnose skill deficits, and individual to evaluate instruction.