Elias Walsh

Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels

ABSTRACT It is common in the implementation of teacher accountability systems to use empirical Bayes shrinkage to adjust teacher value-added estimates by their level of precision. Because value-added estimates based on fewer students and students with “hard-to-predict” achievement will be less precise, the procedure could have differential impacts on the probability that the teachers of fewer students or students with hard-to-predict achievement will be assigned consequences. This article investigates how shrinkage affects the value-added estimates of teachers of hard-to-predict students. We found that teachers of students with low prior achievement and who receive free lunch tend to have less precise value-added estimates. However, in our sample, shrinkage had no statistically significant effect on the relative probability that teachers of hard-to-predict students received consequences.

How Does Value Added Compare to Student Growth Percentiles

We compare teacher evaluation scores from a typical value-added model to results from the Colorado Growth Model (CGM), which 16 states currently use or plan to use as a component of their teacher performance evaluations. The CGM assigns a growth percentile to each student by comparing each students achievement to that of other students with similar past test scores. The median (or average) growth percentile of a teachers students provides the measure of teacher effectiveness. The CGM does not account for other student background characteristics and excludes other features included in many value-added models used by states and school districts. Using data from the District of Columbia Public Schools (DCPS), we examine changes in evaluation scores across the two methods for all teachers and for teacher subgroups. We find that use of growth percentiles in place of value added would have altered evaluation consequences for 14% of DCPS teachers. Most differences in evaluation scores based on the two methods are not related to the characteristics of students’ teachers.

Accounting for Co-Teaching: A Guide for Policymakers and Developers of Value-Added Models

Abstract We outline the options available to policymakers for addressing co-teaching in a value-added model. Building on earlier work, we propose an improvement to a method of accounting for co-teaching that treats co-teachers as teams, with each teacher receiving equal credit for co-taught students. Hock and Isenberg (2012) described a method known as the Full Roster Method (FRM) that is feasible and practical, but it effectively counts co-taught students more than once—these students receive a full weight with each of their teachers, so such students receive extra weight when calculating the relationship between student characteristics and achievement. The improvement, known as the Full Roster-Plus Method, allows co-taught students to receive full weight with their teachers, but all students contribute equally to the calculation of the relationship between student characteristics and achievement. To investigate how the application of this method empirically changes value-added estimates, we use data from District of Columbia Public Schools. We find that there are very small empirical differences between the two methods.

Elementary School Data Issues for Value-Added Models: Implications for Research.

Abstract Researchers often presume that it is better to use administrative data from grades 4 and 5 than data from grades 6 through 8 for conducting research on teacher effectiveness that uses value-added models because (1) elementary school teachers teach all subjects to their students in self-contained classrooms and (2) classrooms are more homogenous at the elementary school level. We examined the first issue by using data on teacher–student links in which teachers of mathematics and/or English/language arts had verified the subjects and students they taught. We compared these data to teacher–student links from the original administrative data. Results show that instruction is often departmentalized in these grades. About one in six elementary school teachers in the original data was linked to a subject that he or she did not teach. To examine the second issue, we computed the variation in baseline student achievement within classes, between classes at the same school, and between schools. We found more within-school variation in pretest scores in middle school grades but an offsetting amount of between-school variation in upper elementary grades.