Heather C. Hill

Effects of Teachers’ Mathematical Knowledge for Teaching on Student Achievement:

This study explored whether and how teachers’ mathematical knowledge for teaching contributes to gains in students’ mathematics achievement. The authors used a linear mixed-model methodology in which first and third graders’ mathematical achievement gains over a year were nested within teachers, who in turn were nested within schools. They found that teachers’ mathematical knowledge was significantly related to student achievement gains in both first and third grades after controlling for key student- and teacher-level covariates. This result, while consonant with findings from the educational production function literature, was obtained via a measure focusing on the specialized mathematical knowledge and skills used in teaching mathematics. This finding provides support for policy initiatives designed to improve students’ mathematics achievement by improving teachers’ mathematical knowledge.

Instructional Policy and Classroom Performance: The Mathematics Reform in California.

Educational reformers increasingly seek to manipulate policies regarding assessment, curriculum, and professional development in order to improve instruction. They assume that manipulating these elements of instructional policy will change teachers’ practice, which will then improve student performance. We formalize these ideas into a rudimentary model of the relations among instructional policy, teaching, and learning. We propose that successful instructional policies are themselves instructional in nature: because teachers figure as a key connection between policy and practice, their opportunities to learn about and from policy are a crucial influence both on their practice and, at least indirectly, on student achievement. Using data from a 1994 survey of California elementary school teachers and 1994 student California Learning Assessment System (CLAS) scores, we examine the influence of assessment, curriculum, and professional development on teacher practice and student achievement. Our results bear out the usefulness of the model: under circumstances that we identify, policy can affect practice, and both can affect student performance.

Developing Measures of Teachers’ Mathematics Knowledge for Teaching

In this article we discuss efforts to design and empirically test measures of teachers’ content knowledge for teaching elementary mathematics. We begin by reviewing the literature on teacher knowledge, noting how scholars have organized such knowledge. Next we describe survey items we wrote to represent knowledge for teaching mathematics and results from factor analysis and scaling work with these items. We found that teachers’ knowledge for teaching elementary mathematics was multidimensional and included knowledge of various mathematical topics (e.g., number and operations, algebra) and domains (e.g., knowledge of content, knowledge of students and content). The constructs indicated by factor analysis formed psychometrically acceptable scales.

A Validity Argument Approach to Evaluating Teacher Value-Added Scores

Value-added models have become popular in research and pay-for-performance plans. While scholars have focused attention on some aspects of their validity (e.g., scoring procedures), others have received less scrutiny. This article focuses on the extent to which value-added scores correspond to other indicators of teacher and teaching quality. The authors compared 24 middle school mathematics teachers’ value-added scores, derived from a large (N = 222) district data set, to survey- and observation-based indicators of teacher quality, instruction, and student characteristics. This analysis found teachers’ value-added scores correlated not only with their mathematical knowledge and quality of instruction but also with the population of students they teach. Case studies illustrate problems that might arise in using value-added scores in pay-for-performance plans.

Policy Is Not Enough: Language and the Interpretation of State Standards

Words have no inherent meaning. Instead, they signify ideas or actions ascribed to them by communities, and meanings for specific words often vary across those communities. Words that carry specialized meanings in one community can be interpreted differently by another, particularly where individuals in the second community have little access to dialogues in the first, or when forces in the second community compete to assign meaning to key words. Observations of one district’s mathematics curriculum writing committee suggested a disconnect of this sort occurred along the policymaker/teacher divide in one state. Where state standards used words like “construct” and “concept” to imply certain mathematics teaching methods, teachers reading these documents imputed more local, and sometimes conventional, definitions to these words. As a result, state standards lost their force. This article describes and analyzes the work of state-local policy reconciliation as it occurred in this committee, and appraises that work’s implications for reform efforts that rely on language as a medium for communication.

When Rater Reliability Is Not Enough Teacher Observation Systems and a Case for the Generalizability Study

In recent years, interest has grown in using classroom observation as a means to several ends, including teacher development, teacher evaluation, and impact evaluation of classroom-based interventions. Although education practitioners and researchers have developed numerous observational instruments for these purposes, many developers fail to specify important criteria regarding instrument use. In this article, the authors argue that for classroom observation to succeed in its aims, improved observational systems must be developed. These systems should include not only observational instruments but also scoring designs capable of producing reliable and cost-efficient scores and processes for rater recruitment, training, and certification. To illustrate how such a system might be developed and improved, the authors provide an empirical example that applies generalizability theory to data from a mathematics observational instrument.

Mathematical Knowledge of Middle School Teachers: Implications for the No Child Left Behind Policy Initiative

This article explores middle school teachers’ mathematical knowledge for teaching and the relationship between such knowledge and teachers’subject matter preparation, certification type, teaching experience, and their students’ poverty status. The author administered multiple-choice measures to a nationally representative sample of teachers and found that those with more mathematical course work, a subject-specific certification, and high school teaching experience tended to possess higher levels of teaching-specific mathematical knowledge. However, teachers with strong mathematical knowledge for teaching are, like those with full credentials and preparation, distributed unequally across the population of U.S. students. Specifically, more affluent students are more likely to encounter more knowledgeable teachers. The author discusses the implications of this for current U.S. policies aimed at improving teacher quality.

Professional Development Research Consensus, Crossroads, and Challenges

Commentaries regarding appropriate methods for researching professional development have been a frequent topic in recent issues of Educational Researcher as well as other venues. In this article, the authors extend this discussion by observing that randomized trials of specific professional development programs have not enhanced our knowledge of effective program characteristics, leaving practitioners without guidance with regard to best practices. In response, the authors propose that scholars should execute more rigorous comparisons of professional development designs at the initial stages of program development and use information derived from these studies to build a professional knowledge base. The authors illustrate with examples of both a proposed study and reviews of evidence on key questions in the literature.

Professional Development Standards and Practices in Elementary School Mathematics.

In this article I argue that the literature describing effective professional development is missing at least 2 key pieces of information-typical professional development that reaches typical teachers and how effective recent standards are in assisting district or school personnel to design and/or choose high-quality staff development experiences. To fill this gap, I used commonly cited professional development standards to analyze observations of 13 professional development sessions in mathematics in an eastern state, seeking to understand what teachers might have learned and how that content related to the standards. These observations were made over 6 months and included professional development provided by teachers, district officials, consultants, and university staff. I found that although much of the observed professional development met published standards, it often treated mathematics and student learning superficially. I suggest reasons for the failure of literature-based standards to discriminate high- from low-quality programs and describe alternatives to such standards in assisting schools and teachers to use professional development resources more effectively.

Validating the Ecological Assumption: The Relationship of Measure Scores to Classroom Teaching and Student Learning

In validating any assessment, a critical question is its relationship to the domain of practice: whether success on the assessment, which is constrained to be either a small sample of the knowledge or even simply an indicator of knowledge needed for a practice, actually predicts proficient performance. Following Kane (2001, 2004), we formulated an assumption and several observable inferences regarding the relationship of our measures of teachers’ mathematical knowledge for teaching to their actual classroom mathematical work. This assumption is the third detailed in Schilling and Hill (this issue):