Drew H. Gitomer

An Argument Approach to Observation Protocol Validity.

This article develops a validity argument approach for use on observation protocols currently used to assess teacher quality for high-stakes personnel and professional development decisions. After defining the teaching quality domain, we articulate an interpretive argument for observation protocols. To illustrate the types of evidence that might compose a validity argument, we draw on data from a validity study of the Classroom Assessment Scoring System for secondary classrooms. Based on data from 82 Algebra classrooms, we illustrate how data from observation scores, value-added models, generalizability studies, and measures of teacher knowledge, student achievement, and teacher and student beliefs could be used to build a validity argument for observation protocols. Strengths and limitations of the validity argument approach as well as the issues the approach raises for observation protocol validity research are considered.

Effect of Observation Mode on Measures of Secondary Mathematics Teaching.

Classroom observation of teachers is a significant part of educational measurement; measurements of teacher practice are being used in teacher evaluation systems across the country. This research investigated whether observations made live in the classroom and from video recording of the same lessons yielded similar inferences about teaching. Using scores on the Classroom Assessment Scoring System–Secondary (CLASS-S) from 82 algebra classrooms, we explored the effect of observation mode on inferences about the level or ranking of teaching in a single lesson or in a classroom for a year. We estimated the correlation between scores from the two observation modes and tested for mode differences in the distribution of scores, the sources of variance in scores, and the reliability of scores using generalizability and decision studies for the latter comparisons. Inferences about teaching in a classroom for a year were relatively insensitive to observation mode. However, time trends in the raters’ use of the score scale were significant for two CLASS-S domains, leading to mode differences in the reliability and inferences drawn from individual lessons. Implications for different modes of classroom observation with the CLASS-S are discussed.

Teaching Through Interactions in Secondary School Classrooms: Revisiting the Factor Structure and Practical Application of the Classroom Assessment Scoring System- Secondary

Valid measurement of how students’ experiences in secondary school classrooms lead to gains in learning requires a developmental approach to conceptualizing classroom processes. This article presents a potentially useful theoretical model, the Teaching Through Interactions framework, which posits teacher-student interactions as a central driver for student learning and that teacher-student interactions can be organized into three major domains. Results from 1,482 classrooms provide evidence for distinct emotional, organizational, and instructional domains of teacher-student interaction. It also appears that a three-factor structure is a better fit to observational data than alternative one- and two-domain models of teacher-student classroom interactions, and that the three-domain structure is generalizable from 6th through 12th grade. Implications for practitioners, stakeholders, and researchers are discussed.

Handbook of Research on Teaching

The fifth edition of the Handbook of Research on Teaching is an essential resource for students and scholars dedicated to the study of teaching and learning. It offers a vast array of topics ranging from the history of teaching to technological and literacy issues.In each authoritative chapter, the authors summarize the state of the field while providing conceptual overviews of critical topics related to research on teaching. Each of the volumes 23 chapters is a canonical piece that will serve as a reference tool for the field. The chapters, all broad treatments of areas of study, will help readers see how particular areas of research connect with the larger issues of teaching and teacher education.The handbook provides readers with an unaparalleled view of the current state of research on teaching across its multiple facets and related fields.

Measurement issues and assessment for teaching quality

Introduction SECTION I: MEASURING TEACHING QUALITY FOR PROFESSIONAL ENTRY Ch 1. Measuring Teacher Quality for Professional Entry Ch 2. Hiring for Teacher Quality at the District Level: Lessons from The New Teacher Project Ch 3. Professionalizing the Occupation of Teaching in a Time of Transition Broadening the Vision of Professional Entry--Synthesis of Section I SECTION II. MEASURING TEACHING QUALITY IN PRACTICE Ch 4. Measuring Teacher Quality in Practice Ch 5. The Policy Uses and Policy Validity of Value-Added and Other Teacher Quality Measures Ch 6. Approximations of Teacher Quality and Effectiveness: View from the State Education Agency Measuring Teacher and Teaching Quality: Considerations and Next Steps--Synthesis of Section II SECTION III. MEASURING TEACHING QUALITY IN CONTEXT Ch 7. Mapping the Terrain of Teacher Quality Ch 8. Measuring Instruction for Teacher Learning Ch 9. Opportunity to Teach: Teacher Quality in Context Crisp Measurement and Messy Context: A Clash of Assumptions and Metaphors--Synthesis of Section III Assessment of Teaching or Assessment for Teaching? Reflections on the Invitational Conference About the Contributors Author Index Subject Index

Road Maps for Learning and Teacher Evaluation

Black, Wilson, and Yao (this issue) make a compelling case for a coherent model of curriculum, pedagogy, and assessment, all built around a well-specified theory of learning. Arguing that dominant assessment policies and practices tend to drive pedagogy and student attitudes toward learning in pernicious ways, Black et al. make an argument that is as much about policy, values, and use as it is about assessment design. They propose an assessment system that is first and foremost about helping teachers understand how well students are learning with respect to a clear set of desired learning outcomes so that teachers can take appropriate steps to help students progress. They also note that for teachers to use learning progressions and assessments well, substantial professional development will likely be needed. In contrasting the endorsed model with existing practices, Black et al. warn that a focus on scores and competition leads students to ego-involvement rather than task-involvement attitudes, with potentially adverse impact on test scores and approaches to learning. Under the proposed model, the goal is for teachers and students to engage in assessments and discourse and reasoning around those assessments that will focus attention on progress toward a very clear set of ambitious learning objectives. Successful implementation of the ideas and practices recommended in the article implies instructional and assessment practices that are observed infrequently in U.S. classrooms. Thus, the new set of common core standards (National Governors Association and Council of Chief State School Officers, 2010) that have been developed and endorsed by numerous states can be viewed as consistent with many of the ideas supported by Black et al. At the core of these standards are sets of learning progressions intended to support more coherent curricular and assessment practices across grades. However, it is important to note that though the progressions are described at a fairly general level, there is no commitment to particular curricula or pedagogical practice. Thus, the core standards make an attempt to define theories of learning but do not provide specifics about the three legs of the learning triangle: curriculum, pedagogy, and assessment. This focus on standards and assessments for students occurs at the same moment that there is an unprecedented focus on evaluating the relative quality of teachers to make decisions about retention, tenure, and compensation. In virtually all systems, the measure that has the most significant influence on a teacher’s evaluation is based on changes in student test scores that are then attributed to a specific teacher. In fact, the strongest impetus for these evaluation systems comes

Classroom assignments as measures of teaching quality

We investigate classroom assignments and resulting student work to identify important characteristics of assignments in terms of instructional quality and their validity as measures of teaching quality. We examine assignment quality within a large-scale project exploring multiple measures including classroom observations, teacher knowledge measures, and value-added estimates based on student achievement scores. Analyses included descriptive statistics, multivariate analyses to understand factors contributing to score variance, and correlational analyses exploring the relationship of assignment scores to other measures. Results indicate relatively low demand levels in all teacher assignments, a marked difference in score distributions for mathematics (math) and English language arts (ELA), and a substantial relationship between what was asked of and produced by students. Relationships between assignments scores, classroom characteristics, and other measures of teaching quality are examined for both domains. These findings help us understand the nature of and factors associated with assignment quality in terms of intellectual demand.

Modifications and accommodations: a preliminary investigation into changes in classroom artifact quality

ABSTRACT Teachers are ultimately responsible for implementing school policy changes, including those driven by current United States (US) mandates that emphasize rigorous career and college readiness standards requiring high expectations for all students. In order to create multiple entryways to instruction for students with disabilities, teachers are faced with an amalgam of possible accommodations and modifications, while simultaneously being expected to assure high standards. This study seeks to provide insight into how teachers in the US are adapting instruction for students who have federally legislated supports and services including modifications or accommodations and are instructed within general education classrooms. Thirty-five unique classroom artifacts (i.e. assignments and assessments) were analyzed for the type of accommodation/modification and alignment to the Next Generation Science Standards, a movement to increase the quality of science instruction. Results indicate that this group of students was assigned tasks that were substantially different from their peers, were afforded fewer opportunities to access challenging, rigorous work, and that teachers may require further support to differentiate instruction. This preliminary investigation within a specific US context raises questions about equitable access to quality instruction that merit further exploration.

An Analysis of the Meaning and Use of Student Learning Objectives

In new state teacher evaluation systems, Student Learning Objectives (SLOs) have emerged to satisfy requirements that teachers in non-tested subjects be evaluated by student growth measures (SGMs). Many of these systems conceive of SLOs as satisfying two distinct purposes: supporting instructional improvement and evaluating personnel. Based on data from a broader analysis of Race to the Top (RttT) state implementations, this chapter reviews initial design efforts, describes SLO implementations, and considers the roles SLOs should play as part of teacher evaluation systems. We argue that SLOs must be re-conceptualized as a measure of teacher practice, the tremendous variation in measure design needs to be closely scrutinized, and state administrators need to be conscious of the purpose for which they are using SLOs so that they may appropriately design valid teacher evaluation systems.