Audrey Amrein-Beardsley

Evaluating Teacher Evaluation

Popular modes of evaluating teachers are fraught with inaccuracies and inconsistencies, but the field has identified better approaches.

Methodological Concerns About the Education Value-Added Assessment System:

Value-added models help to evaluate the knowledge that school districts, schools, and teachers add to student learning as students progress through school. In this article, the well-known Education Value-Added Assessment System (EVAAS) is examined. The author presents a practical investigation of the methodological issues associated with the model. Specifically, she argues that, although EVAAS is probably the most sophisticated value-added model, it has flaws that must be addressed before widespread adoption. She explores in depth the shortage of external reviews and validity studies of the model, its insufficient user-friendliness, and methodological issues about missing data, regression to the mean, and student background variables. She also examines a paradigm case in which the model was used to advance unfounded assertions.

Examining the Development of a Hybrid Degree Program: Using Student and Instructor Data to Inform Decision-Making.

Abstract This paper investigates the questions and considerations that should be discussed by administrators, faculty, and support staff when designing, developing and offering a hybrid (part online, part face-to-face) degree program. Using two Web questionnaires, data were gathered from nine instructors and approximately 450 students to evaluate student and instructor perceptions and opinions of hybrid instruction and activities. In comparison to prior research, the results of this study offer larger and more significant policy and programmatic implications for degrees based on the hybrid format, including instructional technology training and support for students and instructors, creation of common class procedures and expectations, and development of consistent schedules that maximize benefit and flexibility for students and instructors.

The Random Assignment of Students Into Elementary Classrooms Implications for Value-Added Analyses and Interpretations

Value-added models (VAMs) are used to measure changes in student achievement on large-scaled standardized test scores from year to year. When aggregated, VAM estimates are used to measure teacher effectiveness and hold teachers accountable for the value they purportedly add to or detract from student learning and achievement. In this study, researchers examined the extent to which purposeful (nonrandom) and random assignment of students into classrooms occurs in Arizona elementary schools (Grades 3–6). Researchers found that overwhelmingly, students are not randomly assigned and administrators, teachers, and parents play a prodigious role in the process. Findings have current implications for value-added analyses and the extent to which nonrandom assignment practices might impact or bias teachers’ value-added scores.

Recruiting Expert Teachers into High-Needs Schools: Leadership, Money, and Colleagues

espanolLos expertos en calidad docente estan instando a los responsables politicos para identificar y seleccionar profesores especializados (experts) para atraerlos a ensenar en escuelas con necesidades educativas. Para entender lo que esto podria significar, profesores expertos en Arizona fueron entrevistados para conocer sus preferencias relacionadas con el trabajo con el fin de informar politicas de contratacion en el futuro. Los resultados proporcionan una razon para considerar los factores que las autoridades en el ambito de la politica educativa deberian tener en cuenta cuando se piensa en la contratacion de profesores especializados en escuelas con necesidades educativas. Estos factores incluyen la calidad de los gestores, los sueldos, bonificaciones y beneficios y en que medida estos profesores podrian trabajar en diferentes roles para mejorar el aprendizaje de los estudiantes. EnglishTeacher quality experts are urging policymakers to identify and target expert teachers and entice them to teach in high-needs schools. To understand what this might take, expert teachers in Arizona were surveyed to understand their job-related preferences to inform future recruitment policies. Findings should inform others about the important factors educational policymakers might consider when thinking about recruiting expert teachers into high-needs schools. These factors include the quality of the principal; salary, bonuses, and benefits; and the degree to which expert teachers can work in multiple roles to enhance student learning. portuguesOs especialistas em qualidade docente estao incitando os formuladores de politicas para identificar e selecionar docentes especializados (experts) para atrai-los para ensinar em escolas com altas necessidades educacionais. Para entender o que isso poderia significar, especialistas do corpo docente no Arizona foram entrevistados para saber suas preferencias relacionadas com o trabalho nos sentido de informar as politicas de recrutamento futuras. Os resultados fornecem razoes para considerar os fatores que autoridades na area da politica educacional deveriam considerar ao pensar sobre a contratacao de professores especializados nas escolas com altas necessidades educacionais. Esses fatores incluem a qualidade dos gestores, salarios, bonus e beneficios e em que medida estes professores poderiam trabalhar em diferente papeis para melhorar a aprendizagem dos alunos.

Graduates Respond to an Innovative Educational Doctorate Program

Dispute about professional doctoral programs in education has increased, with much hinging on the design of educational doctorate (EdD) programs. In this study, the first cohort of graduates responded to a new, innovative EdD program designed to develop them as leaders, scholars, and practitioners. Twenty graduates completed an exit survey, and a content analysis of their dissertations was performed. Both were done to improve the program. Graduates valued curricular and instructional features, engaged in actions to improve their local settings, endorsed a sense of community that developed, and reported changes in their professional identities. Their dissertation work validated these ideas.

Value-Added Models for Teacher Evaluation and Accountability: Commonsense Assumptions

Despite the concerns regarding value-added models (VAMs), advocates hold strong to VAMs’ theoretical strengths and potentials, while adopting a set of agreed-upon albeit “heroic” set of assumptions, without research in support. These assumptions transcend promotional, policy, media, and research-based pieces, but they have never been made explicit as analyzed as a set or whole. The purpose of this study was to make unambiguous the assumptions that have been made within the VAM narrative that have often been accepted without challenge, and situate these assumptions within the research. Sources for analyses included 470 pieces, published within traditional and nontraditional outlets, from which we derived 27 prevailing assumptions.

Physical Education Teacher Perceptions of Teacher Evaluation.

Purpose: The purpose of this study was to develop a better understanding of current PE teachers’ perceptions of teacher evaluation systems. Method: A mixed methods approach was used and two sources of data collection were used: (a) a short survey of PE teachers (n = 22) in one urban school district and (b) a formal semistructured interview with 10 PE teachers from the same school district. Results: Statements from the survey showed low level of agreement in the following areas: (a) PE was a priority, (b) evidence of student growth and achievement were important factors of teacher evaluation in PE, and (c) PE teachers were confident in evaluation system and evaluators ability to determine effectiveness of PE teacher. Three common themes were revealed from the interviews: (a) PE is valued, but not prioritized; (b) teacher evaluation in PE is greatly needed, yet not transparent; and (c) physical educators are not confident in their evaluators. Conclusion: Results are consistent with current literature suggesting that (a) PE remains a marginalized subject with low priority; (b) teacher evaluation systems are not tailored toward noncore subjects; (c) and PE teachers, like classroom teachers, are not confident that their evaluators can give a fair and valid assessment.

Value-Added Models (VAMs): Caveat Emptor

ABSTRACT Value-added models (VAMs) are being used in education to link the contribution of individual teachers and schools to students’ learning. The use of VAMs has been surrounded by controversy and high-profile public debates. On April 8, 2014 the American Statistical Association (ASA) released a statement on VAMs related to their use in education practice. In this article, we lay out the discussion of the main points raised in the ASA statement within the large amount of scholarly literature published over the past decade in statistical, education, and economics journals. We identify the issues that are critical for the understanding of the VAMs’ strengths and weaknesses, and related consequences of their use for high-stakes decision-making. We conclude that the cautionary points raised in the ASA statement are supported by the findings in the existing research that, with a few exceptions, challenges the assumptions underlying the use of VAMs and demonstrates the issues that should be taken into consideration when using VAMs for consequential decisions.

Value-added models What the experts say

Being an expert involves explaining how things are supposed to work, and, perhaps more important, why things might not work as supposed. In this study, researchers surveyed scholars with expertise in value-added models (VAMs) to solicit their opinions about the uses and potential of VAMs for teacher-level accountability purposes (for example, in the areas of teacher evaluation, promotion, tenure, merit pay, and termination). Researchers found that respondents were most concerned about VAMs and their methodological and pragmatic flaws and limitations; their uses, misuses, and sometimes abuses in practice; and their unintended consequences. Education practitioners will benefit from this shared knowledge, intelligence, wisdom, and expertise to develop a deeper understanding of the merits and pitfalls of value-added models.