Liz Hollingworth

A Bridge Too Far? Challenges in Evaluating Principal Effectiveness

Purpose: The purpose of this article is to examine the assumptions underlying efforts to evaluate principal effectiveness in terms of student test scores, to review extant research on efforts to estimate principal effectiveness, and to discuss the appropriateness of including estimates of principal effectiveness in evaluations of principals. Method: We review 10 different strategies for estimating principal effectiveness based on student test scores, representing all of the strategies currently employed by states and districts. We base our reviews on the literature in three areas: use of test scores, evaluation, and statistical approaches to estimating the effects of individuals and schools on student test scores. Conclusions: We conclude there are currently no strategies to estimate principal effectiveness that accurately capture the independent effect of principals on student test scores; thus, these current strategies send inaccurate signals to both principals and those who make employment decisions about principals. Moreover, we also conclude a substantial proportion of principals could not be included in the most accurate strategies to assess principal effectiveness. Implications: This research has profound implications for states and districts implementing principal evaluation systems, particularly those making high-stakes decisions about principals based on statistical estimates of principal effectiveness. Indeed, such statistical estimates should be used not for making judgments or decisions about principals but rather as a screening tool to identify where states and districts should focus more in-depth and accurate strategies to evaluate principal effectiveness.

Five Ways to Prepare for Standardized Tests Without Sacrificing Best Practice

What can teachers do in their classrooms to help students prepare for a high-stakes, standardized, multiple-choice reading test without sacrificing what they know to be best practice? Decreasing class sizes or other solutions that require substantial changes in school finance are not usually realistic options. This article outlines five suggestions for practicing teachers to improve the reading test scores of their students without making drastic changes to how they already teach.

Why Leadership Matters: Empowering Teachers to Implement Formative Assessment.

Purpose – The focus of this research is to understand the role of the district superintendent, the building principal, and the school leadership team of classroom teachers as catalysts for innovation in instruction and classroom assessment. School characteristics and structures designed to specifically support professional learning communities are to be analyzed, and the responsibility teachers take for student learning using formative assessment is evaluated.Design/methodology/approach – The setting for the study is a small, Midwestern high school in the USA. The high school has a staff of 38 teachers, 15 of whom are on the Formative Assessment Building Leadership Team. The high school principal and district superintendent also participated in the study. The study took the form of a qualitative case study. The data sources were transcripts and artifacts from eight months of interviews, and building‐wide teacher meetings. Professional development inservices were also analyzed.Findings – The principal serv...

Evaluating Principal-Preparation Programs Based on Placement Rates: Problems and Prospects for Policymakers

Recent calls to hold preparation programs accountable for outcomes have led states to develop and adopt preparation program accountability systems. A primary feature of these systems is a focus on outcomes such as placement rates, retention rates, and graduates’ effectiveness in improving K-12 student achievement. Yet, little research has examined the feasibility of employing such outcome measures, let alone the validity and reliability of the inherent judgments. This conceptual study reviews and makes conclusions about the appropriateness of using placement rates to evaluate principal-preparation programs based on theoretical analyses and empirical analysis of statewide placement data from Texas.

Annotations on the Use of the Mexican Norms for the WAIS-III

This article provides crucial information to judge the appropriateness of the Mexican version of the Wechsler Adult Intelligence Scale-Third Edition and recognizes some limitations in both the process of its adaptation to the Mexican population and the norm development process. This is an effort to contribute to the debate initiated by Suen and Greenspan (2008), who argued in court against the use of Mexican norms in a death penalty case, which depended upon establishing the diagnosis of mental retardation. As a part of the defense team, these scholars argued a number of points against the use of the Mexican norms. With input from the lead researcher on the Mexican standardization process, some of the criticisms are addressed, and further information about the norm development process for this test in Mexico is provided in an attempt to be critical about the strengths and weaknesses of the use of existing Mexican norms. Finally, we argue that results from a single test must not be used to make life and death decisions and that test development is a continuous process influenced by culture, language, and indeed by norm-developing procedures and debates.

The Impact of Personal and Program Characteristics on the Placement of School Leadership Preparation Program Graduates in School Leader Positions

Purpose: The purpose of this study is to examine the impact of personal and program characteristics on the placement of graduates of principal preparation programs in assistant principal, principal, and school leadership positions. Research Design: This study relies on Texas principal production data from 1993 through 2007 matched to employment data from 1993 through 2013. The data include personal characteristics of each program’s graduates (age, sex, and race/ethnicity), program characteristics (program type, percentages of female graduates, and percentage of White graduates), labor market location, and certification year. We employ both descriptive statistics and multilevel logistic regression analysis to examine the factors associated with obtaining employment as an assistant principal, principal, and school leader. Findings: At least 50% of graduates obtained placement as a school leader within 5 years and about 70% did so over 10 years. Within 5 years of certification, men, Latinos, and middle-aged graduates had greater odds of employment as a school leader than women, Whites, and younger and older graduates, respectively. Differences arose, however, when examining placement as an assistant principal and principal separately. In particular, Black and Latina/o graduates had greater odds of employment as an assistant principal but had lower odds of employment as a principal than their White peers. Finally, there were few program characteristics associated with placement, and differences between programs explained very little of the variation in placement rates, bringing into question efforts to hold programs accountable for such outcomes.

Evaluating State Principal Evaluation Plans Across the United States

Recent federal legislation has created strong incentives for states to adopt principal evaluation systems, many of which include new measures of principal effectiveness such as estimates of student growth and changes in school climate. Yet, there has been little research on principal evaluation systems and no state-by-state analysis of the principal evaluation systems adopted at the behest of the legislation. This study uses survey data and document review to assess the components of principal evaluation systems in the 50 states and Washington, D.C. Finally, based on recent research, this study critiques the various components of these new evaluation systems.

Annotations on Mexico's WISC-IV: A Validity Study

This project seeks to provide evidence on the internal structure of the Escala Wechsler de Inteligencia para Niños-IV (EWIN-IV; Wechsler, 2007a) through a confirmatory factor analysis and intercorrelational study. Also provided is information on the adaptation process and other sources of validity evidence in support of the EWIN-IV norms. The standardization data for the EWIN-IV were used for all analyses. The factor loadings and correlational patterns found on the EWIN-IV are comparable to those seen in the American versions of the test. The proposed factor and scoring structure of the EWIN-IV was supported.

Pizza Parties, Pep Rallies, and Practice Tests: Strategies Used by High School Principals to Raise Percent Proficient

This study explores ways high school principals are responding to the demands of education reform to raise student test scores on achievement tests used for accountability purposes. Anecdotal evidence suggests administrators have instituted pizza parties and pep rallies to motivate students to do their best and practice tests to prepare students academically. We interviewed principals to find out how they are both motivating and preparing their students for high-stakes tests. The goal of this research is to investigate how widespread these and other activities are and to see if the principals believe these strategies are indeed raising scores.

Performance Assessment for Principal Licensure: Evidence From Content and Face Validation and Bias Review:

This article presents the validity bias review feedback and outcomes of new performance-based assessments to evaluate candidates seeking principal licensure. Until now, there has been little empirical work on performance assessment for principal licensure. One state recently developed a multi-task performance assessment for leaders and has implemented it for statewide use in initial principal licensure decisions; this development process is described here, focusing on content validity and bias review, and incorporates candidate and program faculty validiation as well. The results demonstrate the content validity, relevance, and feasibility of this new performance assessment for leaders, and yield implications for leader assessment generally.