Melanie Catharina Margaretha Ehren

THE RELATIONSHIPS BETWEEN SCHOOL INSPECTIONS, SCHOOL CHARACTERISTICS AND SCHOOL IMPROVEMENT

ABSTRACT:  The effects of school inspections on school improvement have been investigated only to a limited degree. The investigation reported on in this article is meant to expand our knowledge base regarding the impact of school inspections on school improvement. The theoretical framework for this research is partly based on the policy theory behind the Dutch Educational School Supervision Act (the latter includes assumptions about how school inspections lead to school improvement). Interviews and a survey with school inspectors gave insight into how school inspectors implement the Supervision Act and how they assess schools, and stimulate schools to improve. The results of ten case studies showed that all schools started to improve after a school visit. The innovation capacity of the school and the school environment do not seem to contribute to school improvement after school inspections. No effects were found on school-improvement processes of the number of insufficient scores that schools received from inspectors, the extent of feedback and suggestions for improvement, and the number of agreements. The provision of feedback about weaknesses, the assessment of these weak points as unsatisfactory, and the agreements between an inspector and the school regarding improvement activities do appear to make a difference in promoting school improvement.

Towards a theory on the impact of school inspections

ABSTRACT:  This article describes a theory about the ambition of most Inspectorates to realise ‘school improvement through inspection’. Literature about a number of direct and indirect interventions, such as reciprocity, communication and feedback is used to build a theoretical model stating the relations between working methods of school inspectors, reactions of schools and resulting effects and side effects. Finally two types of inspections strategies are described that can be used in different types of schools. We expect schools with a low innovation capacity and few external impulses to be helped best by a directive approach in which an inspector clearly points to the strong and weak points of the school, the probable causes of their level of functioning, and potential ways for improvement. The inspector should pressure the school to change by making written agreements on how to change and by asking the school to work out these agreements in an improvement plan. A school with a high innovation capacity and strong external impulses is expected to do better with a more reserved inspection approach. Inspectors only need to provide this school with some insight into their strong and weak points.

On the Impact of the Dutch Educational Supervision Act Analyzing Assumptions Concerning the Inspection of Primary Education

This article uses a policy scientific approach to reconstruct assumptions underlying the Dutch Educational Supervision Act. We show an example of how to reconstruct and evaluate a program theory that is based on legislation of inspection. The assumptions explain how inspection leads to school improvement. Evaluation of these assumptions is used to predict the (in)effectiveness of this legislation. The article concludes by discussing the advantages and drawbacks of this kind of approach as a starting point for impact and effect studies. As the program theory of inspection includes elements common to other kinds of educational interventions and reforms, these elements can also be considered for other types of program theory.

Editorial article for the special issue on data-based decision making around the world: from policy to practice to results

This editorial article briefly examines the importance of data-based decision making. It discusses the definition as well as rationale for data-based decision making, its purposes, the use of data at different levels of the educational system, and possible promoting and hindering factors of effective data use. It also examines the effects of data use, intended effects (e.g., increased student achievement), as well as unintended consequences (e.g., cheating with tests). We end with suggestions for new research priorities.

Strategic data use of schools in accountability systems

School inspections are expected to have an impact on data use and improvement of schools. Schools are expected to generate data (e.g., self-evaluation reports and student achievement results) as part of the inspection process. This process, in turn, also generates data (e.g., inspection reports) for school improvement. The high-stakes context in which both types of data are generated however has been known to lead to strategic responses of schools. In this study, we analyzed if schools cheat on tests and reshape their test pool in responses to the Dutch (risk-based) school inspections. We found that 5.5% of the schools do not to comply with the guidelines for administering the test; one third of the schools exclude one or more students from the test. These responses, however, do not appear to be related to specific measures in the Dutch school inspections or prior performance of schools on these measures.

Setting expectations for good education: how Dutch school inspections drive improvement

With decentralisation becoming increasingly widespread across Europe, evaluation and accountability are becoming key issues in ensuring quality provision for all (Altrichter & Maag Merki, 2010; Eurydice, 2004). In Europe, the dominant arrangement for educational accountability is school inspections. The purpose of this research is to identify and analyse the ways in which school inspections in The Netherlands impact on the work of schools. The results of 2 years of survey data of principals and teachers in primary and secondary schools show that inspection primarily drives change indirectly, through encouraging certain developmental processes, rather than through more direct and coercive methods, such as schools reacting to inspection feedback. Specifically, results indicate that school inspections which set clear expectations on what constitutes “good education” for schools and their stakeholders are strong determinants of improvement actions; principals and schools feel pressure to respond to these prompts and improve their education.

From "intuition"- to "data"-based decision making in Dutch secondary schools?

Schools these days are confronted with a lot of data, which they have to transform into information to be used for school improvement. However, research shows that most teachers do not use data properly, or do not use data at all. In the Netherlands, a data team intervention was developed and piloted to support schools in the use of data.

Toward a model of school inspections in a polycentric system.

Many education systems are developing towards more lateral structures where schools collaborate in networks to improve and provide (inclusive) education. These structures call for bottom-up models of network evaluation and accountability instead of the current hierarchical arrangements where single schools are evaluated by a central agency. This paper builds on available research about network effectiveness to present evolving models of network evaluation. Network effectiveness can be defined as the achievement of positive network level outcomes that cannot be attained by individual organizational participants acting alone. Models of network evaluation need to take into account the relations between network members, the structure of the network, its processes and its internal mechanism to enforce norms in order to understand the achievement and outcomes of the network and how these may evolve over time. A range of suitable evaluation models are presented in this paper, as well as a tentative school inspection framework which is inspired by these models. The final section will present examples from Inspectorates of Education in Northern Ireland and Scotland who have developed newer inspection models to evaluate the effectiveness of a range of different networks.

Validation of Inspection Frameworks and Methods

This chapter explores the issues of reliability and validity of inspection frameworks and methods, and challenges and tensions in inspection frameworks and methods. Validity is an important aspect of thinking about effective inspection system as invalid inspection systems may lead to flawed judgments which will misguide administrative interventions and policy decisions and which are likely to have a negative impact on schools and teachers. We will introduce Kane’s (J Educ Meas 50(1):1–73, 2013) notion of argument-based approaches to evaluate the validity of inspection frameworks and provide two examples of how such an argument can be constructed. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999; American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 2014) will then be used to describe five types of evidence to evaluate the validity of these arguments. For each of these sources of evidence we present examples of available studies.

Changing School Board Governance in Primary Education through School Inspections.

This paper addresses if, and to what extent, the current working methods of the Dutch Inspectorate of Education affect the governance of school boards in schools for primary education. A key facet of the working method is the inspection meeting with the school board. Drawing upon a large quantitative study (n = 244) we are able to identify some changes in school board governance due to these inspection meetings. School boards that had an inspection meeting indicate changes in their governance of quality assurance and data use, and in the amount of data that they collect on the functioning of their schools. School boards indicate very small amounts of activities with regard to the curriculum and instruction in their schools.