Peter Dahler-Larsen

Fabricating Quality in Education : Data and Governance in Europe

This book argues that data and their use constitute a form of governance of education. It highlights the ways in which education is steered and managed so that a European education policy space is ...

Constitutive Effects of Performance Indicators: Getting beyond unintended consequences

The idea that performance indicators in public management have unintended consequences is almost as old as performance measurement itself. But, is ‘unintended consequences’ an appropriate and insightful idea? The very term rests on an identification of intentions and assumptions about validity that are demonstrably problematic. Based on a distinction between trivial and advanced measure fixation, an argument is made for constitutive effects that are based on less problematic assumptions. Through this conceptual move, the political dimension of performance indicators is appreciated. The conceptual dimensions of constitutive effects are carved out, empirical illustrations of their applicability are offered and implications discussed.

Constitutive effects as a social accomplishment: A qualitative study of the political in testing

Although it is often recognised that testing has unintended effects, not enough intellectual curiosity has been invested in a deeper questioning of what “unintended” means. While this concept often remains dependent on an image of good, underlying, rational intentions, it underestimates the contested, controversial and socially productive nature of testing. A conceptual alternative is offered, termed “constitutive effects” as well as a theoretical framework in which this concept makes sense. Four domains in which constitutive effects are likely to unfold are carved out. A qualitative case study of how a national testing system of Danish for immigrants influences practices in one school in Denmark is offered as an illustration of such constitutive effects. Finally, a discussion unfolds concerning in what sense constitutive effects of testing can be seen as political (in a way that is not captured by the more conventional conceptualisation of “unintended effects”.

Constitutive Effects of Performance Indicator Systems

Evaluation in general and performance indicator systems in particular play an increasing role in society. We do not have a long historical set of experiences which helps us understand what exactly happens when, say, performance data for schools are made public on the internet and in news, because the emerging rules of the game in what some observers call the “knowledge society” (Stehr, 1994, 2001) and “reflexive modernization” (Beck, 1997a, 1997b) have inaugurated new relations between evaluation and performance data on the one hand and political, organizational and practical realities on the other.

Quality assurance and evaluation in Denmark

In recent years, international comparisons have become a powerful lever for change in educational policies. Quality assurance and evaluation (QAE) in many forms is being incorporated in the steering and management of schools. However, the Danish use of QAE is a dynamic result of ongoing tensions and struggles both around and within QAE initiatives. Romantic and communitarian tenets in the traditional Danish view of enlightenment continue to influence the way QAE is used. However, the recent waves of QAE initiatives – comprising ‘transparency’, goal‐setting, tests and a number of new comprehensive and mandatory reporting mechanisms – restructure the world of schools and teachers in important ways.

When Evaluation Meets the ‘Rough Ground’ in Communities

Based on experiences from two evaluation projects, the two partners in the following dialogue discuss the meaning of ‘resistance’ to evaluation, or more specifically, resistance to evaluation touching upon values that are central in some communities. Step by step, the dialogue identifies a tension between evaluation as a modernist undertaking and the traditional values of these communities. Through the dialogue about this tension, the ‘rough ground’ in evaluation is revealed, then the implications for evaluation practice are discussed.

Evaluation, Language, and Untranslatables:

The issue of translatability is pressing in international evaluation, in global transfer of evaluative instruments, in comparative performance management, and in culturally responsive evaluation. Terms that are never fully understood, digested, or accepted may continue to influence issues, problems, and social interactions in and around and after evaluations. Their meanings can be imposed or reinvented. Untranslatable terms are not just “lost in translation” but may produce overflows that do not go away. The purpose of this article is to increase attention to the issue of translatability in evaluation by means of specific exemplars. We provide a short dictionary of such exemplars delivered by evaluators, consultants, and teachers who work across a variety of contexts. We conclude with a few recommendations: highlight frictions in translatability by deliberately circulating and discussing words of relevance that appear to be “foreign”; increase the language skills of evaluators; and make research on frictions in translation an articulate part of the agenda for research on evaluation.

On the Dynamic Nature of Performance Management Regimes

The ambition of this article is to gain a better understanding of the endogenous dynamics of performance management regimes. Based on a review of the literature, we develop a framework that enables us to grasp dimensions and mechanisms of escalation. Hereafter, we demonstrate the use of our framework through two cases. We argue that a range of dimensions of a performance regime may evolve over time, that the dynamics might be repressed by contextual factors and politics, that the dynamics may be reversed leading to de-escalation, and that different dimensions related to the regime do not all necessarily escalate simultaneously.

Theory-Based Evaluation Meets Ambiguity: The Role of Janus Variables

As theory-based evaluation (TBE) engages in situations where multiple stakeholders help develop complex program theory about dynamic phenomena in politically contested settings, it becomes difficult to develop and use program theory without ambiguity. The purpose of this article is to explore ambiguity as a fruitful perspective that helps TBE face current challenges. Literatures in organization theory and political theory are consulted in order to cultivate the concept of ambiguity. Janus variables (which work in two ways) and other ambiguous aspects of program theories are classified and exemplified. Stances towards ambiguity are considered, as are concrete steps that TBE evaluators can take to identify and deal with ambiguity in TBE.

The Changing Role of Evaluation in a Changing Society

The title of this chapter suggests that evaluation is a fluid, not a fixed phenomenon, and that the best way to understand this moving target is by looking at it in a sociological context. More specifically I point out five issues which I will argue are central to that understanding. What qualifies an issue is simply that it is important for evaluation and society, and that it is likely to remain contested because reasonable people disagree about it. Before I come to the issues, however, it is necessary to clarify my perspective and, following from that, my thoughts on the definition of evaluation.