Valérie Pattyn

Validating Methods for Comparing Public Policy: Perspectives from Academics and “Pracademics”. Introduction to the Special Issue

The launch of the Journal of Comparative Policy Analysis almost 20 years ago marked the emergence of a new generation of explicit comparative policy analytic studies (GevaMay 2013). The birth of the journal was an indicator of the revitalizing interest in the field, or what promised to be the “renaissance” of comparative policy analysis (Deleon and Resnick-Terry 1998, p. 13). Although the history of comparative policy analysis (CPA) goes back decades earlier, with a boom especially in the 1970s and 1980s, CPA was not perceived to be the best remedy to the problems faced at that time. Among other reasons, the recognition of the inherent contextuality of policy problems and of their wider polities was considered to jeopardize the possibility of making reliable comparisons across nations and other jurisdictions. In addition, and related to this, because of the very peculiarities of different macrosocial units (Ragin 1987, 2014), the ambition to practise policy sciences from one mega-policy umbrella appeared unrealistic. How to aggregate national policies that are marked by extensive heterogeneity? These complexities, combined with the absence of demand outside the academic community for genuine

Impact of performance audit on the Administration: a Belgian study (2005-2010)

Purpose - This study of the impact of Belgian Court of Audit on the federal Administration for the 2005 to 2010 period aims to highlight the auditors’ influence on the management of governmental organizations through the performance audits they have been conducting since 1998. A set of ten variables allows us to measure the three types of uses of performance auditors’ work by auditees: instrumental, conceptual and strategic uses. Design/methodology/approach - A survey was sent out to a total of 148 respondents identified by the authorities of the targeted organizations. 47 usable questionnaires were completed (32% response rate). Findings - The Court of Audit’s impact on the audited entities did not provoke radical changes in the auditees’ organizational life but the intervention of the auditors was nevertheless noticeable. The nature of the impact was rather conceptual than strategic or instrumental. And the negative consequences on auditees anticipated in the literature were not observed. Research limitations/implications - Given the five-year period covered by the study which was made in 2014 (four years after 2010), it had to deal with the mortality of respondents and the loss of organizational memory. Practical implications - The study gives more accurate insights about the influence that Supreme Audit Institutions actually exert on audited Administrations through their performance audits. Originality/value - Since Supreme Audit Institutions have been mandated to evaluate government’s economy, efficiency and effectiveness for almost 40 years in the western democracies, it is mandatory that their actual ability to influence Administrations be documented more abundantly and independently by academic researchers.

Taking Care of Policy in Times of Crisis: Comparative Lessons from Belgium’s Longest Caretaker Government

ABSTRACT When the financial crisis hit the Eurozone, Belgium, along with several other countries, postponed its consolidation policies until after the general elections. What followed was the longest caretaker rule that any stable democracy had ever experienced. This article analyses the phenomenon of policy continuity and change during this double crisis. It illustrates how the exogenous economic crisis overrode the endogenous political crisis, showing the extent to which international policy determinants expanded the remit of caretaker policy-making. At the same time, our analysis of the nature of caretaker conventions, the nature of multi-level governance, the permanence of administrative personnel, and the re-invention of parliament offers opportunities to draw lessons and deepen comparative research on policy termination and maintenance in the face of crisis.

Policy evaluation in Europe

Policy evaluations are increasingly considered a taken-for-granted prerequisite for a well-performing public sector. In this chapter, we address the question whether this view reflects the actual situation concerning evaluation capacity and culture in Europe. First, we reflect on the history of policy evaluation in Europe, by distinguishing between two ‘waves of evaluation’: the countries in Northwestern Europe that have conducted evaluations since the 1960s and the countries in the rest of Europe for which evaluation is a more recent phenomenon. Next, to illustrate the two waves of evaluation, we zoom in on three political systems that represent the national, regional, and international level in Europe: the United Kingdom (UK), Flanders (Belgium), and the EU. For each system, we map evaluation culture and capacity by analyzing six indicators. The chapter concludes with a reflection on current trends in evaluation research and possibilities for future research.

Policy Analytical Practice Investigated: Exploring Sectoral Patterns in Use of Policy Analytical Techniques

This chapter investigates patterns in the application of policy analytical techniques by government officials across different types of policy sectors in three subnational administrations in Belgium. Even when there is general consensus about the importance of policy analytical capacity, government officials’ deployment of policy analytical tools may vary across policy sectors, both in terms of frequency as well as in terms of type. To explain these variations, the chapter examines the role of three explanatory conditions that were originally identified to account for variance in policy analytical practice at the national level of analysis, but may also be relevant for the diffusion of policy analytical praxis across different types of policy sectors. These conditions are the role of social scientists in a particular sector, the degree of government spending per sector, and the receipt of EU subsidies. The analysis draws on recent survey material in Belgium carried out in three different subnational administrations: the Flemish government administration (Flanders), the administration of the Walloon Regional government (Walloon Region), and the administration of the government of the French-speaking community of Belgium (French-speaking Community).

Qualitative Comparative Analysis as an Evaluation Tool: Lessons From an Application in Development Cooperation

Qualitative comparative analysis (QCA) is gaining ground in evaluation circles, but the number of applications is still limited. In this article, we consider the challenges that can emerge during a QCA evaluation by drawing on our experience of conducting one in the field of development cooperation. For each stage of the evaluation process, we systematically discuss the challenges we encountered and suggest solutions on how these can be addressed. We believe that sharing this kind of lessons learned can help evaluators become more familiar with QCA, shedding light on what it is to be expected when considering the application of QCA for an evaluation, at the same time reducing unfounded fears and promoting awareness of traps and requirements. The article can be insightful and potentially inspirational for both commissioners and evaluators.