Ian Sanderson

Getting Evidence into Practice: Perspectives on Rationality

The UK Government’s renewed emphasis on evidence-based policy making (EBP) is held to reflect a less ideological, more pragmatic ‘third way’ to developing and implementing economic and social policies. The new mantra – ‘what matters is what works’ – signals a resurgence of traditional notions of rationality in policy making. It implies a central role for social research and evaluation in developing robust evidence of how and why policies do, or do not, ‘work’ and policy-making processes that prioritize the influence of such evidence. Of course, there is nothing new in the notion that the application of knowledge to the conduct of social affairs improves the prospects for human betterment. This was the conviction of Francis Bacon in the 17th century (Zagorin, 1998), whose advocacy of systematic critical empiricism founded upon careful experimentation was passionately endorsed by John Dewey, in the early 20th century, in his quest for a philosophical basis for the capacity to achieve social progress:

Performance Management, Evaluation and Learning in ‘Modern’ Local Government

Public sector reforms throughout OECD member states are producing a new model of ‘public governance’ embodying a more modest role for the state and a strong emphasis on performance management. In the UK, the development of performance management in the context of the ‘new public management’ has been primarily ‘top-down’ with a dominant concern for enhancing control and ‘upwards account-ability’ rather than promoting learning and improvement. The development of performance management and evaluation in local government in the UK has been conditioned by external pressures, especially reforms imposed by central government, which have encouraged an ‘instrumental–managerial’ focus on performance measurement. The new Labour governments programme of ‘modernizing local government’ places considerable emphasis on performance review and evaluation as a driver of continuous improvement in promoting Best Value. However, recent research has indicated that the capacity for evaluation in local government is uneven and many obstacles to evaluation exist in organizational cultures. Local authorities need to go beyond the development of review systems and processes to ensure that the capacity for evaluation and learning is embedded as an attribute of ‘culture’ in order to achieve the purpose of Best Value.

Evaluation in Complex Policy Systems

The renewed emphasis in the UK on ‘evidence-based policy making’ has sharpened the focus on the utilization debate in the evaluation community. Traditionally, the emphasis is placed on methodological concerns but this article argues for a sharper focus on the underpinning theoretical bases of policy evaluation, both in terms of its role in policy making and in terms of the substantive theories which inform policy development and implementation. In particular, the article seeks to assess the implications of complexity theory for our theoretical assumptions about policy systems. It is argued that, together with ‘new institutionalist’ thought and recent work on policy implementation structures, notions of complexity have substantial ramifications for the way in which we approach policy evaluation, given the contemporary concern to address ‘cross-cutting’ social problems through ‘joined-up’ policy initiatives. It is suggested that our thinking about evaluation reflects the broader reaction against ‘modernist’ conceptions of the role of social science in our quest to change and improve the world.

Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning

The credentials of the evidence-based policy movement appear to be increasingly subject to challenge based on research that has highlighted the limits on the use of evidence in policy making. However, moves towards a more ‘realistic’ position of evidence-informed policy making risk conflating prescription with description and undermining a normative vision of better policy making. This article argues that we need to review the ideas that underpin our thinking about evidence-based policy making, and move beyond the territory of instrumental rationality to a position founded upon two intellectual pillars: our developing knowledge about complex adaptive systems; and ideas from a pragmatist philosophical position – especially those of John Dewey – about social scientific knowledge and its role in guiding action to address social problems. This leads us to a conception of ‘intelligent policy making’ in which the notion of policy learning is central.

Evaluating Public Policy Experiments: Measuring Outcomes, Monitoring Processes or Managing Pilots?

The increasing use by central government in the UK of pilot programmes as a means of modernizing key public services marks a new style of policy formulation and implementation which in turn calls for a different kind of evaluation support. The ‘rational-objective’, ex post analyses favoured by ministers and officials over the last decade are no longer sufficient. Instead evidence from one of the earliest and most important of the new governments pilot initiatives (the national Best Value pilot programme) suggests that it will increasingly look to evaluators to act as change agents who are able to combine summative analysis of outputs and impacts with more formative approaches focused on developing a detailed understanding of processes. This presents important new challenges for both policy makers and evaluators.

Evaluation, learning and the effectiveness of public services

Attempts to locate the legitimatory nature of the discourse of quality in the wider context of prevailing ideas about the role of government in the promotion of social welfare and how public service organizations can deliver quality services. Elaborates on the prevailing conventional wisdom underpinning the “New Right” project to restructure public service. Gives a critique of this conventional wisdom which addresses the limitations of “consumerist” notions of quality and the role of instrumentally rational discourses in legitimizing the New Right project of restructuring the State. Develops an alternative conception of public service quality and finally outlines the role for evaluation in promoting social learning as the basis for achieving effectiveness in public services.

Beyond performance measurement? Assessing ‘value’ in local government

As reforms of the public sector have introduced market relationships and private sector management practices, a ‘managerialist’ conception of accountability has developed, tied to a panoply of forms of performance measurement and ‘surveillance’ and which is now strongly evident in the governments ‘best value’ framework for local government. This exemplar of ‘instrumental rationality’ provides an inadequate basis for realising the full potential role of evaluation in the process of ‘renewing’ local government. A ‘critical‐pluralist’ approach is proposed as a basis for enhancing the capacity of local government to address complex economic and social problems, to embed learning and improvement and to develop a meaningful ‘dialogue’ with its citizens.

Review: International Handbook of Labour Market Policy and Evaluation

annexes. It also seems to me that the question of the use of the results of an evaluation, in the decision-making processes and by the actors themselves, is overshadowed. At least in my own field of practice, proving the usefulness of evaluation in the short, the medium and the long term is a main issue. Taking into account the target public of the book, I think it would have been interesting to get more into details on the construction of the terms of reference, the role of different actors in evaluation, and especially of the ‘instance d’évaluation’. One may refer to P. Lascoumes and M. Setbon for a detailed discussion about the different roles played by these ‘instances’. As a conclusion, in spite of some problems of confusion and that mentioned above, this work is a useful contribution for local and regional officers in the public sector, as well as for other persons who are interested in issues regarding evaluation of public policies, even though they certainly would need to read other books and articles, and especially practice, to ‘become an expert’. Christina Nirup Chargée de mission évaluation et prospective Ministère de l’Equipement, des Transports et du Logement, Paris, France