Dennis J. Palumbo

The politics of program evaluation

PART ONE: THE POLITICAL CONTEXT OF EVALUATION Introduction - Dennis J Palumbo Politics and Evaluation Where Politics and Evaluation Research Meet - Carol Weiss PART TWO: EVALUATION POLITICS AND THE POLICY CYCLE Linking Program Evaluation to User Needs - Eleanor Chelimsky Evaluations Political Inherency - Michael Quinn Patton Practical Considerations for Design and Use What Should Evaluation Mean to Implementation? - Angela Brown and Aaron Wildavsky Policy Termination as a Political Phenomenon - Peter DeLeon PART THREE: EVALUATION POLITICS AND RESEARCH METHODS The Countenances of Fourth-Generation Evaluation - Egon G Guba and Yvonna S Lincoln Description, Judgement and Negotiation The Influence of Theory on What We See - Lawrence A Scaff and Helen M Ingram The Political Uses of Evaluation Research - Susan Tolchin Cost-Benefit Analysis and the Cotton Dust Standard The Politics of Meaning - Rita Mae Kelly

Street-Wise Social Policy: Resolving the Dilemma of Street-Level Influence and Successful Implementation

Street-level influence over the delivery of social policy is paradoxical; it promotes flexibility and innovation, yet allows indifference and abuse. Even in highly bureaucratized human service organizations, policy implementation requires policy adaptation (Mashaw 1983). Streetlevel workers who are close to problems and clients are likely to know what works in local environments and for particular groups (Handler 1986). Street-level workers are an important a source of innovation, yet most have little formal authority to make programmatic decisions. Their good ideas are often ignored by those higher up. Street-level adaptations of policy are not always positive, however. Many street-level workers use their influence over policy implementation to serve their own interests; they change policy to make their work easier and safer or to thwart policy with which they do not agree rather than to serve the needs of clients or the public (Hogwood and Gunn 1984; Levine, Musheno, and Palumbo 1980). Street-level influence over policy implementation is, therefore, both a prerequisite for justice in the delivery of human services and a source of considerable abuse. Street-level influence

Implementation theory and the theory-driven approach to validity

Abstract The emphasis on validity, a traditionally important question in science, has recently been heavily criticized. The Chen and Rossi approach attempts to answer some of these criticisms by redressing the balance between internal and external validity. The approach opens up the “black box” of process evaluation. Thus, implementation becomes a crucial factor in assessing the validity of program interventions. Inadequate attention to how a program is being or has been implemented poses threats to the four kinds of validity identified by Chen and Rossi. But there are at least four theories of implementation (top-down, bottom-up, adaptive, and evolutionary). These theories and their relationship to validity are discussed. The paper concludes that it may not be possible to achieve external validity, particularly if the adaptive and evolutionary theories of implementation are correct, and that it may be better to move away from causal to other forms of theorizing to get out of the validity dilemma.

Evaluating Gang Resistance Education and Training (GREAT): Is the Impact the Same as That of Drug Abuse Resistance Education (DARE)?.

Gangs became a major concern of law enforcement and the public in the late 1980s and remain so in the first half of the 1990s. One response for addressing the problem has been the development and dissemination of a prevention program called Gang Resistance Education and Training (GREAT). This program is similar to the popular Drug Abuse Resistance Education (DARE) program of the 1980s. No published evaluations exist about GREAT although there have been numerous evaluations of DARE. The latter show that DARE has a very small effect on the drug behavior of children. This research reports an evaluation of GREAT in several sites in the United States that show that it also has a very small effect on children. However, as is the case with DARE, GREAT will continue because of the powerful symbolic political and public relations utility it has for various stakeholders.

Conflict versus consensus models in policy evaluation and implementation

Abstract Evaluators serve many masters whose interests do not always coincide. Traditional evaluation approaches, such as evaluability assessment and utilization-focused evaluation, assume that a consensus can be reached among “key policymakers, managers, and staff” about what goals should guide a program and about how well it is being implemented and what impact it is having. However, the two evaluations that are described in this paper show that this consensus model is inappropriate in many cases because there is not a single “reality” about programs. There are often multiple realities that are socially constructed by different stakeholders, and these realities often are in conflict. The two evaluations that are described are juvenile programs (one public and one private) for females, and a home arrest program using electronic monitoring. In the first, implementors disagreed about whether control and security should receive priority or treatment and rehabilitation. In the second, some policy formulators stressed lowering costs of corrections, others stressed public safety, and still others emphasized diverting offenders from prison. Also, different implementors gave priority to various goals, such as control and discipline, intermediate punishment and increasing options for corrections, and treatment and rehabilitation. Given these multiple and conflicting interpretations, a constructionist approach to evaluation is much more appropriate than a traditional, positivist consensus model.

The preconditions for successful evaluation: Is there an ideal paradigm?

The dominant paradigm in evaluation research is undergoing serious challenge. This article explores the ideal role of evaluation in decisionmaking, the methodologies for conducting evaluations, the congruence between evaluation methodology and actual organizational behavior, and the relationship between evaluators and program managers. We conclude that although there are serious disparities between the ideal and the actual in each of these four areas, and especially in the congruence between evaluation methodology and organizational behavior, there is not likely to be a change in the dominant paradigm because it is difficult for practitioners to use the language and values of new organizational perspectives.

The Social Construction of Intermediate Punishments

Intermediate sanctions have been said to provide judges with a wider range of sentencing options so that they might better match the severity of punishment with the seriousness of the crime, while diverting nonviolent offenders from prison without posing a risk to public safety. However, the social construction of intermediate sanctions assumes that government is a rational actor that wants to achieve just deserts and crime reduction. The authors argue that instead, intermediate sanctions are socially constructed via political symbolism that is meant to convince the public that government is continuing to be tough on crime while reducing prison costs. Through discussion of shock incarceration and intensive probation, the authors contend that a more favorable way of “doing criminology” and influencing public policy is through discourse about principles of social justice.

Community Corrections as an Organizational Innovation: What Works and Why

The contemporary emphasis of criminal justice policy on incapacitation of felony offenders has ironically opened up a window of opportunity for the expansion of alternatives to incarceration, including community corrections. This study analyzes the organizational diffusion of state-mandated community corrections policy in Connecticut, Colorado, and Oregon. Specifically, we measure the degrees of implementation in each state and analyze the organizational conditions that contribute to successful implementation. Also, we present a model of transformative rationality that points to the theoretical underpinnings of successful implementation. It identifies organizational conditions that are necessary to maintain a commitment to the fundamental premises of policy while simultaneously encouraging constructive adaptation of the policy to local environments.

Policy Evaluation for Local Government

Illustrations Acknowledgments Introduction: New Directions in Policy Evaluation for Local Government by Terry Busson and Philip Coulter Part I. Identifying Prospects and Problems Part II. Satisfying Citizens Part III. Achieving High Performance Part IV. Coping with Fiscal Stress Bibliography Index About the Editors and Contributors

Social science and social policy

This book examines the role of the social sciences in shaping and evaluating social policy. It considers the past, present and potential role of the social sciences in policy creation, implementation, and evaluation processes. The authors present a considerable range of opinions about the ability of the social sciences to provide useful information is presented. The common view is that the social sciences can contribute to policy-making processes to a modest extent.