Elizabeth Whitmore

Arguments for a Common Set of Principles for Collaborative Inquiry in Evaluation

In this article, we critique two recent theoretical developments about collaborative inquiry in evaluation—using logic models as a means to understand theory, and efforts to compartmentalize versions of collaborative inquiry into discrete genres—as a basis for considering future direction for the field. We argue that collaborative inquiry in evaluation is about relationships between trained evaluation specialists and nonevaluator stakeholders (i.e., members of the program community, intended program beneficiaries, or other persons with an interest in the program) and that practice should, in the first instance, be sensitive to stakeholder interests and context, and it should be principle-driven.

Accompanying the process: social work and international development practice

In recent years, international development approaches have ranged from ones in which ’experts’ identify, define and propose solutions to someone else’s problems to the emphasis on ’partnership’ now advocated by many development agencies. The term partnership implies a relationship of equals but, where North and South are involved, in practice this is rarely the case. Meanwhile, the reality is that most people’s lives have not improved over the past 30 years of ’development’ efforts; indeed, the economic situation for the majority of people in the world has worsened significantly (Murphy, 1991). There is clearly a need for attention to the effectiveness of ’development’ programs. Frequently missing in development work, whether done by government agencies or by non-governmental organizations (NGOs), is an approach which combines both structural and conjunctural analysis which, we would argue, forms the essential basis for thorough understanding and effective action. Structural analysis involves examination of the wider economic, social and political structures which oppress people in both rich and poor countries and which are pervasive in the relationship between ’partners’. Conjunctural analysis relates to the immediate situation and

Introducing Evidence-Based Principles to Guide Collaborative Approaches to Evaluation Results of an Empirical Process

This article introduces a set of evidence-based principles to guide evaluation practice in contexts where evaluation knowledge is collaboratively produced by evaluators and stakeholders. The data from this study evolved in four phases: two pilot phases exploring the desirability of developing a set of principles; an online questionnaire survey that drew on the expertise of 320 practicing evaluators to identify dimensions, factors or characteristics that enhance or impede success in collaborative approaches in evaluation (CAE); and finally a validation phase involving a subsample of 58 evaluators who participated in the main phase. The principles introduced here stem from the experiences of evaluators who have engaged in CAE in a wide variety of evaluation settings and contexts and the lessons they have learned. They are understood to be interconnected and loosely temporally ordered. We expect the principles to evolve over time, as evaluators learn more about collaborative approaches in context. With this in mind, we pose questions for consideration to stimulate further inquiry.

Let There Be Light: Response to Fetterman et al. 2014

Fetterman and colleagues, in their letter to the American Journal of Evaluation (AJE) editor (Fetterman, Rodrı́guez-Campos, Wandersman, & O’Sullivan, 2014), make a plea for ‘‘more light than heat.’’ On that front, we are completely in accord. Indeed, in many respects, we find ourselves on the same side of the coin when it comes to collaborative inquiry in evaluation. Yet, as we lay out in this response to their letter, there are some points of departure that we wish to highlight because we do have differences and we believe it is important to be clear about them. Let us elaborate. On the other side of the aforementioned coin are those in our field, who argue for the supremacy of traditional objectivist forms of evaluation that view stakeholder participation in the production of evaluation knowledge as a form of heresy. Stufflebeam’s (1994) polemic ‘‘Where the future of evaluation should not go and where it needs to go’’ is a shining example. That critique was written in response to Fetterman’s 1993 American Evaluation Association (AEA) presidential address that extolled the virtues of a new approach in our field, empowerment evaluation. Fetterman’s term as president was followed by the establishment of collaborative, participatory, and empowerment (CPE) evaluation that has grown to become AEA’s sixth most subscribed topical interest groups (TIGs). But collaborative inquiry in evaluation, and more generally in the social sciences, did not start with empowerment evaluation or with the establishment of the CPE-TIG. It started a good number of years earlier in international development toward the late 1970s (Brisolara, 1998), and it had a particularly democratizing and transformative flavor to it. Over time many and varied species emerged in both developing and western contexts including, but by no means limited to, participatory action research (PAR), democratic evaluation, educative research, participatory rural appraisal (PRA), illuminatory evaluation, emancipatory and critical action research, collaborative action research, teacher research, deliberative democratic evaluation, and developmental evaluation (see Cousins & Chouinard, 2012, pp. 19–21 for elaboration). Of course, we need to add to the list CPE approaches, but what sets this group apart, at least in the words of Fetterman and associates, is the propensity for self-identification with the respective approach. ‘‘Collaborative evaluators are . . . . Participatory evaluators jointly share . . . . Empowerment evaluators view . . . .’’ (Fetterman et al., 2014, p. x, our emphasis). We have never encountered a colleague claiming to be a PARer,

The Transnationalization of Popular Movements: Social Policy Making From Below

ABSTRACT With the globalization of world economies, development of social policy has also become transnational. In light of the devastating human consequences of globalization and the ineffectiveness of ameliorative efforts rooted in neoliberal “economic correctness,” effective leadership is most likely to come from those rooted in popular struggles. To create a countervailing force to the global hegemony of neoliberalism, popular organizations are recognizing that they must also think and work transnationally. We suggest some tools for conceptualizing the issues to aid in the development of appropriate strategies for confronting this urgent reality, and to facilitate the formation of social policy from below.

Accompanying the Process: Principles for International Development Practice

ABSTRACT This paper explores issues in North-South development partnership, in the context of an experience of South-North cooperation in integrating gender awareness into social work curriculum in Nicaragua. Reflections on this experience produce a recommendation of acompanamiento as a model of international development practice. Informed by a ‘gender and development’ approach, this model can help our southern partners become genuine architects of their own change processes, rather than passive recipients of development assistance.

How Do Evaluators Differentiate Successful From Less-Than-Successful Experiences With Collaborative Approaches to Evaluation?

Objectives: In this exploratory study, we wanted to know how evaluators differentiate collaborative approaches to evaluation (CAE) perceived to be successful from those perceived to be less-than-successful. Method: In an online questionnaire survey, we obtained 320 responses from evaluators who practice CAE (i.e., evaluations on which program stakeholders coproduce evaluation knowledge). Respondents identified two specific CAE projects from their own experience—one they believed to be “highly successful” and another they considered “far less successful than [they] had hoped.”—and offered their comments and reflections about them. They rated the respective evaluations on 5-point opinion and frequency scales about (i) antecedent stakeholder perspectives, (ii) the purposes and justifications for collaborative inquiry, and (iii) the form such inquiry takes. Findings: The results showed that successful evaluations, relative to their less-than-successful counterparts, tended to reflect higher levels of agreement among stakeholders about the focal program; higher intentionality estimates of evaluation justification and espoused purposes; and wider ranges and deeper levels of stakeholder participation. No differences were found for control of technical decision-making, and evaluators tended to lead evaluation decision making, regardless of success condition. Discussion: The results are discussed in terms of implications for ongoing research on CAE.