Lehn M. Benjamin

The Merits of Mixing Methods in Evaluation

Evaluators are challenged to understand human behavior in all of its natural complexity and individuality. Our work is conducted in natural settings, where history and context matter, where human behavior traces complex patterns of influence and relationship, where what is meaningful to those in the setting is both phenomenological and structural, arising from both lived experiences and the societal institutions that frame and shape those experiences. Engaging this complexity requires not a privileging of just one way of knowing and valuing, but rather a marshalling of all of our ways of understanding in a framework that honors diversity and respects difference. This framework is advanced in this article as a mixed-method way of thinking. A presentation of key concepts in the framework is followed by case examples from the US.

Account Space: How Accountability Requirements Shape Nonprofit Practice

Improving nonprofit accountability is one of the most important issues facing the sector. Improving nonprofit accountability in ways that are attentive to what we might consider unique and valuable about how nonprofits address public problems is the challenge at hand. This article presents a framework for examining the consequences of accountability systems for nonprofit practice. Drawing on empirical findings from three case studies and early sociological work on accounts, the framework considers four questions (i.e., When do organizations give accounts? What is the purpose of the account? When are those accounts accepted or rejected by important stakeholders? And with what consequence?) but makes a distinction between a verification and explanatory accountability process. By making this distinction and clarifying the relationship between these two accountability processes, the proposed framework can be used to identify conflicts between accountability systems and nonprofit practice and to understand how efforts to ensure accountability can spur a change in nonprofit practice, change stakeholder expectations for nonprofits or leave both intact.

Community development financial institutions: current issues and future prospects

ABSTRACT: Community development financial institutions (CDFIs) help to address the financial needs of under-served, predominantly low-income communities. CDFIs include community development banks, credit unions, business and microenterprise loan funds, and venture capital funds. Although CDFIs are a rapidly growing and an increasingly important area of community economic development, they have not received proportionate attention from academic researchers. This article begins to address the gap. It outlines the history of the CDFI industry and details how CDFIs are responding to three specific development needs: basic financial services; affordable credit for home purchase, rehabilitation, and maintenance; and loan and equity capital for business development. The article then considers the strengths and limitations of CDFIs, concentrating especially on the relationship between CDFIs and conventional financial institutions. It concludes by examining the impact that these alternative financial institutions realistically can hope to achieve.

Bearing More Risk for Results: Performance Accountability and Nonprofit Relational Work

Performance accountability systems require nonprofits to bear more risk for achieving results. Although a growing body of work has examined nonprofit accountability, less attention has been given to the concept of risk. This article points to a potential conflict between performance accountability frameworks and nonprofit work. This conflict can be best understood as a one between managing risk in task-driven relationships, in which relationships are formed simply to achieve desirable results, and managing risk in developmentally driven relationships, in which performing a task is intended not only to achieve desirable results but also to build enduring capacity to take action on common problems.

The Potential of Outcome Measurement for Strengthening Nonprofits’ Accountability to Beneficiaries

This article considers one mechanism that could create a clearer accountability path between nonprofits and their beneficiaries: Outcome measurement. Outcome measurement focuses attention on a nonprofit’s beneficiaries and whether they are better off as a result of the nonprofit’s work. The article analyzed 10 outcome measurement guides targeted to nonprofits, totaling more than 1,000 pages of text. The analysis shows that the guides were neither uniform in the conceptualization of nonprofit beneficiaries nor in how they directed nonprofits to use outcome measurement with their beneficiaries. Despite scholars’ suggestion that a nonprofit’s relationship to their beneficiaries is a key accountability relationship, the guides suggest that beneficiaries have an ambiguous standing, relative to other stakeholders, in the nonprofit accountability environment.

Nonprofit Organizations and Outcome Measurement: From Tracking Program Activities to Focusing on Frontline Work

Why do we continue to see evidence that nonprofit staff feel like outcome measurement is missing important aspects of their work? Based on an analysis of over 1,000 pages of material in 10 outcome measurement guides and a focused literature review of frontline work in three types of nonprofit organizations, this article shows that existing outcome measurement frameworks focus on how staff implement programs rather than how staff work with clients. Outcome measurement guides direct nonprofits to track program activities completed and the outcomes resulting from those program activities. In contrast, the accounts of frontline work in nonprofits show that nonprofit staff start by building a relationship with the person they are serving and then adjusting programs and services to better meet the needs and goals of this individual. Consequently, outcome measurement may go some distance in helping us understand nonprofit performance but may also mischaracterize nonprofit performance.

From Program to Network The Evaluator’s Role in Today’s Public Problem-Solving Environment

Today’s public policy discussions increasingly focus on how networks of public and private actors collaborate across organizational, sectoral, and geographical boundaries to solve increasingly complex problems. Yet, many of evaluation’s key concepts, including the evaluator’s role, assume an evaluand that is programmatically or organizationally defined and bounded. This article explores the implications of this changing public policy environment for the evaluator’s role by examining one case: an evaluand that was a loose collaboration of four individuals in dispersed organizations working to reframe public policy and to change professional practice in early care and education. We describe this evaluand and the dimensions of it that challenged our evaluative role. We conclude by suggesting an alternative conception of the evaluator’s role that can serve evaluators in this changing policy environment.

Evaluator's Role in Accountability Relationships Measurement Technician, Capacity Builder or Risk Manager?

While evaluators have considered accountability pressures and examined the consequences of performance measurement for organizations, they have paid less attention to the accountability relationship in which their work is used. Drawing on theoretical literature and empirical data, this article shows how these accountability relationships drive the use of evaluative frameworks, like performance measurement, and shape the consequences for the organizations involved.The article argues that in settings permeated by accountability concerns, evaluators need to understand these accountability relationships that so significantly shape their work, especially if they hope to improve accountability systems or create the space for evaluation to serve other important purposes.This article provides evaluators with a set of questions and points of analysis that they can use to examine accountability relationships and expand what is possible for evaluation in settings where accountability is the dominant concern.

Nonprofit Performance: Accounting for the Agency of Clients

Performance is a key concern for nonprofits providing human services. Yet our understanding of what drives performance remains incomplete. Existing outcome measurement systems track the programmatic activities staff complete and the extent to which participants respond in programmatically intended ways. However, clients do not just receive services and respond as intended and staff do not simply complete program activities. Drawing on a data set of 47 interviews with frontline staff in eight human service nonprofits, we show how frontline staff work in a partnership with clients to set an agenda for change and achieve desired results. We call this co-determination work and argue that it represents a critical and often neglected dimension of nonprofit performance.

Mediating Accountability: How Nonprofit Funding Intermediaries Use Performance Measurement and Why It Matters for Governance

Performance measurement has become an important tool for ensuring accountability in a governance environment, where addressing public problems often takes place outside the direct purview of government. Although a good deal of attention has been given to governments use of performance measurement in these settings, either in contracting relationships or interorganizational networks, this paper argues that ensuring accountability in a governance environment requires greater attention to how nonstate actors, or the other principals, use performance measurement. This paper focuses on nonprofit funding intermediaries and their use of performance measurement. Nonprofit funding intermediaries gather funds from a range of public and private donors and regrant these monies to a defined set of local nonprofits. As such, they occupy somewhat unique positions in a web of actors all seeking to solve public problems. We offer a conceptual overview of intermediaries and then critically examine how three nonprofit funding intermediaries used performance measurement.