Charles Windle

Ethical problems in program evaluation: Advice for trapped evaluators☆

This paper argues that ethical problems in program evaluation are increased when conflicting or incompatible evaluation models are applied concurrently. Three models are illustrated: an Amelioration Model, generating better information for a programs own decision makers; an Accountability Model, focusing on public data disclosure and citizen participation in evaluation; and an Advocacy Model, in which the evaluation is designed to advance the programs interests in the competition for resources. Ethical problems inherent in each model, and arising from combining models are presented, along with case examples and discussion. The authors suggest several directions that program evaluators can consider to reduce their vulnerability to ethical problems.

Performance measures for mental health programs: Something better, something worse, or more of the same?

Current pressures for establishment of accountability systems based on performance measures for mental health programs are likely to improve services only if such systems are accompanied by supportive research, preparatory orientation, wide participation, tested data systems, and elimination of other redundant accountability procedures. A 4-phase, 3-functional level model is proposed to guide implementation.

Mandated Evaluation in Community Mental Health Centers

Experience with federally mandated Community Mental Health Center (CMHC) program evaluation (P.L. 94-63) is examined in terms of three issues: (1) the purposes which evaluation serves; (2) the values about evaluation held by key participants; and (3) limitations in both evaluation technology and CMHC context and resources. Eight generic principles for government role in evaluation are derivedfrom this experience. The principles suggest two different roles—accountability to the public and program amelioration. Government requirements should be concerned solely with the former. Specific recommendations for implementing these two roles are given.

A framework for understanding participation in Community Mental Health Services

Citizen participation has been associated with confusing rhetoric and conflicting philosophical rationales, both in its long history in America and in the community mental health movement. While CMHCs have emphasized participation less than many other social programs begun in the 1960s, participatory roles have evolved. Some of the confusion surrounding this complex topic can be clarified by distinguishing three dimensions of participation—power, participants and program functions. Conflicting societal trends leave the future of participation in the community mental health movement uncertain.

Correlates of community mental health center underservice to non-whites.

The relationships between relatively low utilization rates for non-whites and catchment area demography and center service characteristics were examined for 142 federally funded community mental health centers. Center characteristics were less strongly related to relative utilization by non-whites than were area demographic characteristics. Several characteristics of the black population were among those most highly associated with relative non-white utilization rates.

Patterns of Program Evaluation in Community Mental Health Centers

In an exploratory study of the program evaluation efforts of federally funded community mental health centers prtor to the enactment of mandatory program evaluation, survey data were submitted to factor- and cluster-analyses. Results yielded: (1) a typology of evaluation activities performed by community mental health centers (2) a meaning ful patterning of (organizational) approaches to program evaluation suggesting developmental stages of an increasing range of types of activities; and (3) the tentative conclusion that, at a low level of evaluation effort, client-change evaluations are less related to impact on program development than are evaluations using descriptive statistics about chents.

Evaluation in the community mental health centers program: A bold new reproach?

The Federal Community Mental Health Centers Program (CMHC)-from 1963 to 1981-was heralded as a revolution in mental health care. Championed by many, and severely criticized by others, the actual impact of the program on the nations mental health remains unclear. The authorization to evaluate the CMHC Program came originally from congressional legislation (PL 90-174), and later from the policies and regulations of NIMH under a series of Federal laws, notably PL 94-63. From 1976-1980, two dominant evaluation strategies were prevalent: funds expended by NIMH each year for studies of CMHC services or program-wide evaluations, and a much larger expenditure by CMHCs to conduct their own, independent evaluations following federal guidelines. As the Centers Program was turned over to the states in the form of block grants (PL 97-35), a group of professionals involved with setting and carrying out federal CMHC evaluation policy of both varieties met in public forum to debate the impact of these two evaluation approaches. While some participants cited gains in evaluation technology and impact upon local management of CMHCs, others found the lack of a coordinated and systematic approach to evaluating the CMHC Program to have been an opportunity missed. The impact of CMHC evaluation efforts are also discussed in terms of their major contribution to the field of evaluation research as a whole.

Developmental Trends in Program Evaluation.

Abstract It is suggested that the ontogeny of program evaluators replicates the phytogeny of the profession of program evaluation. A trend across the purposes of program evaluation from program amelioration to program advocacy and ultimately to program accountability is projected.

The shift to research synthesis

ConclusionThe above techniques can help establish more accurately the state of knowledge applicable to improving services. These techniques can be expected to be used more as the United States struggles to make its health and mental health care systems more efficient.

COMMUNITY PARTICIPATION IN EVALUATION

Publisher Summary Although the idea of citizen participation in agency evaluation is supported theoretically by lay citizens and agency officials, it has actually been rarely practiced. Part of the difficulty appears to stem from the fact that each of the groups supporting the concept does so for very different and sometimes conflicting reasons that lead to different forms of citizen involvement. It is one thing for an agency administrator to support an idea that promises to win greater community support for the programs of an agency, but quite another to give residents of the community decision–making power over which programs should be supported. Citizens who are willing to give time and effort when there is evidence that this input can make a difference, are, however, alienated by the prospect of becoming mere rubber stamps for someone elses decisions. There are two major areas of ambiguity surrounding the concept of citizen participation: (1) the extent and conditions under which citizen participation is an aid to services; and (2) the compatibility of those conditions with effective services. The chapter describes—(a) the limitations in program evaluation as it is usually conducted by alleged evaluation specialists; (b) trends toward accountability in program evaluation that lower dependency on evaluation specialists; (c) a conceptualization of broad participation in program evaluation; (d) examples of citizen participation in evaluation; (e) three issues in citizen participation: power, appropriate participants, and participant capacity; and (f) suggestions for improving the practice of citizen evaluation.