Ross F. Conner

Selecting a Control Group An Analysis of the Randomization Process in Twelve Social Reform Programs

This study examines the randomization process in twelve different social reform projects in the fields of health, education, and law. All of the programs used the same type of evaluation plan: a true experimental design involving randomization of clients to treat ment and control conditions. The implementation of this plan is discussed for each project. In addition, the projects are analyzed in terms of five aspects of the assignment process. The relationships between these aspects and the successful or unsuccessful implementations of the twelve evaluation designs are discussed. Finally, recommenda tions are made about issues which should be considered by program administrators and researchers before planning and implementing a true experimental evaluation.

PARTICIPATORY ACTION RESEARCH FOR ENVIRONMENTAL HEALTH: ENCOUNTERING FREIRE IN THE URBAN BARRIO

ABSTRACT: The community-based approach to health research and intervention is a model of inquiry rooted in Freire’s participatory action research (PAR). We need to show, in concept and practice, what it is about PAR that may be well suited for the types of health issues we encounter in inner-city environments. What type of learning results, how does this respond to particular health issues in the urban context, and what are the particular challenges faced in translating Freire’s model into today’s urban setting? To investigate these questions, we describe a recent PAR project in Southeast Los Angeles, California—an area known to some as “Asthmatown.” One salient finding of the research is that PAR allows the integration of complex and multiple forms of knowledge, and this is a necessary response to the complex and multiplex nature of cumulative impacts. There are challenges to translating the model to the urban setting, however, such as the difficulties of participation in today’s urban milieu. The research leads to some lessons for practitioners, such as the need to build “constant” elements into PAR projects. Lastly, we reflect on implications of this model for institutional reform.

Evaluator Preconceptions About Organizational Decision Making Rational Versus Incremental Perspectives

Evaluators have various preconceptions, usually implicit, about how decisions are made in social programs and how the results of evaluation research will be used in that process. This article demonstrates how an evaluators choice of a decision-making model has significant impact on the conduct and fate of the research. This focus adds a new dimension to the understanding of the ways political and organizational processes influence evalua tion research.

Professionalization of evaluative research: Conflict as a sign of health

Morell and Flaherty (1978) present arguments about the current status of evaluative research as a profession. This paper contains a critique, using a process model of professional development, of the perspectives employed by Morell and Flaherty. Issues discussed are the nature and substance of conflicts in evaluative research, including conflicts over methodological perspectives, the role of the evaluator in program planning, and the appropriate uses and users of evaluative information. The paper concludes with a discussion of the needs within the profession of evaluative research if it is to remain viable and responsive to change.

Toward a social ecological view of evaluation use

In her address to the 1997 meeting of the American Evaluation Association, Weiss (1998) revisits a discussion of evaluation use that began ten years earlier when she gave another speech on use of evaluation to the Association (Weiss, 1988a). Her remarks are, wisely, not focused on the details and issues of the old discussion (Patton, 1988; Weiss 1988b) but instead on new conceptions of and perspectives on evaluation use. Her charge had been to talk about theories of use, and she takes this task seriously. While she does not explicitly set out a theory of use, she does consider three important questions that are central to evaluation use: What do we mean by use? What is it that is used? and Who are the users? I will build on Weiss’ answers to her three questions and her comments on more effective use to set out the outlines of a theory of evaluation use. It is a measure of the effectiveness of Weiss’ talk that I am moved to take this step, in spite of some misgivings that it is foolish to try to provide a rough shape to an issue that has long troubled evaluators (for example, Patton, 1978, 1997; Posavac and Carey, 1997; Rossi and Freeman, 1982, 1993; Suchman, 1967; Weiss, 1972). In the sections below, I will briefly review and comment on Weiss’ main points, focusing in particular on several ideas that motivated my preliminary conceptualizaion of a theory of evaluation use. Then, I will sketch out a social ecological theory of evaluation use that may help to take the discussion of use to the next step.

A cross-cultural assessment of health promotion/disease prevention programs

This paper presents a preliminary report on a study of current health promotion/disease prevention efforts outside of the United States. This research has involved visits over the past three years to a variety of health promotion programs in Eastern and Western Europe, Asia and the Asian subcontinent, and Central and South America. The main goal of this study was to develop new insights into the more and less effective ways to promote health and prevent illness and disease by understanding how other cultures conceive of and approach issues related to health, wellness and illness. Another important goal was to analyze the ways in which these health promotion programs evaluated their effectiveness. The findings from the study demonstrated revealing differences in approaches to health promotion between Eastern and Western cultures and between developed and developing countries. In addition, there were important differences in epistemological approaches related to evaluation.

Measuring Need and Demands in Evaluation Research: Results of a National Survey of College and University Administrators about Desired Evaluation Services.

This article focuses on the distinction between needs assessment and demand assessment and presents a methodology for operationalizing and measuring demands. We report the results of a survey of a national sample of 32 university and college administrators to assess their need and demand for an adult student opinion package. The study was noteworthy for its differentiation of need for a service (i.e., interest in the service) and demand for a service (i.e., willingness to pay), and its attempt to determine the cost at which demand decreased significantly in relation to need for various evaluation services. The results indicated that levels of need and demand are positively related but not identical. The article concludes with a discussion of the demand concept, a critique of our operationalization of the concept, and a discussion of the use of the results of the study.

Training prison inmates as paralegals: An experimental project

Abstract A program to train inmates as paralegals was established at a large state prison in the Midwest. Qualified inmates were randomly assigned to the training or to a control condition. Both groups were administered measures assessing legal knowledge, the use of legal knowledge, attitudes about the criminal justice system, and self-perceptions before and after training. The trainee group showed a significant increase in its amount of legal knowledge after training. Members of the trainee group also reported a significant increase in the number of other inmates requesting aid from them on legal problems. In addition, interviews were conducted with two random samples of the general inmate population, before and after the paralegal training program. These interviews revealed strong support for the training program, and this support increased over the period of the program. Limited data on longer-term effectiveness of the paralegals performance in the prison legal aid clinic and in paralegal jobs following parole supported the positive, shorter-term results.

The Colorado Healthy Communities Initiative

Ross F. Conner is a professor at University of California, Irvine (UCI), director of the UCI Center for Community Health Research, and a former president of the American Evaluation Association. He is most noted for his work in the area of community-based evaluation. His evaluation of the Colorado Healthy Communities Initiative, which involved 28 different community-based healthpromotion planning and implementation projects, received the American Evaluation Association’s 2002 Outstanding Evaluation Award. In this interview, we learn about this 8-year award-winning study and the challenges facing evaluators when conducting community-based evaluation.

Book Reviews: Scientific Inquiry and the Social Sciences: A Volume in Honor of Donald T. Campbell. Edited by Marilynn B. Brewer and Barry E. Collins. J ossey-Bass, San Francisco (523 pages,

be used. I am sure that exceptions are to be found in economics, business, and finance; however, I do not think the readers of this book will be from those disciplines. As for me, the less said and printed about this design for social science research and evaluation, the better. Chapters 8, 9, 10, and 1 I cover a number of miscellaneous topics that could be assigned as supplementary reading in a regular beginning research methods course. The topics include mortality of subjects, independence of observations, unit of analysis questions, correlational analysis, structural equation, interactions, secondary analysis, and combining results across studies. The book is readable and easy to understand provided thc reader knows quite a bit about research methodology. The book is recommended to students, researchers and evaluators with this knowledge.