Nick L. Smith

Improving Evaluation Theory Through The Empirical Study of Evaluation Practice

Articles should deal with topics applicable to the broad field of program evaluation. Implications for practicing evaluators should be clearly identified. Examples of contributions include, but are...

The certainty of judgments in health evaluations

Abstract Most evaluators seem to presume that all evaluative studies should result in conclusive evidence. Often, however, less than conclusive evidence may not only suffice, but be the only real alternative under conditions of limited evaluation resources. Three levels of evidence common to evaluation studies are discussed: suggestive evidence, preponderant evidence, and conclusive evidence. Ten factors are also presented which can be used to determine the level of certainty appropriate for a given evaluation study. Consideration of these factors enables the evaluator to specify when less than conclusive evidence is acceptable.

Empowerment Evaluation as Evaluation Ideology

As with many forms of evaluation, empowerment evaluation can be viewed as an ideology that promotes a particular set of social and professional values. Judging the quality and utility of empowerment evaluation thus requires a critical appraisal of the implications of adopting those values.

The Weiss-Patton Debate: Illumination of the Fundamental Concerns.

Feature articles should deal with topics applicable to the broad field of program evaluation. Implications for practicing evaluators should be clearly identified. Examples of contributions include but are not limited to reviews of new developments in evaluation; descriptions of a current evaluation effort, research problem, or technique; and discussions of theoretical, legal, or ethical issues affecting evaluation. Manuscripts should include appropriate references and not exceed 10 double-spaced typewritten pages in length.

Professional Reasons for Declining an Evaluation Contract

Professional evaluation is a market-based activity in which economic considerations play a dominant role. But are there ever professional reasons for turning down an evaluation contract? Considering how evaluations can promote either guild maintenance interests or societal improvement interests illuminates several dilemmas in the proper conduct of evaluation and suggests the need for greater explicit attention to what it means for the evaluation profession to contribute to the public interest.

Clarifying the merits of argument in evaluation practice

Abstract In prior writings, we have explored alternative means of justifying claims in evaluation practice. After a brief review of these previous investigations, we provide an analysis employing Toulmins (1958) jurisprudence metaphor in the assessment of evaluative arguments. Applications and implications for evaluation practice are discussed.

Alternative Teacher Preparation Programs: A Cost-Effectiveness Comparison.

pensive, without knowing whether any of them produce the desired effects. Treating outcomes without attention to costs can result in selecting program alternatives that are only marginally more effective, but exorbitantly more expensive than other alternatives. Only by incorporating both costs and outcomes within comparative studies of program alternatives can one reliably determine which alternative is most effective for a given cost, or how much it would cost to obtain a desired level of effect. A few writings designed to help researchers use cost-effectiveness analysis in program evaluation do exist. Alkin (1970) provided an early chapter advocating its use in the evaluation of instructional programs. Levin (1975) provided a sound general introduction to the topic, followed by an instructional analysis of specific applications in educational evaluation (Levin, 1981). Thompson (1980) also discussed the use of cost-effective-

Toward the justification of claims in evaluation research

Abstract Four types of claims generally arise in evaluation research: research claims, policy claims, evaluation claims, and management claims. The relationships among these types of claims can be seen both in the similarity of the questions under the types, and in their inlerrelatedness in making evaluation recommendations. By construing these claims as question-answer propositions, it is possible to identify the specific kinds of questions associated with each of the four types of claims. Under this view, evaluation research is predominantly a form of applied inquiry, and the justification of evaluation claims becomes the validation of question-answer propositions.

Communication strategies in evaluation

These original contributions provide fresh insights into the critical role of communication in successful programme evaluations. The authors draw on their expertise in a variety of fields, including health, criminal justice, housing, commmunity development and education, and on their first-hand experience with alternative communication techniques. They explain how forms of communication and representation affect the nature and usefulness of evaluative findings, and describe a broad range of practical strategies for presenting findings. This innovative collection will be welcomed by evaluators as a source of pragmatic techniques for making concerned clients more receptive to their work.

Evaluation Utility and Client Involvement in Accreditation Studies

ere is currently a general presumption among funding agencies, evaluation clients, and some evaluators themselves that evaluation is not as useful as it should be to local decision makers. In fact, very few empirical studies of the utility of evaluation have yet been conducted and the findings to date are mixed (Alkin, Daillak, & White, 1979; Patton, 1978). Further, it has been suggested that greater involvement of local groups in the evaluation process will increase the utility of evaluation to these individuals (cf. Wolf, 1979; National Institute of Education, Note 2). This paper reports on an empirical study of the relationships between the level of involvement of local decision makers in accreditation evaluations and their perceptions of the utility of those evaluation efforts. A study of local school board involvement in school accreditation evaluations was conducted to determine (a) how school boards are involved in such studies, (b) the relationships between the level of school board involvement and the utility of those evaluations to the