J. Bradley Cousins

Current Empirical Research on Evaluation Utilization

This paper reviews empirical research conducted during the past 15 years on the use of evaluation results. Sixty-five studies in education, mental health, and social services are described in terms of their methodological characteristics, their orientation toward dependent and independent variables, and the relationships between such variables. A conceptual framework is developed that lists 12 factors that influence use; six of these factors are associated with characteristics of evaluation implementation and six with characteristics of decision or policy setting. The factors are discussed in terms of their influence on evaluation utilization, and their relative influence on various types of use is compared. The paper concludes with a statement about implications for research and practice.

Developing expert leadership for future schools

Part 1 A perspective on developing expert leadership for future schools: a conception of expert leadership for future schools what research tells us about the present state of schools leadership. Part 2 The nature of expert leadership for future schools: envisioning the future schools expert school leadership on the high ground and in the swamp expert leadership on the high ground expert leadership in the swamp the special role of values in school leadership teacher development - a central problem for leaders of future schools collaborative school cultures - a key part of the solution. Part 3 Developing expert leadership for future schools: the socialization of school-leaders characteristics of formal programs for developing leadership on the high ground characteristics of formal programmes for developing expert school leadership in the swamp performance appraisal and selection of school leaders - performance appraisal for growth performance appraisal and selection of school leaders - selection processes and measurement issues implications for district leaders.

Evaluation use: Theory, research, and practice since 1986

Abstract As part of a larger effort by members of the American Evaluation Association (AEA) Topical Interest Group on Evaluation Use (TIG-EU), we undertook an extensive review and synthesis of literature in evaluation use published since 1986. We observe several recent developments in theory, research and practice arising from this literature. These developments include: the rise of considerations of context as critical to understanding and explaining use; identification of process use as a significant consequence of evaluation activity; expansion of conceptions of use from the individual to the organization level; and diversification of the role of the evaluator to facilitator, planner and educator/trainer. In addition, understanding misutlilization has emerged as a significant focus for theory and to a limited extent, research. The article concludes with a summary of contemporary issues, particularly with regard to their implications for evaluation practice.

The Nature, Causes and Consequences of Principals′ Practices: An Agenda for Future Research

Growing appreciation for the potential impact of principals on their schools has stimulated a significant body of research concerning the principalship. While many aspects of the principalship have been the object of study, it is often difficult to determine the relationship among these studies and how these studies, as a whole, contribute to a better understanding of the principalship. It is also difficult to judge which aspects of the principalship would provide the most productive focus for subsequent research. The review reported in this article addressed both sets of difficulties by analysing a total of 135 empirical studies conducted between 1974 and 1988; 60 of these studies were reported between 1985 and 1988 and received more attention than the earlier 75. Results of the analysis identify aspects of the principalship about which much is known, approaches to research which appear to have exhausted their usefulness and areas in which further study seems likely to be of most value. One major conclus...

Utilization Effects of Participatory Evaluation

Participatory evaluation (PE) turns out to be a variably used and ill-defined approach to evaluation that, juxtaposed to more conventional forms and approaches, has generated much controversy in educational and social and human services evaluation. Despite a relatively wide array of evaluation and evaluation-related activities subsumed by the term, evaluation scholars and practitioners continue to use it freely often with only passing mention of their own conception of it. There exists much confusion in the literature as to the meaning, nature, and form of PE and therefore the conditions under which it is most appropriate and the consequences to which it might be expected to lead. In spite of this confusion, interest in collaborative, empowerment, and participatory approaches to evaluation has escalated quite dramatically over the past decade as evidenced in a bourgeoning literature on such topics. This interest, in my view, is testament to the promise such collaborative approaches hold for enhancing evaluation utilization and bringing about planned sustainable change.

Antibullying Interventions in Schools: Ingredients of Effective Programs

Because bullying is a serious problem in Canadian schools, antibullying programs have been widely implemented to redress the problem. School principals in Ontario (N=395) completed a questionnaire to document the severity of bullying, the amount of anti-bullying resources, and the variety of antibullying activities in their schools. Results reveal that reductions in bullying in previous years, sufficiency of resources for resolving bullying, and amounts of antibullying programming were all positively associated antibullying program outcomes. These data suggest that the investment of time, effort, and money in school-based antibullying initiatives can lead to safer and more peaceful schools environments.

Arguments for a Common Set of Principles for Collaborative Inquiry in Evaluation

In this article, we critique two recent theoretical developments about collaborative inquiry in evaluation—using logic models as a means to understand theory, and efforts to compartmentalize versions of collaborative inquiry into discrete genres—as a basis for considering future direction for the field. We argue that collaborative inquiry in evaluation is about relationships between trained evaluation specialists and nonevaluator stakeholders (i.e., members of the program community, intended program beneficiaries, or other persons with an interest in the program) and that practice should, in the first instance, be sensitive to stakeholder interests and context, and it should be principle-driven.

Differences in Stakeholder Perceptions about Training Evaluation: A Concept Mapping/Pattern Matching Investigation.

Concept mapping and pattern matching techniques were used in exploratory research to investigate diAerences in stakeholder perceptions of training results and evaluation in a major division of a multinational network-design and engineering company. Referencing a research framework informed by multiple constituency views of organizational eAectiveness, this single case study examined group perceptions of the results and evaluation of training among 39 knowledge workers organized into three equal groups of 13. These groups consisted of (1) line managers as sponsors of training, (2) product developers as participants in training, and (3) training professionals as providers of training. A set of 100 statements were ‘‘brainstormed’’ by these groups to describe ideal results of training in terms of the success of the case organization. After sorting the 100 statements for conceptual similarity, all study participants also rated each statement’s importance twice — once as a training result, and once as a training evaluation criterion. Using the concept maps developed by the groups, a set of 21 pattern matches were performed to investigate stakeholder diAerences both within (intra) and between (inter) groups. Correlation (Pearson r) coeAcients were also calculated for each pattern match and displayed in tabular form for comparison. The overall results revealed that all stakeholder groups agreed reasonably well about the importance of training results in the organization. However, substantial diAerences were found regarding how each stakeholder group rated these results in terms of their importance for training evaluation. These diAerences were further found to correspond quite well with the unique organizational role of each stakeholder group. The results are discussed in terms of the potentials and limitations of concept mapping and pattern matching in training evaluation research, and their implications for training evaluation practice. 7 2000 Published by Elsevier Science Ltd. All rights reserved.

A Review and Synthesis of Current Research on Cross-Cultural Evaluation.

As a fairly new and emergent construct, there remain many gaps in our knowledge about how to integrate notions of culture and cultural context into evaluation theory and practice, as well as gaps in our knowledge about how to conduct and implement evaluations in immigrant and indigenous communities. In this article, the authors provide a comprehensive review of the empirical literature on evaluations conducted in cultural communities, with an emphasis on the relationship between evaluators and stakeholders in the cross-cultural program context. The analysis of the literature selected for review leads to the development of a theoretical framework describing the inter-related and multi-textual dimensions (relational, ecological, methodological, organizational and personal) that interweave throughout the evaluation, and that ultimately inform the relationship between evaluators and stakeholders in the cross-cultural program context. The article concludes with an agenda for future research.

Understanding Organizational Learning for Educational Leadership and School Reform1

The conventional ‘managed change’ paradigm as a foundation for the contemporary school reform agenda has fallen into disfavour in the face of consistently contradictory results (Louis, 1994; Lundberg, 1989). Emerging conceptions of schools of the future portray schools as ‘learning organizations’ encompassing a thriving community of learners (Fullan, 1993) or as ‘self-renewing’ organizational entities where innovation is not generated and passed down through the hierarchy but should be, formulated by individual scholar teachers and faculties under a canopy of technical and social support woven by the cadre and designed to enhance learning for all members of the organization, including themselves as cadre members. The formation of this holistic learning community promotes the interdependence that is necessary for collective growth in the midst of widely divergent individual needs. (Joyce, Wolf & Calhoun, 1993, p. 10) But the adoption of such a fundamentally different approach to change poses enormous challenges for educators. As Fullan suggests, the hardest core to crack is the learning core. ‘Changing formal structures is not the same as changing norms, habits, skills and beliefs’ (1993, p. 49).