Eileen Piggot-Irvine

Establishing criteria for effective professional development and use in evaluating an action research based programme

A dialectical, or mutually informing and influencing, relationship exists between research, programmes for development and improved teaching and learning. Among a raft of other attributes, current perceptions of effective professional development (summarised in the paper) point to deep, collaborative, active and ongoing features as important. Such development, in turn, should result in improvements to teaching and learning, and both these outcomes, and the professional development programme itself, should be able to be evaluated. A review of an action research based leadership programme against the features of effective professional development is used as an example of evaluation. The evaluation indicates a strong and robust approach, but also highlights limitations of independent evaluator depth of knowledge of such a programme and the need to encourage a move towards more rigorous data collection showing evidence of change in evaluation.

Incorporating action research in school senior management training

Outlines how an action research approach can be used for management training and education. The emphasis on developing competence in problem‐solving processes, using action research, is supported by the literature and this is illustrated by several examples of institutional projects and award‐bearing programmes. Details how the authors’ interpretation of action research, in the school management training context, can both challenge and develop educational leaders.

Induction of newly qualified teachers in New Zealand

In this research, commissioned by the New Zealand Teachers Council, the quality of induction of provisionally registered teachers (PRT) (newly qualified) was examined utilising qualitative ‘success case studies’ within early childhood, primary, secondary, and indigenous Māori medium settings. The establishment of criteria for effective induction (from the literature and previous research) guided the identification of 20 ‘success’ sites across the sectors. In-depth data collection of each case was conducted via focus groups, one-to-one interviews and documentary analysis. The findings of the research highlighted exemplary induction practices across the sectors, with the most important associated with PRTs having access to a community, or ‘family’, of support during their induction. An interesting finding, which contrasted with previous research, was that PRTs in the secondary sector had levels of satisfaction with their role that were as high as those in other sectors. The key limitation to effectiveness was linked to lack of time for discussions and observations of the PRTs practice.

The Application of a Multiphase Triangulation Approach to Mixed Methods The Research of an Aspiring School Principal Development Program

Mixed methods research has emerged as a credible alternative to unitary research approaches. The authors show how a combination of a triangulation convergence model with a triangulation multilevel model was used to research an aspiring school principal development pilot program. The multilevel model is used to show the national and regional levels in the research design, whereas the convergence model illustrates how data were analyzed through a series of phases so that both formative and summative findings could emerge. The authors illustrate through some of the findings how their design contributed to adjustments to the program and to their findings. The challenges of undertaking such a mixed methods design reveal a complexity not always evident in abstract conceptual models.

Conceptualizing indicator domains for evaluating action research

The focus of this paper is to share thinking about meta-level evaluation of action research (AR), and to introduce indicator domains for assessing and measuring inputs, outputs and outcomes. Meta-level and multi-site evaluation has been rare in AR beyond project implementation and participant satisfaction. The paper is the first of several associated with the Evaluative Study of Action Research (ESAR) in which we wish to establish the ways that espoused intents articulated in projects are realized and why certain approaches are adopted and seen to be effective. We seek to increase understanding of outcomes and impact of AR. Description is provided of multiple issues of complexity associated with establishing evaluative criteria and indicators categorized according to inputs, process, outputs, outcomes and impact. We explore theory associated with definition and practice of evaluation prior to presentation of the indicators. We think the time is ripe for deeper examination of AR practice and welcome the associated conversation and critique of our proposed indicators.

Principal Development: Self-directed Project Efficacy

The inclusion of self-directed projects as an element within a New Zealand principal development programme was designed to reflect increasing support internationally for such a context-specific ‘inquiry’ approach. The results reported in this article suggest that considerable clarity is required for such projects if they are to realize the transformative potential that is touted. A mixed methods design employing electronic questionnaires, observation, interviews, focus groups and documentary analysis was used for the evaluation of the effectiveness of projects in the development programme. Data collected early in the programme indicated considerable concern about administration and clarity of expectations. Although the results revealed increasing appreciation of, and engagement with, the leadership projects through to the presentation phase near the end of the programme, this positivity subsided somewhat later with stakeholders providing a balanced view of recognition of worth of the projects alongside reiteration of concern over poor administration and clarity of expectations. The results suggest that the rationale, expectations, quality criteria, scope and presentation requirements must be explicitly and continuously outlined throughout projects. Further, in terms of the evaluation approach, it is suggested that future nationally funded evaluations of such programmes should be extended to include a longitudinal, deeper, impact study of such elements as projects on teaching and learning, and transformed practice of participants.

Aspiring principal development programme evaluation in New Zealand

Purpose – The New Zealand Ministry of Education has constructed a wide‐ranging “Professional Development Plan” providing a four‐stage national pathway for progression to principalship; the first stage has been the conduction of the National Aspiring Principals Pilot (NAPP) programme in five regional locations. The purpose of this paper is to outline the evaluation of the programme.Design/methodology/approach – A mixed method approach for evaluation was employed where qualitative and quantitative data were collected almost simultaneously, and compared and contrasted.Findings – Key findings indicated overall sound programme delivery, curriculum coherence, high relevancy to stakeholders and good rates of principal appointments. Greater emphasis is placed on distinctive findings that have not been reported elsewhere. These included: primary‐sector participants rating course facilitation, online learning, and “relevancy of the course for principal development” more positively than their secondary counterparts;...

One School's Approach to Overcoming Resistance and Improving Appraisal: Organizational Learning in Action.

This article reports on the action research (AR) approach adopted by one New Zealand (NZ) primary school to review and improve its appraisal system. Historically the staff had demonstrated considerable negativity towards appraisal. The classic reconnaissance, implementation and evaluation phases of AR were adopted by the case study school as a rigorous approach for improvement. A particular emphasis in this article is on the outcomes of the evaluation phase of the AR where the school lead team and an independent evaluator conducted a review of the effectiveness of the improvements via survey, a focus group interview and documentary analysis. Strong positive feedback was provided on the effectiveness of the lead team AR activity, as well as the outcomes associated with the appraisal system. Respondents’ suggestions for further improvement were combined with those linked to documentary analysis to identify areas for further enhancement of rigour in the system. These suggested enhancements have led to a subsequent iteration of improvement in the school that strongly locates the latter as a learning organization (LO), which demonstrates the openness and commitment to development also associated with organizational learning (OL).

Confronting Evaluation Blindness: Evidence of Impact of Action Science-Based Feedback

This article outlines an attempt at realizing mutuality between evaluation and action science in feedback within an aspiring principal leadership development programme in New Zealand. The evaluation was specifically designed to overcome the formative—summative divide through ongoing, often immediate, dialogue using an action science approach. The approach initially focused on detecting and correcting any errors or gaps between ‘‘espoused theory’’ (intentions, goals) and ‘‘theory-in-use’’ (programme implementation) to enable continuous improvements. There is some evidence that the approach may have strengthened the programme. Specifically, in a case study region that had the most intensive feedback, outcomes included lowering of defensiveness and increasing demand for feedback with some facilitators; increased aspirant ratings for the programme; enhanced aspirant confidence in applying for principalships; a greater number of aspirants gaining principalships; and higher response rates to questionnaires. Despite the traditional limitations of time and threat linked to the feedback approach, the multiple positive outcomes noted that might be attributable to the feedback signal that the causal links between such an approach and improved programme outcomes is, at the very least, worthy of further investigation.

Action Research Networks: Role and Purpose in the Evaluation of Research Outcomes and Impacts.

The focus of this paper is to share thinking about networks in action research (AR) and to consider their role, purpose, and how networks’ outcomes and impacts might be evaluated. Networks are often a by-product of AR projects, yet research focused on the network itself as part of a project is rare. The paper is one of several associated with the Evaluative Study of Action Research (ESAR) in which we wish to understand how projected outcomes and impacts are realised and why certain approaches are chosen and seen to be effective. Description is provided of the development and function of networks and their connection to, and within, AR. We then explore the first of a series of networks in the ESAR, the core team network, and undertake a formative evaluation of the functioning of this network. The paper concludes with a look ahead to the evaluation of each of the various levels of networks anticipated within the ESAR.