A. McConney

Getting to the Bottom Line: A Method for Synthesizing Findings Within Mixed-method Program Evaluations

Evaluators who are concerned more with pragmatics than with competing epistemologies have brought multi- and mixed-method evaluations into common practice. Program evaluators commonly use multiple methods and mixed data to capture both the breadth and depth of information pertaining to the evaluand, and to strengthen the validity of findings. However, multiple or mixed methods may yield incongruent results, and evaluators may find themselves reporting seemingly conflicting findings to program staff, policy makers, and other stakeholders. Our purpose is to offer a method for synthesizing findings within multi- or mixed-method evaluations to reach defensible evaluation (primarily summative) conclusions. The proposed method uses a set of criteria and analytic techniques to assess the worth of each data source or type and to establish what each says about program effect. Once on a common scale, simple math allows synthesis across data sources or types. The method should prove a useful tool for evaluators.

Focusing improvement and quality assurance: Work samples as authentic performance measures of prospective teachers' effectiveness

The authors suggest that the gap between common evaluation practice in teacher preparation and the consensus reform goal of high levels of student learning for all students is an impediment to reinventing teacher preparation, professional development, and the professionalization of teaching. In the context of Oregon’s six-year-old educational transition to a standards-based design for schools and schooling, prospective teachers in Oregon have, since 1989, been required to design, develop, and implement “teacher work samples” as credible evidence of their effectiveness in fostering student learning as a condition for recommendation for an initial teaching license. This article describes teacher work sample methodology and evidence for its reliability and validity in evaluating the performance of prospective teachers. In short, this article describes a methodology that responds to one critical part of the National Commission on Teaching and America’s Future’s challenge to reinvent teacher education and improve student learning.

School socioeconomic status and student outcomes in reading and mathematics: A comparison of Australia and Canada

Previous research has established that student outcomes are strongly associated with the socioeconomic composition of a school, also known as school socioeconomic status. Less is known, however, about the ways in which the relationship varies for different students, schools and national education systems. Here, we conduct a secondary analysis of an international dataset to examine the strength of the relationship between school socioeconomic status and achievement in math and reading for Canada and Australia. The history, economy and culture of these two countries are similar, as are many aspects of their education systems. One important difference, however, is the degree to which their education systems are marketised. Our findings show that in both countries, school socioeconomic status is strongly associated with academic achievement for all students, regardless of their individual socioeconomic status. Nevertheless, the relationship between school socioeconomic status and academic achievement is substantially stronger in Australia than in Canada. We conclude that student outcomes are more equitable in Canada than in Australia, and suggest that this may be due to differences in the ways in which the two education systems are funded and structured.

Considering ‘teacher resilience’ from critical discourse and labour process theory perspectives

This article considers the construct of ‘teacher resilience’ from critical discourse and labour process perspectives in order to cast new light on what has been traditionally viewed from a psychological perspective. In this respect, the construct of resilience is placed in the broad political landscape of teachers’ work and the labour process of teaching, within a neoliberal globalised economic paradigm. Importantly, this article argues that any conceptualisations of teacher resilience should be critically appraised and not simply ‘taken for granted’. While the concept of developing ‘teacher resilience’ as a means, for example, of addressing alarmingly high rates of early career teacher attrition may sound like a good idea, it is important to consider the way such constructs can be used to shape and potentially control teacher identity and the nature of teachers’ work.

Bridging the Gap? A Comparative, Retrospective Analysis of Science Literacy and Interest in Science for Indigenous and Non-Indigenous Australian Students.

Previous research has shown that indigenous students in Australia do not enjoy equal educational outcomes with other Australians. This secondary analysis of PISA 2006 confirmed that this continues to be the case in science literacy for secondary students. However, the analysis also revealed that indigenous Australian students held interest in science equal to that of their non‐indigenous peers, and that observed variations in science literacy performance were most strongly explained by variations in reading literacy. These findings hold important implications for teachers, teacher educators, policy‐makers, and researchers. Firstly, acknowledging and publicly valuing indigenous Australian science knowledge through rethinking school science curriculum seems an important approach to engaging indigenous students and improving their literacy in science. Secondly, appropriate professional learning for practising teachers and the incorporation of indigenous knowing in science methods training in teacher preparation seems warranted. Additionally, we offer a number of questions for further reflection and research that would benefit our understanding of ways forward in closing the science literacy gap for indigenous students. Whilst this research remains firmly situated within the Australian educational context, we at the same time believe that the findings and implications offered here hold value for science education practitioners and researchers in other countries with similar populations striving to achieve science literacy for all.

Explaining the achievement gap between Indigenous and non-Indigenous students: An analysis of PISA 2009 results for Australia and New Zealand

This study investigates the relative roles of home and school variables in accounting for achievement gaps between Indigenous and non-Indigenous students in Australia and New Zealand. Using data from the Programme for International Student Assessment [PISA] 2009, our findings show that achievement gaps between Indigenous and non-Indigenous students are associated with both home and school resources, not only in terms of unequal allocations but also in relation to differences in the rates at which home and school affordances are converted into positive educational outcomes. In both countries, home resources accounted for more of the achievement gap than differences in schooling resources. However, the achievement gap between Indigenous and non-Indigenous students is substantially larger in Australia than in New Zealand, apparently related to greater inequity in the allocation of school resources. We suggest that education policymakers in Australia ensure a more equitable allocation of school resources between Indigenous students and their non-Indigenous peers.

Do zoo visitors come to learn? An internationally comparative, mixed-methods study

Zoo visitors go to see animals, but are they there to learn? This mixed-methods study examines visitor learning from both zoos’ and visitors’ perspectives using qualitative and quantitative data. Five hundred and forty zoo visitor interviews from nine case studies provide insight into visitor intentions, which indicate that the majority of visitors (72%) arrive at zoos with a learning agenda. Over 170 zoos across 48 countries also report, via an online questionnaire, that the majority of their visitors come to learn. In contrast, however, 28 face-to-face zoo education staff interviewed at the nine case study sites suggests a different conclusion. The study also indicates that zoos appear to determine visitors’ intentions through a number of methods, but are most heavily reliant on informal measures with only 15% of zoos using both formal and informal processes. Juxtaposing these findings suggests that zoos’ reliance on informal measures of visitor intentions appears to provide them with a less than accurate picture of their visitors’ learning agendas.

Is 'Teach for All' knocking on your door?

Abstract Over the past few decades there has been a rapid expansion in alternative ‘fast track’ routes for teacher preparation. Among the most aggressive of these are Teach for All (TFA) schemes characterized not only by their ultra fast entry to teaching (6 - 7 week course) but also by their underlying philosophy that the so called ‘crisis’ in poor rural and urban schools can be solved by attracting the ‘best and brightest’ university graduates for a two year appointment in ‘difficult to staff’ schools. With its missionary zeal TFA is heralded by some as one way to solve socio- -educational problems that governments cannot. Others condemn such schemes as not only patronizing, but also as part of an ideologically driven and deliberate neoliberal attack on public education, teachers, teacher professionalism and working class or ‘other’ communities. Recently Teach for All came knocking on New Zealand’s door. Concerned about the possible implications of this for the teaching profession and education more generally, the New Zealand Post Primary Teachers Association (PPTA) Te Wehengarua commissioned a review of the international literature on TFA schemes. This paper synthesizes some of the key findings of this review with particular focus on TFA’s marketing strategies and the connections TFA schemes have with so called social entrepreneurs or venture philanthropists, many of whom are actively and aggressively engaged in shaping educational reforms in line with neoliberal agendas.

Quest for Quality: Recruitment, Retention, Professional Development, and Performance Evaluation of Teachers and Principals in Baltimore City's Public Schools.

This article rests in the context of the 2001 evaluation of Baltimore City Public School System’s (BCPSS) 5-year-old reform effort, conducted by Westat and its collaborators.2 In that context, the article describes findings on key human resource issues for the BCPSS. Briefly, the part of the evaluation on which this article is based was charged with appraising the success of efforts to improve the quality, stability, and effectiveness of the BCPSS instructional workforce (teachers and principals). The overall design of the evaluation, as well as findings in other key areas of the school system reform effort (e.g., governance and management, instruc-tional interventions, achievement outcomes) are described in the other articles that comprise this special issue. Four areas of the human resource effort are examined for both teachers and principals in the Baltimore City Public School System (BCPSS): (a) recruitment policies and practices, (b) mentoring and other retention strategies, (c) professional development, and (d) performance evaluation. Four methods of data gathering were used: one quantitative (survey), and three qualitative (document review, key informant interview, and focus group interview). These have been described elsewhere in this special issue, and will not be further detailed here.

Investigating teacher presence in courses using synchronous videoconferencing

Abstract This research examines teacher presence in high school distance courses that are delivered by synchronous videoconference. In rural and remote areas, many school districts are using videoconferencing as way to reach dispersed students. This collective case study uses mixed methods to unpack the notion of presence from the perspective of teachers and their students. This study reports four key findings which have implications for building presence in a videoconference course: teachers’ confidence and experience aligned with higher presence; teaching videoconference and face-to-face classes simultaneously led to challenges with developing presence; immediacy behaviors correlated with higher presence; and, students’ learning preference related to perceived teacher presence. These findings confirm many of the issues raised in the literature about technology integration but also contribute new perspectives on teaching presence in a videoconference.