Michael Carey

Does a rater's familiarity with a candidate's pronunciation affect the rating in oral proficiency interviews?

This study investigated factors that could affect inter-examiner reliability in the pronunciation assessment component of speaking tests. We hypothesized that the rating of pronunciation is susceptible to variation in assessment due to the amount of exposure examiners have to nonnative English accents. An inter-rater variability analysis was conducted on the English pronunciation ratings of three test candidate interlanguages: Chinese, Korean, and Indian English. Pronunciation was rated by 99 International English Language Testing System (IELTS) examiners across five geographically dispersed test centres. The examiners had either prolonged exposure, or no, or little exposure to the interlanguage of the candidates. A significant proportion of examiners rated pronunciation higher when they had prolonged exposure, and lower when they had no, or little, exposure to the candidates’ interlanguage. The location of the test centre also had a significant effect on the pronunciation rating, independent of the familiarity variable, with a significant proportion of NNS raters scoring candidates from their home country higher than those who were not. It is recommended that interlanguage phonology familiarity should be considered in the design of speaking tests and rater training and that research is required into test centre bias and the phonological judgements and awareness of OPI raters.

Introductory Statistics Textbooks and the GAISE Recommendations

ABSTRACT The six recommendations made by the Guidelines for Assessment and Instruction in Statistics Education (GAISE) committee were first communicated in 2005 and more formally in 2010. In this article, 25 introductory statistics textbooks are examined to assess how well these textbooks have incorporated the three GAISE recommendations most relevant to implementation in textbooks (statistical literacy and thinking; use of real data; stress concepts over procedures). The implementation of another recommendation (using technology) is described but not assessed. In general, most textbooks appear to be adopting the GAISE recommendations reasonably well in both exposition and exercises. The textbooks are particularly adept at using real data, using real data well, and promoting statistical literacy. Textbooks are less adept—but still rated reasonably well, in general—at explaining concepts over procedures and promoting statistical thinking. In contrast, few textbooks have easy-usable glossaries of statistical terms to assist with understanding of statistical language and literacy development. Supplementary materials for this article are available online.

Improving the quality of assessment by using a community of practice to explore the optimal construction of assessment rubrics

Abstract A focus on quality assurance of assessment processes in tertiary education within Australia and throughout the world has resulted in a changing landscape of assessment types and grading schemes over the last decade. The use of criteria and standards-based assessment systems are now very commonplace in tertiary education. There are a variety of models now used, but typically they include a criteria sheet and a levelled rubric. An alternative to the traditional matrix-style rubric is the Continua Model of a Guide to Making Judgments (GTMJ). In this paper, we analyse available assessment models and their capacity to guide the marking, grading and moderation of student assessment tasks. We specifically address standards descriptors used to identify the quality expected at each standard. The research was undertaken through a community of practice within the School of Education at a tertiary institution where the collective goal of enhancing assessment grading tools to improve student outcomes was approached through a process of peer review. In our results section, we analyse the efficacy of an internal peer-review model as part of a community of practice and the professional learning about grading tools that occurs.

Are Assessment Exemplars Perceived to Support Self-Regulated Learning in Teacher Education?

Assessment exemplars are a tool to guide students to what is valued by assessors in a specific assessment task, in short, as examples which illustrate, typically, dimensions of quality. Often high quality exemplars are provided. We were interested in researching the perceived efficacy and impact of a variety of assessment exemplars, ranging from low to high quality, in teacher education courses at a regional university. We surveyed 72 students and found that students accessed exemplars regularly and found them useful in providing detailed guidance that went beyond the descriptions of assessment tasks found in course outlines and assessment rubrics. They valued various types of exemplars, a range of quality, and the inclusion of annotated and un-annotated versions of exemplars.

English Language Teaching in Libya After Gaddafi

This chapter discusses the impact of the political, cultural and social background of Libyan society on the current quality of English education in the country. Libya has witnessed several attempts to reform English education since the 1970s; however, these attempts were based on introducing grammar-based curricula which were designed by non-native speakers of English, few of whom were Libyans. In an effort to reform English education in the country, the Libyan Ministry of Education developed new English curricula in 2000 based on CLT principles to be used in Libyan primary and high schools in place of the previous curricula which aimed mainly to teach grammar and reading. However, Libyan English teachers have not been able to help their students of English to achieve the objectives of the new curricula because the teachers predominantly use the Grammar Translation Method (GTM) and teacher-centred pedagogy. Despite the government’s intentions to innovate with communicative curricula, the teachers do not actually use these curricula in primary and high school classrooms. The quality of English language teacher education in Libya is underdeveloped and this situation can be seen as a result of a greater issue; the Libyan government’s accreditation procedures for universities, programs and courses are not well-developed.

Preparing preservice teachers to be data literate: a Queensland case study

ABSTRACT Data driven pedagogy has been introduced into Australian schools over the last decade to improve student learning outcomes, but are teacher educators preparing preservice teachers to be data literate? We used a survey to investigate ready-to-graduate preservice teachers’ beliefs regarding their knowledge and skills in data literacy. Before entering a course on data literacy their belief responses indicated that they had basic data interpretation knowledge, but they were unsure of how to translate that knowledge into the classroom to improve student outcomes. The survey was administered again after a data literacy course, which included a five-week supervised professional experience task of observing data being used in schools to drive pedagogy. The post-course survey responses showed students increased confidence in their ability to interpret and use data, suggesting that a data literacy course linked to the professional experience task improves teachers’ sense of preparedness to use data in the classroom. Keywords: Data literacy; professional learning.