Guoxing Yu

Assessing learning dispositions: is the Effective lifelong learning inventory valid and reliable as a measurement tool?

Background: The Effective lifelong learning inventory (ELLI) has been used as a diagnostic self assessment tool in schools and universities and other learning contexts as a means of raising an individuals awareness of their own learning dispositions and encouraging them to take responsibility for their own learning. The demand for the ELLI, in practice, has been significant, and since it is administered online, since 2004 a dataset of over 10,000 cases has accumulated. Purpose: This paper focuses on an exploration of the internal reliability, validity and stability of the seven scales of the ELLI. Some suggestions of its relevance to practice are made, though this is not the focus of the paper and is discussed elsewhere. Programme description: The inventory is available to formal learning organisations, and is administered by teachers who have been trained in its application, drawing on research and best practice through an evaluated programme. Sample: The total sample in this study is 10,496 individuals from 122 institutions and 413 classrooms. Design and methods: An exploratory factor analysis and scale reliability computations were undertaken and a ‘sensitivity’ study based on a subgroup of the original sample, collected in 2002 is reported. An Analysis of Variance study is repeated, comparing means between age ranges. Results: The study demonstrates again that the scales remain stable and continue to reach acceptable reliability levels in the new sample, in five age ranges and that the mean score on all positive learning dispositions reduces significantly until the 16–19 age range. Conclusions: The scales demonstrate a significant degree of stability, reliability and internal consistency over time. The constructs measured are thus useful for research, and institutional self-evaluation, and can be used, in practice, with some confidence. There remains a significant ongoing research agenda relating to the plasticity of the construct of learning dispositions and the ‘positioning’ of dispositions within an ecology of learning.

Exploring school effects across southern and eastern African school systems and in Tanzania

The Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) II data are analysed, using multilevel modelling techniques, to explore the key issues underlying the development of school effectiveness models. Differences between schools in Grade 6 pupils’ reading and mathematics achievements are examined and the percentage of variance in pupil outcomes attributable to school and country levels is estimated before and after adjusting for various factors. A wide range of explanatory factors has been investigated, such as pupil gender, age, socio‐economic status, homework, absence, grade repetition, school location, resources, leadership and community involvement. Only a minority of these factors was found to be statistically significantly related to pupil achievements. Tanzanian educational stakeholder views on the implications of the SACMEQ II analysis findings are also reported. In conclusion the findings suggest that a contextualised approach incorporating stakeholders’ views is needed to build understanding of school effectiveness in low‐income countries.

On the supranational spell of PISA in policy

Abstract Background: PISA results appear to have a large impact upon government policy. The phenomenon is growing, with more countries taking part in PISA testing and politicians pointing to PISA results as reasons for their reforms. Purpose: The aims of this research were to depict the policy reactions to PISA across a number of jurisdictions, to see whether they exhibited similar patterns and whether the same reforms were evident. Sources of evidence: We investigated policy and media reactions to the 2009 and 2012 PISA results in six cases: Canada, China (Shanghai), England, France, Norway and Switzerland. Cases were selected to contrast high-performing jurisdictions (Canada, China) with average performers (England, France, Norway and Switzerland). Countries that had already been well reported on in the literature were excluded (Finland, Germany). Design and methods: Policy documents, media reports and academic articles in English, French, Mandarin and Norwegian relating to each of the cases were critically evaluated. Results: A policy reaction of ‘scandalisation’ was evident in four of the six cases; a technique used to motivate change. Five of the six cases showed ‘standards-based reforms’ and two had reforms in line with the ‘ideal-governance’ model. However, these are categorisations: the actual reforms had significant differences across countries. There are chronological problems with the notion that PISA results were causal with regard to policy in some instances. Countries with similar PISA results responded with different policies, reflecting their differing cultural and historical education system trajectories. Conclusions: The connection between PISA results and policy is not always obvious. The supranational spell of PISA in policy is in the way that PISA results are used as a magic wand in political rhetoric, as though they conjure particular policy choices. This serves as a distraction from the ideological basis for reforms. The same PISA results could motivate a range of different policy solutions.

The Use of Summarization Tasks: Some Lexical and Conceptual Analyses

This article reports the lexical diversity of summaries written by experts and test takers in an empirical study and then interrogates the (in)congruity between the conceptualisations of “summary” and “summarize” in the literature of educational research and the operationalization of summarization tasks in three international English language tests in relation to their task directions and assessment criteria. These analyses show that summarization is not a uniform construct; rather, it is multidimensional in terms of its purposes, functions, and practices in real-life contexts, and as a consequence not particularly well operationalized in language tests and so in need of precise definitions for specific assessment contexts. This conclusion has implications for designing summarization tasks and assessing task performance. First, clear and transparent task directions are essential to ensure that test takers and evaluators share a common understanding of a test task; test takers need to be instructed what kind of summary they are expected to produce, particularly what information to include and to exclude. Second, because summarization may well be a unique type of writing process, it is important to employ parameters different from and additional to those for independent composition writing in order to measure the quality of a summary effectively.

Exploring the Relationships Between Independent Listening and Listening-Reading-Writing Tasks in Chinese Language Testing: Toward a Better Understanding of the Construct Underlying Integrated Writing Tasks

ABSTRACT Integrated assessment tasks have been increasingly used in language tests, but the underlying constructs of integrated tasks remain elusive. This study aimed to improve understanding of the construct of integrated writing tasks in Chinese Language examinations in Hong Kong by looking at the language competences measured in the Listening-Reading-Writing Task and how they relate to the outcome of the Independent Listening Task. The performance of 226 native Chinese Secondary Five students on both tasks were subject to correlation analysis, joint factor analysis, and regression analysis. It was found that the students’ performance in the Independent Listening Task and the Listening-Reading-Writing Task was statistically significantly correlated, but the two tasks did not seem to have common factors as shown in the joint factor analysis. The indicators of elaboration, evaluation, and creation in the Independent Listening Task were significantly correlated with multiple indicators in the Listening-Reading-Writing Task, and evaluation and creation together explained 8.9% of the variance in the total score of the Listening-Reading-Writing Task. The findings support the framework (i.e., the “four pillars” of integrated writing competence) applied in public examinations in Hong Kong. They also imply that the two types of writing tasks are complementary in the assessment of Chinese Language competence.

Test-Takers’ Cognitive Processes during a Listen-To-Summarize Cloze Task

Listen-to-summarize cloze (LSC) tasks are not rare in high-stakes language tests; however, we know little about what test-takers do during the tasks. In our study, we examined 16 students’ think-aloud protocols (TAPs) which were recorded while they were completing two LSC tasks. The analysis of the 16 TAPs indicated iterative cognitive processes of LSC task completion. Simultaneously, the different gap types in the LSC tasks required different levels of cognitive processes. We also discussed the possible cognitive depth to which LSC tasks can target and the challenges in designing appropriate gap types to assess different levels of cognitive processes.

The Implementation of EAP Instruction in a Local University in China

This chapter provides an overview of the key aspects of the implementation of an EAP curriculum, from the initial analysis of needs for the curriculum to the ongoing evaluation of the curriculum for further development. The demands of the course from teachers’ and students’ perspectives, and the development of course syllabus and teaching materials fit for the local context, are some of the key aspects that we considered during the process of implementing the EAP course. Our experience in implementing and evaluating the EAP course clearly evidenced the vital importance for curriculum innovators to take into consideration the global and national trends in EAP instruction, the local context, and students’ and teachers’ perspectives. The ongoing formative evaluation of the EAP curriculum provides further support and incentives for curriculum development within the context of ever-changing students’ and teachers’ needs and capacity.

Computer-Based English Language Testing in China: Present and Future

ABSTRACT In this special issue on high-stakes English language testing in China, the two articles on computer-based testing (Jin & Yan; He & Min) highlight a number of consistent, ongoing challenges and concerns in the development and implementation of the nationwide IB-CET (Internet Based College English Test) and institutional computer-adaptive English tests, respectively: conceptualizing the construct of computer-based language testing, ensuring fairness for test takers with differing levels of computer literacy, and achieving comparability between tests or tasks of different delivery modes. In this article, we provide an overview of the research studies on computer-based English language testing conducted by Chinese scholars and published in major Chinese academic journals in recent decades, aiming to identify the research topics, gaps, and agendas that could have implications beyond Chinese contexts in promoting better use of computer technologies in and for English language testing.

Signposts to literacy for sustainable development. Hamburg, Germany: UNESCO Institute for Lifelong Learning

This edited volume is the main product of an ambitious EU-funded research project examining the current situation of, and prospects for, large post-war housing estates in Europe. Research was conducted in 10 countries (Britain, France, Germany, Hungary, Italy, the Netherlands, Poland, Slovenia, Spain, Sweden), in 16 cities, and on 29 housing estates. Field research followed a common framework to facilitate analytical comparisons, and a multinational team wrote each chapter with cross-national examples. The book is organized into three parts. The first part presents basic information on the history of the estates; their physical characteristics and the changing demographic and socio-cultural backgrounds of residents; the social theory underpinning the research; and the almost universal trend toward privatization. The second part of the book questions the main assumptions underlying policies for the large housing estates and examines key ideas, such as social cohesion, social mixing, displacement, gentrification, and residualization. In the book’s third section, which covers specific policy approaches and their effectiveness, the contributors discuss demolitions, partnerships, citizen participation, employment assistance, programs addressing youth delinquency, and women’s needs, and the more systematic application of knowledge to problem-solving on the large estates. Contributors to the volume take the position that the decline of large public housing estates in Europe has not been universal, and that where decline has taken place, it has not been inevitable. They do not think demolition is the best option for most estates, and suggest that effective remedies will require the inclusive participation of many different kinds of stakeholders, including residents, and a sophisticated understanding of the new relationships between global economic structures and local housing markets. Several chapters stand out. Alan Murie and associates (Chapter 5) describe Europe’s varied experience with privatization: new non-profit and for-profit management organizations, sales to existing tenants, and the stock transfer of public housing from the public sector to housing associations of various types. The reasons for privatization vary considerably. In the UK, the Right-to-Buy schemes initiated in the 1980s reflected the ideological position of the Thatcher government to broaden property ownership. In Eastern Europe in the 1990s, time-limited sales at dramatically discounted prices offered local authorities a mechanism for off-loading their economic burdens as they embraced the market. Other places have been more cautious about privatization. The Netherlands, for instance, has used limited privatization to regenerate declining public housing estates and to achieve tenure mixing. Bråmo and Andersson’s analysis of who moves out of Swedish estates (Chapter 9), and van Beckhoven and associates’ discussion of participatory approaches in Spain and the Netherlands (Chapter 12), are particularly strong. As the Swedish case illustrates, the challenge is to reduce middle-class “leakage” from the estates by creating more options for these households within the

Language Testing and Assessment in Applied Linguistics: identifying reciprocity in applied linguistic research

This seminar aims to build on the growing presence of assessment and testing concerns with applied linguistics and to explore ways in which language assessment can contribute theoretical insights to second language acquisition research. The seminar provides an opportunity to challenge current thinking concerning language testing and assessment in applied linguistics from both sociocultural and psycholinguistic perspectives. A key aim is to contribute to a research agenda through dialogue between different communities in sub-fields within applied linguistics and language assessment.