Geoff Brindley

Outcomes-based assessment in practice: some examples and emerging insights

The implementation of outcomes-based assessment and reporting systems in educational programs has been accompanied by a range of political and technical problems, including tensions between the summative and formative purposes of assessment and doubts surrounding the validity and reliability of teacher-constructed assessment tasks. This article examines ways in which these problems have been manifested and addressed, using two recent examples from school and adult immigrant education in Australia. The first example concerns a recent controversy surrounding the use of national literacy benchmarks for primary school learners. Analysis of the issues suggests that some learner groups may be disadvantaged by the practice of reporting aggregate outcomes in terms of minimum standards, but that government policy is unlikely to change as long as the accountability function of assessment remains paramount in the public eye. The second example discusses the teacher-developed assessment tasks that are used to assess the achievement of language competencies in the Australian Adult Migrant English Program (AMEP). It is argued that problems of consistency and comparability that have been identified by research can be addressed through the development of fully-piloted task banks and the provision of appropriate forms of professional development. Greater attention needs to be given to the role of the teacher if outcomes-based assessments are to provide high quality information.

Outcomes-based assessment and reporting in language learning programmes: a review of the issues:

In recent years educational authorities in many countries have introduced outcomes-based assessment and reporting systems in the form of national standards, frameworks and benchmarks of various kinds which are used both for purposes of system accountability and for assessing individual progress and achievement in language learning. However, in some cases the introduction of these systems has proved problematic, owing to a number of political, technical and practical factors. These include the difficulty of combining formative assessment with summative reporting, the differing information requirements of different audiences, concerns about the validity and reliability of outcome statements and the lack of appropriate resources to support implementation. Such problems may be able to be alleviated by closer consultation between policy-makers, administrators and practitioners, by undertaking further research into the validity and consistency of outcome statements and by strengthening the links between assessment and reporting. A major investment in teacher professional development is necessary if teachers are to be responsible for carrying out their own assessments. Ongoing research needs to be conducted into the effects of outcomes-based assessment and reporting on student learning.

Exploring task difficulty in ESL listening assessment

This article reports on an exploratory study that investigated the comparability of listening assessment tasks used to assess and report learning outcomes of adult ESL learners in Australia. The study focused on the effects of task characteristics and task conditions on learners’ performance in competency-based listening assessment tasks that require learners to demonstrate specific listening behaviours. Key variables investigated included the nature of the input and the response mode. Quantitative and qualitative analyses of test scores suggest that speech rate and item format influence task and item difficulty. However, the complexity of the interaction between text, item and response makes it difficult to isolate the effects of specific variables. Implications of these findings for assessment task validity and reliability are considered and practical consequences for assessment task design in outcomes-based systems are discussed.

Assessing Listening Abilities

Over the last two decades, research has highlighted the important role that listening plays in language acquisition (Brown and Yule 1983, Ellis, et al . 1994, Faerch and Kasper 1986, Feyten 1991, Long 1985), and listening comprehension skills have begun to receive a lot more systematic attention in language teaching classrooms. A wide range of books, articles, and materials aimed at assisting teachers to develop learners’ listening skills are now available, and a variety of comprehension-based methodologies have been proposed (see, for example, Anderson and Lynch 1988, Courchene, et al . 1992, Rost 1990; 1994, Underwood 1989). However, although many of the tasks used for teaching listening are virtually identical to those which appear in tests, assessment of listening ability has received relatively limited coverage in the language testing literature.