Elana Shohamy

Language policy : hidden agendas and new approaches

Policies concerning language use are increasingly tested in an age of frequent migration and cultural synthesis. With conflicting factors and changing political climates influencing the policy-makers, Elana Shohamy considers the effects that these policies have on the real people involved. Using examples from the US and UK, she shows how language policies are promoted and imposed, overtly and covertly, across different countries and in different contexts. Concluding with arguments for a more democratic and open approach to language policy and planning, the final note is one of optimism, suggesting strategies for resistance to language attrition and ways to protect the linguistic rights of groups and individuals.

Test impact revisited: washback effect over time

In countries with centralized educational systems national tests are used as primary devices through which changes in the educational system are introduced. This article examines the impact over time of two national tests which have been in operation for a number of years: one in Arabic as a second language (ASL) and one in English as a foreign language (EFL) through questionnaires, interviews and document analysis from a sample of teachers, students and language inspectors. Results showed different washback patterns for the two tests. Slight modifications in the ASL test created no effect in classroom activities, test preparation, or the status and prestige of the subject tested, yet, the inspectors expressed satisfaction and wished to continue the administration of the test as they feared that without the tests proficiency levels would drop. Slight modifications in the EFL test, on the other hand, created major impact in terms of teaching activities, time devoted for test preparation, production of new teaching material, etc. Although negative attitudes were expressed by teachers regarding the quality of the test, they would like it to continue. Inspectors believe that the test creates a meaningful change and is powerful enough to trigger changes without a need to provide training and a new curriculum. The study shows that washback varies over time, owing to many factors such as the status of the language and the uses of the test.

Democratic assessment as an alternative

The article describes the strong power of tests and the fact that tests lead to far-reaching and high-stakes decisions and consequences about individuals and groups. Further, there is evidence that tests are often introduced by those in authority as disciplinary tools, often in covert ways for the purpose of manipulating educational systems and for imposing the agendas of those in authority. Yet, such uses of tests as instruments of power violate fundamental values and principles of democratic practices. The article proposes a number of assessment strategies which are based on democratic principles so that society can guard and protect itself from such undemocratic practices. The principles include the need: • for citizens in democratic societies to play a participatory and active role and transfer and share power from elites to and with local bodies; • for those who develop powerful tools to be responsible for their consequences; • to consider voices of diverse and different groups in multicultural societies; and • to protect the rights of citizens from powerful institutions. These lead to assessment practices which are aimed at monitoring and limiting the uses of tests, especially those that have the potential to exclude and discriminate against groups and individuals. Specifically, assessment practices include the need: • to examine the uses of tests through critical language testing (CLT); • to develop assessment models that are based on shared and collaborative models; • to assume a growing responsibility for those who are engaged in test development and use; • to examine the consequences of tests; • to include different voices in assessment, especially in multicultural societies; and • for test-takers to protect and guard their rights from the authority and misuses of tests.

The validity of direct versus semi- direct oral tests

The article reports the results of a study which examined the validity of two oral language tests - one direct and one semi-direct. While concurrent validity of the two types of tests (as obtained via correlations in a number of earlier studies) was high, a variety of qualitative analyses indicated that the two tests differed in a number of aspects. Differences were found in the elicitation tasks (in the number and types of functions and topics) and in the language samples obtained (in communicative strategies and in the discourse features). Conclusions are drawn regarding the need to validate tests from multiple perspectives.

Testing methods, testing consequences: are they ethical? are they fair?

Language tests employing methods which are not fair to all test takers are unethi cal. Equally, uses of language tests which aim to exercise control and manipulate stakeholders rather than provide information regarding proficiency levels are also unethical. Ways of reducing these sources of unfairness are discussed and language testers are urged to exercise vigilance at all times to ensure that the tests they develop are democratic and fair.

The test-takers' choice: an investigation of the effect of topic on language-test performance

A fundamental issue in validating topic-based tests of language proficiency is the effect of the topic on the test takers’ performance. Topic-based test developers must ensure that test takers are neither advantaged nor disadvantaged in terms of their test results when presented with a given test topic. We have termed this threat a ‘topic effect’ and argue that this topic effect may constitute a source of construct-irrelevant variance (Messick, 1989). We contend that investigating the possibility of a topic effect is a critical step in establishing the validity of all topic-based tests. This research investigates the potential presence of a topic effect for the Canadian Academic English Language (CAEL) Assessment using the mechanism of choice. The principal research question is to determine if test-takers given a choice of topic perform significantly differently than test-takers not given a choice. ESL university applicants (n = 254) were randomly assigned to one of two conditions: no choice of topic or choice among five topics. Overall Proficiency Level, Reading, Lecture and Essay scores were compared for the two conditions. Ordinal level data were analysed using the Mann Whitney U, Chi-Square and Kolmogorov-Smirnov tests. While the scores for the choice groups were usually higher than the scores for the no-choice groups, the differences were not statistically significant. Despite the lack of significance, we felt that the scores warranted closer examination. For the topic where the difference between the choice and no-choice groups was largest, a textual analysis of the essays was undertaken to look for instances of the use of information not provided in the test. Again, no difference was found between the choice and no-choice groups. The results provide support for the validity of inferences drawn from this test. Because choice is an essential element of the research design, a second focus of the study is to explore the advantages and disadvantages of the use of choice in language testing settings from the perspective of both the tester and the test-taker. The potential value of choice as a testing feature is discussed and a call for further research is made.

Language tests as language policy tools

This paper contextualizes language tests in relation to educational and national language policies by demonstrating how these language measures may be used as mechanisms for affecting de facto language policies. This phenomenon is of special relevance given current controversies in nation states between multilingual and multicultural realities and government policies that perpetuate homogenous policies with regard to national languages. The introduction of language tests in certain languages delivers messages and ideologies about the prestige, priorities and hierarchies of certain language(s), and not others, leading to policies of suppression of diversity. Tests also influence language policy with regard to the nature of language, as derived from the criteria used for judging language quality via rating scales, guidelines and frameworks, thus leading to a view of language as standardized and homogenous. Yet language tests, when they incorporate research findings about language learning and language use, can serve as tools for creating more valid and real language policies that mediate and negotiate between ideology and practice. The discussion of these issues is supported by examples from a range of international contexts in this paper.

The Relationship between Language Testing and Second Language Acquisition, Revisited.

Abstract The paper examines the relationship between and the relevance of second language acquisition (SLA) and language testing (LT). Based on three dimensions of potential contributions of LT to SLA [(1) defining the construct of language ability; (2) applying LT findings to test SLA hypotheses; and (3) providing SLA researchers with quality criteria for tests and tasks] and three dimensions of potential contribution of SLA to LT [(1) identifying language components for elicitation and criteria assessment; (2) proposing tasks for assessing language; and (3) informing language testers about differences and accommodating these differences], this paper examines the interfaces of the two fields based on articles published in recent issues of the journals “Language Testing” and “Studies in Second Language Acquisition”. The relevance of LT to SLA is examined based on written interviews with leading scholars in SLA who were asked about the relevance of LT to their work. The results indicate very limited interfaces between the two fields as well as limited relevance of LT to SLA. The conclusions and implications discuss to the potential need of LT to broaden its focus and scope by addressing broader views of language learning and language processing such as: viewing language in its complexities and dynamics; involving the learners and test takers; marketing better LT theories to those out of the field; expanding the context beyond psychometrics; expanding the types of instruments used beyond tests; addressing educational issues; and working towards relevance.

Second Language Assessment

Second language assessment is concerned with procedures and techniques for measuring second language knowledge. Thus, the main focus of the field has been the definition of second language knowledge and the design of assessment procedures that match such definitions. While the field is generally known as ‘language testing’ it is also referred to as language assessment due to the recent trend to use multiple assessment procedures, and not just tests. Language testing/assessment is a dynamic field that holds annual conferences, including the Language Testing Research Colloquium (LTRC), publishes a journal, Language Testing, and many of whose practitioners belong to an international organization, the International Language Testing Association (ILTA).

Reflections on Research Guidelines, Categories, and Responsibility

N The guidelines for quantitative and qualitative research, published in TESOL Quarterly, are comprehensive, detailed, informative, and broad in scope. They present a variety of research designs and approaches, especially regarding qualitative research. They are useful for planning research, writing up research articles and reports, and reading research critically. At the same time, however, the guidelines raise a number of questions: Does the field need strict boundaries for research designs? Are the categories so distinct from one another? Can qualitative research designs be differentiated from one another in such ways? For example, is narrative research so different from critical ethnography, or conversational analysis? Can researchers not address these qualitative designs quantitatively as well? Do these categories represent research designs, research traditions, research paradigms, or methods of data analysis? Related to these questions is whether researchers need such detailed guidelines. What message do these guidelines send them? The guidelines are based on the assumption that research is fixed, finite, and follows a clear order. Yet, in an era when research is opening up to a variety of options, well beyond those included in the guidelines, such a prescription may be perceived as an imposition or a dogma of how research should be done. What is the right way to do research, anyway? When qualitative research emerged not long ago, positivists rejected qualitative methods because they felt such methods could not be trusted-that it was not real research. (Some journals still do not publish qualitative studies.) Researchers doing qualitative research rejected that criticism, however, claiming that quantitative research is restrictive and prescriptive, and it cannot address deeper insights and understandings of language phenomena. As the guidelines show, qualitative approaches are nowadays very widely accepted. Thus, TESOL researchers must be cautious that the guidelines do not restrict innovations in research designs by imposing