Micheline Chalhoub-Deville

Second Language Interaction: Current Perspectives and Future Trends.

This article considers how the nature of interaction may best be represented in the second language (L2) construct. The starting point is Bachman’s model of communicative language ability which, it is argued, incorporates interaction from an individual-focused cognitive perspective. The alternative view advocated here is that individual ability and contextual facets interact in ways that change them both. Thus, ‘an ability - in language user - in context’ view is to be preferred over Bachman’s ‘ability - in language user’ representation. Acceptance of this new approach entails a local, context-bound view of language ability, which is difficult to reconcile with the tester’s need for score generalizability. The way forward is to recognize that, while some contexts activate stable ability features, others produce more variable performance from learners. Thus, the focus of both theory formulation and empirical research should be on how to account for inconsistent performance in particular contexts from a social interactional perspective.

Computer-Adaptive Testing in Second Language Contexts

The widespread accessibility to large, networked computer labs at educational sites and commercial testing centers, coupled with fast-paced advances in both computer technology and measurement theory, along with the availability of off-the-shelf software for test delivery, all help to make the computerized assessment of individuals more efficient and accurate than assessment using traditional paper-and-pencil (P&P) tests. Computer adaptive testing (CAT) is a form of computerized assessment that has achieved a strong foothold in licensure and certification testing and is finding greater application in many other areas as well, including education. A CAT differs from a straightforward, linear test in that an item(s) is selected for each test taker based on his/her performance on previous items. As such, assessment is tailored online to accommodate the test takers estimated ability and confront the examinee with items that best measure that ability.

Theoretical models, assessment frameworks and test construction:

The article reviews the usefulness of several models of proficiency that have influenced second language testing in the last two decades. The review indicates that several factors contribute to the lack of congruence between models and test construction, and makes a case for distinguishing between theoretical models, which attempt to represent the proficiency construct in various contexts and oper ational assessment frameworks, which depict the construct in particular contexts. Additionally, the article underscores the significance of an empirical, contex tualized and structured approach to the development of assessment frameworks.

Validation in language testing

Developing rating scales for the Cambridge assessment of spoken English, M. Milanovic et al validation of a new holistic rating scale using Rash multi-faceted analysis, B. Tyndall and D. Mann Kenyon hypothesis testing in construct validation, S. Cushing Weigle and B. Lynch an investigation into the validity of written indicators of second language proficiency, A. Cumming and D. Mellow multiple-choice summary, A. Huhta and E. Randell using the information curve to assess the efficiency of computerized adaptive language testing, M. Laurier comparing the scaling of speaking tasks by language teachers and by the ACTFL guidelines, C. Stansfield and D. Mann Kenyon a communicative test in analysis - strategies in reading authentic texts, I. Wijgh what makes an ESP reading test appropriate for its candidates? C. Clapham examining washback - the Sri Lankan impact study, D. Wall and C. Alderson the role of the segmental dictionary in professional validation, A. Davies language testing courses, K. Bailey and J.D. Brown.

Accountability-assessment under No Child Left Behind: Agenda, practice, and future

The push for educational reform seems to be ubiquitous. For example, educational reform efforts have been documented in Australia (Brindley, 2001), Canada (Klinger, DeLuca, & Miller, 2008), Israel (Shohamy, 2001), the UK (Broadfoot, 1996), Germany (Rupp & Vock, 2007), and the USA (Wixson et al., 2003; Chalhoub-Deville, 2009a). Political rhetoric aimed at reform often seeks to underscore that education lies at the heart of economic development, international competitiveness, and social harmony (Haertel & Herman, 2005). The demand for accountability, however, which requires states to monitor and report achievement of all students, including English language learners (ELLs), and which entails a punitive system of sanctions for schools and educators based on student performance (defined in large part by test scores) is unprecedented, at least in the USA. Increasingly educational reform efforts in the USA have mandated the use of standardized tests and expanded a system of rewards/sanctions associated with test results. As Linn (2005) observes, ‘[i]n the 1960s ... attempts to use tests for school accountability generally failed to be approved ... In the following two decades, states moved toward statewide testing programs that had higher stakes initially for students and later for schools and educators’ (p. 2). As we stated in a previous publication,

Modified scoring, traditional item analysis and Sato's caution index used to investigate the reading recall protocol

The written recall protocol is increasingly being used in second language reading research as a measure of comprehension. Although the recall protocol is an essay-like instrument, the total score derived is based entirely on summing the discrete propositions correctly recalled. Consequently, item and reliability analyses comparable to those run on multiple-choice tests can and should be performed on recall protocols and on other integrative measures. The purpose of this study is to demonstrate how modified scoring, item and internal consistency analyses, along with Satos caution indices, can be used to evaluate the quality of the recall protocol as a reading comprehen sion measure. Issues concerning the assumptions underlying classical local independence are discussed with regard to the reading recall protocol and other integrative measures. Results indicate that the procedures can be applied and can yield interpretable results. These results need to be replicated using other texts and other weighting systems. Only when recall protocols are routinely subjected to item and reliability analyses comparable to those performed on other measures, can the instrument be considered a viable alternative.

Validity theory: Reform policies, accountability testing, and consequences

Educational policies such as Race to the Top in the USA affirm a central role for testing systems in government-driven reform efforts. Such reform policies are often referred to as the global education reform movement (GERM). Changes observed with the GERM style of testing demand socially engaged validity theories that include consequential research. The article revisits the Standards and Kane’s interpretive argument (IA) and argues that the role envisioned for consequences remains impoverished. Guided by theory of action, the article presents a validity framework, which targets policy-driven assessments and incorporates a social role for consequences. The framework proposes a coherent system that makes explicit the interconnections among policy ambitions, testing functions, and the levels/sectors that are affected. The article calls for integrating consequences into technical quality documentation, demands a more realistic delineation of stakeholders and their roles, and compels engagement in policy research.

Principles of Quantile Regression and an Application.

Newer statistical procedures are typically introduced to help address the limitations of those already in practice or to deal with emerging research needs. Quantile regression (QR) is introduced in this paper as a relatively new methodology, which is intended to overcome some of the limitations of least squares mean regression (LMR). QR is more appropriate when assumptions of normality and homoscedasticity are violated. Also QR has been recommended as a good alternative when the research literature suggests that explorations of the relationship between variables need to move from a focus on average performance, that is, the central tendency, to exploring various locations along the entire distribution. Although QR has long been used in other fields, it has only recently gained popularity in educational statistics. For example, in the ongoing push for accountability and the need to document student improvement, the calculation of student growth percentiles (SGP) utilizes QR to document the amount of growth a student has made. Despite its proven advantages and its utility, QR has not been utilized in areas such as language testing research. This paper seeks to introduce the field to basic QR concepts, procedures, and interpretations. Researchers familiar with LMR will find the comparisons made between the two methodologies helpful to anchor the new information. Finally, an application with real data is employed to demonstrate the various analyses (the code is also appended) and to explicate the interpretations of results.

THE INTERSECTION OF TEST IMPACT, VALIDATION, AND EDUCATIONAL REFORM POLICY

The article addresses the intersection of policy, validity, and impact within the context of educational reform in U.S. schools, looking in particular at the No Child Left Behind (NCLB) Act (2001). The discussion makes a case that it is important to reconsider the established views regarding the responsibility of test developers and users in investigating impact given the conflated roles of developers and users under NCLB. The article also introduces the concept of social impact analysis (SIA) to argue for an expansion of the traditional conceptualization of impact research. SIA promotes a proactive rather than a reactive approach to impact, in order to inform policy formulation upfront.

Differential and long-term language impact on math:

Literature provides consistent evidence that there is a strong relationship between language proficiency and math achievement. However, research results show conflicts supporting either an increasing or a decreasing longitudinal relationship between the two. This study explored a longitudinal data and adopted quantile regression analyses to overcome several limitations in past research. The goal of the study is to detect more accurate and richer information on the long-term relationship between language and math, taking into consideration the socioeconomic status, gender, and ethnicity background at the same time. Results confirmed a persistent relationship between math achievement and all the factors explored. More importantly, it revealed that the strength of the relationship between language and math differed for students with various abilities both within and across grades. Model comparison suggests that language demand contributes to the achievement gap between ELLs and non-ELLs in math. There also seems to be a disadvantage for the geographically isolated group in academic achievement. Interpretation and implications for teaching and assessment are discussed.