Bart Deygers

Determining the Scoring Validity of a Co-Constructed CEFR-Based Rating Scale.

Considering scoring validity as encompassing both reliable rating scale use and valid descriptor interpretation, this study reports on the validation of a CEFR-based scale that was co-constructed and used by novice raters. The research questions this paper wishes to answer are (a) whether it is possible to construct a CEFR-based rating scale with novice raters that yields reliable ratings and (b) allows for a uniform interpretation of the descriptors. Additionally, this study focuses on the question whether co-constructing a rating scale with novice raters helps to stimulate a shared interpretation of the descriptors over time. For this study, six novice raters employed a CEFR-based scale that had been co-constructed by themselves and 14 peers to rate 200 spoken and written performances in a missing data design. The quantitative data were analysed using item response theory, classical test theory and principal component analysis. The focus group data, collected after the rating process, were transcribed and coded using both a priori and inductive coding. The results indicate that novice raters can reliably use the CEFR-based rating scale, but that the interpretations of the descriptors, in spite of training and co-construction, are not as homogeneous as the inter-rater reliability would suggest.

University entrance language tests: A matter of justice:

University entrance language tests are often administered under the assumption that even if language proficiency does not determine academic success, a certain proficiency level is still required. Nevertheless, little research has focused on how well L2 students cope with the linguistic demands of their studies in the first months after passing an entrance test. Even fewer studies have taken a longitudinal perspective. Set in Flanders, Belgium, this study examines the opinions and experiences of 24 university staff members and 31 international L2 students, of whom 20 were tracked longitudinally. Attention is also given to test/retest results, academic score sheets, and class recordings. To investigate the validity of inferences made on the basis of L2 students’ scores, Kane’s (2013) Interpretation/Use Argument approach is adopted, and principles from political philosophy are applied to investigate whether a policy that discriminates among students based on language test results can be considered just. It is concluded that the receptive language requirements of university studies exceed the expected B2 level and that the Flemish entrance tests include language tasks that are of little importance for first-year students. Furthermore, some of the students who failed the entrance test actually managed quite well in their studies – a result that entails broad implications concerning validation and justice even outside the study’s localized setting.

One Framework to Unite Them All? Use of the CEFR in European University Entrance Policies

ABSTRACT Fifteen years after its publication, the Common European Framework of Reference for Languages is a commonly used document in language tests and policies across Europe. This article considers the CEFR’s impact on university entrance language tests and policies that are used to regulate the entrance of international L2 students who wish to study in a national language of the host country. Using a qualitative approach, this study aims to (a) outline and compare the target language demands toward L2 students entering European universities and to (b) determine the impact of the CEFR on European university entrance policies, tests, and testers. This article offers an overview of the university entrance language requirements for foreign L2 students in 28 European countries or regions with an autonomous educational policy. It is based on structured interviews with 30 respondents involved with university entrance test development and knowledgeable about university entrance policies in their context. The results show that the CEFR is omnipresent in European university entrance language tests and that the B2 is the most commonly used level in that context. The data also show that normative CEFR use is very common and that in many contexts CEFR levels are misused for marketing purposes or to control university admission.

The B2 Level and the Dream of a Common Standard

ABSTRACT In Flanders, Belgium, university admission of undergraduate international L2 students requires a certificate of an accredited test of Dutch. The two main university entrance tests used for certification share highly comparable oral components and CEFR-based oral rating criteria. This article discusses to what extent ratings on the oral components of these tests can be compared. The data used are the ratings of the oral performances of the same 82 candidates on both oral test components, which were administered within the same week. The correlation on the overall scores is high, but lower on the oral test component. Further analyses, including linear regression and multifaceted Rasch analysis, indicate that the B2 level was interpreted differently in the two tests. The results show that using the same language proficiency scales as the basis for rating scale criteria may lead to superficial correspondences or a perceived equivalence but does not necessarily lead to greater comparability of shared criteria. The findings of this study are especially useful for contexts in which different tests use similar criteria that are based on the same descriptors, and comparability is only assumed.

The Use of the CEFR in Higher Education: A Brief Introduction to This Special Issue

One of the intended purposes of The Common European Framework of Reference (CEFR) is to provide a common basis for language textbooks, curricula, and examinations across Europe. It is probably fair...