Rick Morgan

Success in college for students with discrepancies between performance on multiple-choice and essay tests.

Students with high scores (top third) on the essay portion of an Advanced Placement examination and low scores (bottom third) on the multiple-choice portion of the examination were compared with students with the opposite pattern (top third on the multiple-choice questions and bottom third on the essay questions). Across examinations in different subject areas (history, English, and biology), students who were relatively strong in the essay format and weak in the multiple-choice format were as successful in their college courses as students with the opposite pattern, especially in those courses where grades are typically not determined by multiple-choice tests. Although differential essay and multiple-choice test performance was not related to course grades, it was related to performance on other tests. Students in the high multiple-choice/low essay group performed much better than the other group on other multiple-choice tests, especially the verbal section of the SAT. In relation to their performance on multiple-choice tests, students in the high essay/low multiple-choice group performed well on other essay tests.

Developing and Validating Band Levels and Descriptors for Reporting Overall Examinee Performance

This study presents the development and empirical validation of score levels and descriptors specifically designed for reporting purposes to provide test takers with more than just a number on a score scale. In the context of a test primarily intended for 11- to 15-year-old students learning English as a second/foreign language, the study examined the number of band levels that could be meaningfully distinguished, the reliability of the classification of students into these band levels, and the development of overall performance descriptors that would provide meaningful information to score users. The performance data from 2,931 students who took the test were used. The band level solution was determined by balancing considerations for the reliability of classification decisions and the desire for the levels to represent meaningful performance differences. To construct meaningful descriptors for the band levels, multiple sources of information were examined, including the scoring rubrics, the characteristics of test items, typical student performance profiles, and the performance of norm groups on the test. The importance of establishing the psychometric quality of band levels and the empirical basis for performance descriptors, as well as the implications for similar efforts, are discussed.

Enhancing the Interpretability of the Overall Results of an International Test of English-Language Proficiency

The purpose of this study was to enhance the meaning of the scores of an English-language test by developing performance levels and descriptors for reporting overall test performance. The levels and descriptors were intended to accompany the total scale scores of TOEFL Junior® Standard, an international test of English as a second/foreign language. The study addressed two issues: the number of performance levels that could be meaningfully reported and the information that should be included in the performance level descriptors. Data from 3,607 students who took an operational test form were used. Although our methodology built to some extent on earlier work (Papageorgiou et al., 2015), we demonstrate how content and construct differences between the tests of each study dictated use of different types of data in order to construct meaningful performance descriptors and select the cut-offs for the levels. We emphasize the importance of establishing the psychometric quality of the reported overall levels and the empirical basis for the content of their performance descriptors, and we discuss the implications of our work for similar efforts in the future.