Investigating the effect of source characteristics on task comparability in integrated writing tasks

Abstract

Abstract The study presents an attempt to explore the impact of source characteristics on task comparability in integrated writing tasks. To this end, two read-listen-write tasks of TOEFL iBT were selected to differ in topic, structural organization, and lexical and conceptual overlap, suggested to be significant in affecting summary quality ( Cho, Rijmen, & Novak, 2013 ; Li, 2014 ; Yu, 2009 ). The performance of a group of TOEFL candidates on the two tasks was qualitatively analyzed and quantitatively compared according to the skills integrated tasks aim to assess: 1. comprehension of the source information, 2. discourse synthesis, 3. citation of the sources, and 4. source borrowing strategies (Cumming et al., 2005). Comparisons were also made of examinees’ overall scores on the test, assigned by a group of EAP specialists in a Canadian university based on TOEFL iBT integrated writing rating scale. Significant differences were observed in test-takers’ overall scores, comprehension of source information, type and quality of source borrowing strategies, and tendency either to use their own original ideas and structures or to rely more on those of one source in their summaries. Implications are provided for test developers and test users as well as for future research to be conducted.