Elena Cotos

Enhancing writing pedagogy with learner corpus data

Learner corpora have become prominent in language teaching and learning, enhancing data-driven learning (DDL) pedagogy by promoting ‘learning driven data’ in the classroom. This study explores the potential of a local learner corpus by investigating the effects of two types of DDL activities, one relying on a native-speaker corpus (NSC) and the second combining native-speaker and learner corpora. Both types of activities aimed at improving second language writers’ knowledge of linking adverbials and were based on a preliminary analysis of adverbial use in the local learner corpus produced by 31 study participants. Quantitative and qualitative data, obtained from writing samples, pre/post-tests, and questionnaires, were converged through concurrent triangulation. The results showed an increase in frequency, diversity and accuracy in all participants’ use of adverbials, but more significant improvement was made by the students who were exposed to the corpus containing their own writing. The findings of this study are thus interpreted as suggestive that combining learner and native-speaker data is a feasible and effective practice, which can be readily integrated in DDL-based instruction with positive impact.

Validity Arguments for Diagnostic Assessment Using Automated Writing Evaluation.

Two examples demonstrate an argument-based approach to validation of diagnostic assessment using automated writing evaluation (AWE). Criterion®, was developed by Educational Testing Service to analyze students’ papers grammatically, providing sentence-level error feedback. An interpretive argument was developed for its use as part of the diagnostic assessment process in undergraduate university English for academic purposes (EAP) classes. The Intelligent Academic Discourse Evaluator (IADE) was developed for use in graduate EAP university classes, where the goal was to help students improve their discipline-specific writing. The validation for each was designed to support claims about the intended purposes of the assessments. We present the interpretive argument for each and show some of the data that have been gathered as backing for the respective validity arguments, which include the range of inferences that one would make in claiming validity of the interpretations, uses, and consequences of diagnostic AWE-based assessments.

Automatic Identification of Discourse Moves in Scientific Article Introductions

This paper reports on the first stage of building an educational tool for international graduate students to improve their academic writing skills. Taking a text-categorization approach, we experimented with several models to automatically classify sentences in research article introductions into one of three rhetorical moves. The paper begins by situating the project within the larger framework of intelligent computer-assisted language learning. It then presents the details of the study with very encouraging results. The paper then concludes by commenting on how the system may be improved and how the project is intended to be pursued and evaluated.

Genre-Based Automated Writing Evaluation for L2 Research Writing

Foreword by Carol A. Chapelle Introduction PART I: DESIGNING GENRE-BASED AWE FOR L2 RESEARCH WRITING 1. Learning and Teaching Challenges of Research Writing 2. Automated Writing Evaluation (AWE) 3. Conceptualizing Genre-based AWE for L2 Research Writing 4. Prototyping Genre-based AWE for L2 Research Writing: The Intelligent Academic Discourse Evaluator (IADE) PART II: IMPLEMENTING AND EVALUATING GENRE-BASED AWE FOR L2 RESEARCH WRITING 5. Exploring the IADE Genre-based Prototype 6. Evaluating the IADE Genre-based Prototype 7. From Prototyping to Principled Practical Realization Conclusion: Glimpse into the Future

Rapid Tagging and Reporting for Functional Language Extraction in Scientific Articles

This paper describes the development of a web-based application for tagging scientific articles, in part to create machine learning training datasets for automated functional language identification and extraction (AFLEX). The initial intent for this work was to provide a new member of the ecosystem of tools that facilitate the structured automation of systematic reviews, an area of work that typically requires critical analysis of multiple research studies and provides an exhaustive summary of literature related to a research question. However, the tools modular interface allows use across disciplines. A user may upload PDF or text documents and quickly tag selected parts of the document with a customizable set of discipline-specific tags, and export results to CSV or JSON formats. An integrated back-end database stores tagging data for comparison between taggers or visual display of results on the web browser. While other discipline-specific text tagging tools exist, the authors have not encountered a cloud-based customizable tool for PDF and text annotation as flexible as the AFLEX Tag Tool developed by the authors.

Evaluating the IADE Genre-Based Prototype

This chapter recounts empirical evidence supporting theory-driven claims about four CALL qualities investigated to evaluate IADE: Language Learning Potential, Meaning Focus, Learner Fit, and Impact. In reporting the findings, I first introduce the students’ perspective and then present the insights derived from introspective and inferential interpretations. Throughout the chapter, I diligently infiltrate students’ voices and writing excerpts to illustrate phenomena that help construe the effects of this corpus-based AWE prototype.

Conceptualizing Genre-Based AWE for L2 Research Writing

This chapter presents a conceptual model for the design of genre-based AWE for L2 research writing. The model is needs-based and combines theoretical and operational frameworks. The theoretical framework underscores desired learning phenomena, drawing from Systemic Functional Linguistics (SFL), the Interactionist Approach to Second Language Acquisition (SLA), and Skill Acquisition Theory (SAT). I discuss these theories here in terms of their main tenets as well as in terms of their relevance to L2 writing, indicating areas that could be strengthened by genre-based AWE.

Prototyping Genre-Based AWE for L2 Research Writing: The Intelligent Academic Discourse Evaluator

This chapter introduces a conceptual design model and demonstrates how this model was applied to a prototype genre-based AWE program — the Intelligent Academic Discourse Evaluator (IADE). I explain how the intended purpose of IADE dictated by the needs of targeted learners and teachers can be realized through a synthesis of the theoretical and operational principles covered in Chapter 3. This complex rationale is a judgmental analysis, which is an important segment of the evaluation argument (Chapelle, 2001) about IADE. The judgmental analysis will further be warranted by empirical analyses in Part II of this volume. I also include here a brief description of two empirical prerequisites — the preparation of training data from a specialized corpus and the development of the analysis engine for feedback generation.

Learning and Teaching Challenges of Research Writing

Research writing poses a great challenge for graduate students who are novice writers struggling with transitioning from peripheral to full participation in the discourse of their disciplinary community. At the same time, teaching research writing is often daunting for writing instructors due to the unfamiliar disciplinary conventions of research genres. Addressing these learning and pedagogical challenges necessitates an understanding of the individual-cognitive and socio-disciplinary dimensions underpinning research writing. In this first chapter, I elaborate on what these dimensions entail and how they intertwine in the construct of research writing competence. To further reason about how that applies to L2 research writing pedagogy, I discuss two epistemologically different genre teaching traditions — linguistic and rhetorical. I then put forth a rationale for enhancing L2 research writing instruction with genre and corpus-based AWE technology that can foster fundamental linguistic and rhetorical principles.

Conclusion: Glimpse into the Future

RWT was designed to enhance L2 research writing pedagogy with formative AWE affordances for engaging students in intra- and inter-personal communicative activity aimed at creating exemplars of genre writing, while customizing their own revision process. As I have demonstrated throughout the book, RWT development has undergone rigorous judgmental and empirical evaluations, which support the suitability of this tool for use with L2 novices to scholarly writing. Ellis (2004) argues that ‘[w]hen genre-based literacy pedagogy is supported by technology, the potential for student control over learning increases. While the technology increases the potential for control, it is not the technology per se that motivates its design’ (p. 231). I concur, and I would also add that it is not the technology that drives implementation. Therefore, the work described in this volume is only the first step towards conceptualizing an AWE-augmented approach to L2 genre and corpus-based writing instruction.