Laura K. Allen

Cohesion network analysis of CSCL participation

The broad use of computer-supported collaborative-learning (CSCL) environments (e.g., instant messenger–chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54% of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.

Analyzing Discourse Processing Using a Simple Natural Language Processing Tool

Natural language processing (NLP) provides a powerful approach for discourse processing researchers. However, there remains a notable degree of hesitation by some researchers to consider using NLP, at least on their own. The purpose of this article is to introduce and make available a simple NLP (SiNLP) tool. The overarching goal of the article is to proliferate the use of NLP in discourse processing research. The article also provides an instantiation and empirical evaluation of the linguistic features measured by SiNLP to demonstrate their strength in investigating constructs of interest to the discourse processing community. Although relatively simple, the results of this analysis reveal that the tool is quite powerful, performing on par with a sophisticated text analysis tool, Coh-Metrix, on a common discourse processing task (i.e., predicting essay scores). Such a tool could prove useful to researchers interested in investigating features of language that affect discourse production and comprehension.

Pssst... textual features... there is more to automatic essay scoring than just you

This study investigates a new approach to automatically assessing essay quality that combines traditional approaches based on assessing textual features with new approaches that measure student attributes such as demographic information, standardized test scores, and survey results. The results demonstrate that combining both text features and student attributes leads to essay scoring models that are on par with state-of-the-art scoring models. Such findings expand our knowledge of textual and non-textual features that are predictive of writing success.

Chapter 3.1 A Multi-Dimensional analysis of essay writing: What linguistic features tell us about situational parameters and the effects of language functions on judgments of quality

This study applied the Multi-Dimensional analysis used by Biber (1988) to examine the functional parameters of essays. Co-occurrence patterns were identified within an essay corpus (n = 1529) using linguistic indices provided by Coh-Metrix. These patterns were used to identify essay groups that shared features based upon situational parameters. Results revealed that the linguistic features reliably co-occurred according to the parameters. Namely, four dimensions were interpreted and associated with essay quality, prompt, and grade level. Confirmatory analyses revealed that the dimensions reliably distinguished among the parameters. Results provide insight into the situational parameters that affect writing, and the extent to which essays vary among and between themselves. The results have important implications for composition pedagogy, writing assessment, and writing theory.

The writing pal: A writing strategy tutor

Any books that you read, no matter how you got the sentences that have been read from the books, surely they will give you goodness. But, we will show you one of recommendation of the book that you need to read. This adaptive educational technologies for literacy instruction is what we surely mean. We will show you the reasonable reasons why you need to read this book. This book is a kind of precious book written by an experienced author.

Am I Wrong or Am I Right? Gains in Monitoring Accuracy in an Intelligent Tutoring System for Writing

We investigated whether students increased their self-assessment accuracy and essay scores over the course of an intervention with a writing strategy intelligent tutoring system, W-Pal. Results indicate that students were able to learn from W-Pal, and that the combination of strategy instruction, game-based practice, and holistic essay-based practice led to equivalent gains in self-assessment accuracy compared to heavier doses of deliberate writing practice (offering twice the amount of system feedback).

Spendency: Students' Propensity to Use System Currency

Using students’ process data from the game-based Intelligent Tutoring System (ITS) iSTART-ME, the current study examines students’ propensity to use system currency to unlock game-based features, (i.e., referred to here as spendency). This study examines how spendency relates to students’ interaction preferences, in-system performance, and learning outcomes (i.e., self-explanation quality, comprehension). A group of 40 high school students interacted with iSTART-ME as part of an 11-session experiment (pretest, eight training sessions, posttest, and a delayed retention test). Students’ spendency was negatively related to the frequency of their use of personalizable features. In addition, students’ spendency was negatively related to their in-system achievements, daily learning outcomes, and performance on a transfer comprehension task, even after factoring out prior ability. The findings from this study indicate that increases in students’ spendency are systematically related to their selection choices and may have a negative effect on in-system performance, immediate learning outcomes, and skill transfer outcomes. The results have particular relevance to game-based systems that incorporate currency to unlock features within games as well as to the differential tradeoffs of game features on motivation and learning.

Promoting Metacognitive Awareness within a Game-Based Intelligent Tutoring System

Metacognitive awareness has been shown to be a critical skill for academic success. However, students often struggle to regulate this ability during learning tasks. The current study investigates how features designed to promote metacognitive awareness can be built into the game-based intelligent tutoring system (ITS) iSTART-2. College students (n=28) interacted with iSTART-2 for one hour, completing lesson videos and practice activities. If students’ performance fell below a minimum threshold during game-based practice, they received a pop-up that alerted them of their poor performance and were subsequently transitioned to a remedial activity. Results revealed that students’ scores in the system improved after they were transitioned (even when they did not complete the remedial activity). This suggests that the pop-up feature in iSTART-2 may indirectly promote metacognitive awareness, thus leading to increased performance. These results provide insight into the potential benefits of real-time feedback designed to promote metacognitive awareness within a game-based learning environment.

The Dynamical Analysis of Log Data Within Educational Games

Games and game-based environments frequently provide users multiple trajectories and paths. Thus, users often have to make decisions about how to interact and behave during the learning task. These decisions are often captured through the use of log data, which can provide a wealth of information concerning students’ choices, agency, and performance while engaged within a game-based system. However, to analyze these changing data sets, researchers need to use methodologies that focus on quantifying fine-grained patterns as they emerge across time. In this chapter, we will consider how dynamical analysis techniques offer researchers a unique means of visualizing and characterizing nuanced decision and behavior patterns that emerge from students’ log data within game-based environments. Specifically, we focus on how three distinct types of dynamical methodologies, Random Walks, Entropy analysis, and Hurst exponents, have been used within the game-based system iSTART-2 as a form of stealth assessment. These dynamical techniques provide researchers a means of unobtrusively assessing how students behave and learn within game-based environments.

Teaching iSTART to understand Spanish

iSTART is a web-based reading comprehension tutor. A recent translation of iSTART from English to Spanish has made the system available to a new audience. In this paper, we outline several challenges that arose during the development process, specifically focusing on the algorithms that drive the feedback. Several iSTART activities encourage students to use comprehension strategies to generate self-explanations in response to challenging texts. Unsurprisingly, analyzing responses in a new language required many changes, such as implementing Spanish natural language processing tools and rebuilding lists of regular expressions used to flag responses. We also describe our use of an algorithm inspired from genetics to optimize the Fischer Discriminant Function Analysis coefficients used to determine self-explanation scores.