Erica L. Snow

Feedback and Revising in an Intelligent Tutoring System for Writing Strategies

This study investigates students’ essay revising in the context of an intelligent tutoring system called Writing Pal (W-Pal), which combines strategy instruction, game-based practice, essay writing practice, and automated formative feedback. We examine how high school students use W-Pal feedback to revise essays in two different contexts: a typical approach that emphasizes intensive writing practice, and an alternative approach that offers less writing practice with more direct strategy instruction. Results indicate that students who wrote fewer essays, but received W-Pal strategy instruction, were more likely to make substantive revisions that implemented specific recommendations conveyed by the automated feedback. Additional analyses consider the role of motivation and perceived learning on students’ revising behaviors.

Pssst... textual features... there is more to automatic essay scoring than just you

This study investigates a new approach to automatically assessing essay quality that combines traditional approaches based on assessing textual features with new approaches that measure student attributes such as demographic information, standardized test scores, and survey results. The results demonstrate that combining both text features and student attributes leads to essay scoring models that are on par with state-of-the-art scoring models. Such findings expand our knowledge of textual and non-textual features that are predictive of writing success.

iSTART-2: A Reading Comprehension and Strategy Instruction Tutor.

Any books that you read, no matter how you got the sentences that have been read from the books, surely they will give you goodness. But, we will show you one of recommendation of the book that you need to read. This adaptive educational technologies for literacy instruction is what we surely mean. We will show you the reasonable reasons why you need to read this book. This book is a kind of precious book written by an experienced author.

If the Gear Fits, Spin It!: Embodied Education and in-Game Assessments

Two embodied gears games were created. Better learners should use fewer gear switches to reflect their knowledge. Twenty-three 7th graders, playing as dyads, used gestures to manipulate virtual gears. The Kinect sensor tracked arm-spinning movements and switched gear diameters. Knowledge tests were administered. Statistically significant knowledge gains were seen. For Game 1 gear spun one direction, switching significantly predicted only pretest knowledge. For Game 2 gear spun two directions switching was also negatively correlated with both tests. For game 2, those who used fewer switches during gameplay understood the construct better scoring higher on both tests. Dyadic analyses revealed the winner used significantly fewer switches. In-process data can provide a window onto knowledge as it is being encoded. However, games should stay within the learners ZPD, because if the game is too easy Game 1, meaningful data may be difficult to gather. The use of in ludo data from games with high sensitivity may attenuate the need for repetitive traditional, post-intervention tests.

Promoting Metacognition Within a Game-Based Environment

Metacognition refers to students’ ability to reflect upon what they know and what they do not know. However, many students often struggle to master this regulatory skill. We have designed and implemented two features to promote metacognition within the game-based system iSTART-2. These two features have been tested and shown to have positive impacts on students’ ability to reflect upon their performance. Future work is being planned to further explore the most effective way to implement these features and the ultimate impact they have on learning outcomes. We are seeking advice and feedback on the methodology and metacognitive feature design that will be included in a series of follow-up studies. The implications of this work for both iSTART-2 and the AIED field are discussed.

Expectations of Technology: A Factor to Consider in Game-Based Learning Environments

This study investigates how students’ prior expectations of technology affect overall learning outcomes across two adaptive systems, one game-based (iSTART-ME) and one non-game based (iSTART-Regular). The current study (n=83) is part of a larger study (n=124) intended to teach reading comprehension strategies to high school students. Results revealed that students’ prior expectations impacted learning outcomes, but only for students who had engaged in the game-based system. Students who reported positive expectations of computer helpfulness at pretest showed significantly higher learning outcomes in the game-based system compared to students who had low expectations of computer helpfulness. The authors discuss how the incorporation of game-based features in an adaptive system may negatively impact the learning outcomes of students with low technology expectations.

iSTART Therefore I Understand: But Metacognitive Supports Did Not Enhance Comprehension Gains.

iSTART is an intelligent tutoring system designed to provide self-explanation instruction and practice to improve students’ comprehension of complex, challenging text. This study examined the effects of extended game-based practice within the system as well as the effects of two metacognitive supports implemented within this practice. High school students (n = 234) were either assigned to an iSTART treatment condition or a control condition. Within the iSTART condition, students were assigned to a 2 × 2 design in which students provided self-assessments of their performance or were transferred to Coached Practice if their performance did not reach a certain performance threshold. Those receiving iSTART training produced higher self-explanation and inference-based comprehension scores. However, there were no direct effects of either metacognitive support on these learning outcomes.

Am I Wrong or Am I Right? Gains in Monitoring Accuracy in an Intelligent Tutoring System for Writing

We investigated whether students increased their self-assessment accuracy and essay scores over the course of an intervention with a writing strategy intelligent tutoring system, W-Pal. Results indicate that students were able to learn from W-Pal, and that the combination of strategy instruction, game-based practice, and holistic essay-based practice led to equivalent gains in self-assessment accuracy compared to heavier doses of deliberate writing practice (offering twice the amount of system feedback).

Spendency: Students' Propensity to Use System Currency

Using students’ process data from the game-based Intelligent Tutoring System (ITS) iSTART-ME, the current study examines students’ propensity to use system currency to unlock game-based features, (i.e., referred to here as spendency). This study examines how spendency relates to students’ interaction preferences, in-system performance, and learning outcomes (i.e., self-explanation quality, comprehension). A group of 40 high school students interacted with iSTART-ME as part of an 11-session experiment (pretest, eight training sessions, posttest, and a delayed retention test). Students’ spendency was negatively related to the frequency of their use of personalizable features. In addition, students’ spendency was negatively related to their in-system achievements, daily learning outcomes, and performance on a transfer comprehension task, even after factoring out prior ability. The findings from this study indicate that increases in students’ spendency are systematically related to their selection choices and may have a negative effect on in-system performance, immediate learning outcomes, and skill transfer outcomes. The results have particular relevance to game-based systems that incorporate currency to unlock features within games as well as to the differential tradeoffs of game features on motivation and learning.

Promoting Metacognitive Awareness within a Game-Based Intelligent Tutoring System

Metacognitive awareness has been shown to be a critical skill for academic success. However, students often struggle to regulate this ability during learning tasks. The current study investigates how features designed to promote metacognitive awareness can be built into the game-based intelligent tutoring system (ITS) iSTART-2. College students (n=28) interacted with iSTART-2 for one hour, completing lesson videos and practice activities. If students’ performance fell below a minimum threshold during game-based practice, they received a pop-up that alerted them of their poor performance and were subsequently transitioned to a remedial activity. Results revealed that students’ scores in the system improved after they were transitioned (even when they did not complete the remedial activity). This suggests that the pop-up feature in iSTART-2 may indirectly promote metacognitive awareness, thus leading to increased performance. These results provide insight into the potential benefits of real-time feedback designed to promote metacognitive awareness within a game-based learning environment.