Erik Harpstead

RumbleBlocks: Teaching science concepts to young children through a Unity game

RumbleBlocks was developed at the Entertainment Technology Center (ETC) to teach engineering principles of tower stability to children ages 4-7. The game features tower construction, tower piece removal, and tower comparison levels which were designed with feedback from early childhood educators and learning researchers, and iteratively improved with feedback from child play tests. This paper emphasizes the development process, and initial formative play tests with children. It was developed using the Unity3D game engine, allowing for export as a stand-alone application, web player, or to mobile devices. First results are promising in terms of educational effectiveness, with more studies planned for the future.

Using extracted features to inform alignment-driven design ideas in an educational game

As educational games have become a larger field of study, there has been a growing need for analytic methods that can be used to assess game design and inform iteration. While much previous work has focused on the measurement of student engagement or learning at a gross level, we argue that new methods are necessary for measuring the alignment of a game to its target learning goals at an appropriate level of detail to inform design decisions. We present a novel technique that we have employed to examine alignment in an open-ended educational game. The approach is based on examining how the game reacts to representative student solutions that do and do not obey target principles. We demonstrate this method using real student data and discuss how redesign might be informed by these techniques.

Learning Curve Analysis for Programming: Which Concepts do Students Struggle With?

The recent surge in interest in using educational data mining on student written programs has led to discoveries about which compiler errors students encounter while they are learning how to program. However, less attention has been paid to the actual code that students produce. In this paper, we investigate programming data by using learning curve analysis to determine which programming elements students struggle with the most when learning in Python. Our analysis extends the traditional use of learning curve analysis to include less structured data, and also reveals new possibilities for when to teach students new programming concepts. One particular discovery is that while we find evidence of student learning in some cases (for example, in function definitions and comparisons), there are other programming elements which do not demonstrate typical learning. In those cases, we discuss how further changes to the model could affect both demonstrated learning and our understanding of the different concepts that students learn.

What Drives People: Creating Engagement Profiles of Players from Game Log Data

A central interest of game designers and game user researchers is to understand why players enjoy their games. While a number of researchers have explored player enjoyment in general, few have talked about methods for enabling designers to understand the players of their specific game. In this paper we explore the creation of engagement profiles of game players based on log data. These profiles take into account the different ways that players engage with the game and highlight patterns associated with active play. We demonstrate our approach by performing a descriptive analysis of the game Forza Motorsport 5 using data from a sample of 1.2 million users of the game and discuss the implications of our findings.

Replay Analysis in Open-Ended Educational Games

Designers of serious games have an interest in understanding if their games are well-aligned, i.e., whether in-game rewards incentivize behaviors that will lead to learning. Few existing serious games analytics solutions exist to serve this need. Open-ended games in particular run into issues of alignment due to their affordances for wide player freedom. In this chapter, we first define open-ended games as games that have a complex functional solution spaces. Next, we describe our method for exploring alignment issues in an open-ended educational game using replay analysis. The method uses multiple data mining techniques to extract features from replays of player behavior. Focusing on replays rather than logging play-time metrics allows designers and researchers to run additional metric calculations and data transformations in a post hoc manner. We describe how we have applied this replay analysis methodology to explore and evaluate the design of the open-ended educational game RumbleBlocks. Using our approach, we were able to map out the solution space of the game and highlight some potential issues that the game’s designers might consider in iteration. Finally, we discuss some of the limitations of the replay approach.

Beanstalk: A unity game addressing balance principles, socio-emotional learning and scientific inquiry

Beanstalk is an educational game for children ages 6-10 teaching balance-fulcrum principles while folding in scientific inquiry and socio-emotional learning. This paper explores the incorporation of these additional dimensions using intrinsic motivation and a framing narrative. Four versions of the game are detailed, along with preliminary player data in a 2×2 pilot test with 64 children shaping the modifications of Beanstalk for much broader testing.

Student Agency and Game-Based Learning: A Study Comparing Low and High Agency

A key feature of most computer-based games is agency: the capability for students to make their own decisions in how they play. Agency is assumed to lead to engagement and fun, but may or may not be helpful to learning. While the best learners are often good self-regulated learners, many students are not, only benefiting from instructional choices made for them. In the study presented in this paper, involving a total of 158 fifth and sixth grade students, children played a mathematics learning game called Decimal Point, which helps middle-school students learn decimals. One group of students (79) played and learned with a low-agency version of the game, in which they were guided to play all “mini-games” in a prescribed sequence. The other group of students (79) played and learned with a high-agency version of the game, in which they could choose how many and in what order they would play the mini-games. The results show there were no significant differences in learning or enjoyment across the low and high-agency conditions. A key reason for this may be that students across conditions did not substantially vary in the way they played, perhaps due to the indirect control features present in the game. It may also be the case that the young students who participated in this study did not exercise their agency or self-regulated learning. This work is relevant to the AIED community, as it explores how game-based learning can be adapted. In general, once we know which game and learning features lead to the best learning outcomes, as well as the circumstances that maximize those outcomes, we can better design AI-powered, adaptive games for learning.