Casper Harteveld

The research and evaluation of serious games: Toward a comprehensive methodology

The authors present the methodological background to and underlying research design of an ongoing research project on the scientific evaluation of serious games and/or computer-based simulation games (SGs) for advanced learning. The main research questions are: (1) what are the requirements and design principles for a comprehensive social scientific methodology for the evaluation of SGs?; (2) to what extent do SGs contribute to advanced learning?; (3) what factors contribute to or determine this learning?; and (4) to what extent and under what conditions can SG-based learning be transferred to the real world? In the Netherlands between 2005 and 2012, several hundred SG sessions with 12 SGs were evaluated systematically, uniformly and quantitatively to create a dataset, which comprises data on 2488 respondents in higher education or work organizations. The authors present the research model, the quasi-experimental design and the evaluation instruments. This focus in this paper is on the methodology and dataset, which form a sound foundation for forthcoming publications on the empirical results. [ABSTRACT FROM AUTHOR]

Triadic Game Design

Many designers, policy makers, teachers, and other practitioners are beginning to understand the usefulness of using digital games beyond entertainment. Games have been developed for teaching, recruiting and to collect data to improve search engines. This book examines the fundamentals of designing any game with a serious purpose and provides a way of thinking on how to design one successfully. The reader will be introduced to a design philosophy called Triadic Game Design.; a theory that all games involve three worlds: the worlds of Reality, Meaning, and Play. Each world is affiliated with aspects. A balance needs to be found within and between the three worlds. Such a balance is difficult to achieve, during the design many tensions will arise, forcing designers to make trade-offs. To deal with these tensions and to ensure that the right decisions are made to create a harmonic game, a frame of reference is needed. This is what Triadic Game Design offers.

Balancing Play, Meaning and Reality: The Design Philosophy of LEVEE PATROLLER

Most serious games have been developed without a proper and comprehensive design theory. To contribute to the development of such a theory, this article presents the underlying design philosophy of LEVEE PATROLLER, a game to train levee patrollers in the Netherlands. This philosophy stipulates that the design of a digital serious game is a multiobjective problem in which trade-offs need to be made. Making these trade-offs takes place in a design space defined by three equally important components: (a) Play, (b) Meaning, and (c) Reality. The various tensions between these three components result in design dilemmas and trilemmas that make it difficult to balance a serious game. Each type of tension is illustrated with one or more examples from the design of LEVEE PATROLLER.

Learning in Single-Versus Multiplayer Games

From the observations of successful entertainment games, it is hypothesized that implementing a single-player option may require a different approach from that in a multiplayer option, in terms of game design. To find out whether this could be true and to understand what the specific approaches could entail for educational games, three separate yet related investigations were conducted to examine single- versus multiplayer games: an investigation of two educational games designed and evaluated by the authors, a theoretical investigation from a game and learning perspective, and an empirical investigation of 23 case studies. From these three investigations, it turned out that a “single-player approach” is data intensive, has formal rules, and uses direct transfer and individual learning. On the other hand, a “multiplayer approach” is less straightforward. From a game perspective, it can be characterized as process intensive and having social rules. When related to learning, however, it could be positioned on any dimension. This exploration shows that the approaches differ to a large extent and that designers have to find a fit between what option they choose and approach they take.

Design for Engaging Experience and Social Interaction

One of the goals of game designers is to design for an engaging experience and for social interaction. The question is how. We know that games can be engaging and allow for social interaction, but how do we achieve this or even improve on it? This article provides an overview of several scientific approaches that deal with this question. It highlights the idea that the articles in this symposium each have a specific contribution route for answering this question: By means of theory building, user experience, and design research, they aim to increase our understanding of this design question. This guest editorial also highlights the specific aims of each article and the insights we can retrieve from them. Although much more research is needed, taking our cue from these articles, we are able to provide some answers as to how and when engagement and social interaction are established, and to what extent.

On the Automatic Assessment of Computational Thinking Skills: A Comparison with Human Experts

Programming and computational thinking skills are promoted in schools worldwide. However, there is still a lack of tools that assist learners and educators in the assessment of these skills. We have implemented an assessment tool, called Dr. Scratch, that analyzes Scratch projects with the aim to assess the level of development of several aspects of computational thinking. One of the issues to address in order to show its validity is to compare the (automatic) evaluations provided by the tool with the (manual) evaluations by (human) experts. In this paper we compare the assessments provided by Dr. Scratch with over 450 evaluations of Scratch projects given by 16 experts in computer science education. Our results show strong correlations between automatic and manual evaluations. As there is an ample debate among educators on the use of this type of tools, we discuss the implications and limitations, and provide recommendations for further research.

Opening the Black Box of Play: Strategy Analysis of an Educational Game

A significant issue in research on educational games lies in evaluating their educational impact. Although game analytics is often leveraged in the game industry, it can also provide insight into player actions, strategy development, and the learning process in educational games separate from external evaluation measures. This paper explores the potential of game analytics for learning by analyzing player strategies of an educational game that is designed to support algorithmic thinking. We analyze player strategies from nine cases in our data, combining quantitative and qualitative game analysis techniques: hierarchical player clustering, game progression visualizations, playtraces, and think-aloud data. Results suggest that this combination of data analysis techniques provides insights into level progression and learning strategies that may have been otherwise overlooked.

Assessing Computational Thinking in Students' Game Designs

Designing games requires a complex sequence of planning and executing actions. This paper suggests that game design requires computational thinking, and discusses two methods for analyzing computational thinking in games designed by students in the visual programming language Scratch. We present how these two analyses produce different narratives of computational thinking for our case studies, and reflect on how we plan to move forward with our larger analysis.

Guest editorial: games for learning and dialogue on humanitarian work

Purpose – The purpose of this guest editorial is to provide the introduction and context of the Special Issue on Games for Learning and Dialogue on Humanitarian Work. The Special Issue aims to promote the development, deployment, and analysis of games for the humanitarian sector: it investigates how games can meaningfully engage people and organizations in experiencing, understanding and improving complex systems. Design/methodology/approach – The editorial describes the need and motivation for building a body of knowledge on the use of games in the humanitarian sector. It further gives an overview of the three papers included in this Special Issue and how they contribute to building such a body of knowledge. Findings – Games enable participants to experience the complexity of humanitarian systems, linking decisions with consequences. Even though game-like approaches have been used for decades in disaster management, there is little written about it. The papers included in this Special Issue provide insig...

Contrasting stories on overcoming governance challenges: the implementation of the EU Water Framework Directive in the Netherlands

The European Union Water Framework Directive (WFD) has provided the European Member States with a range of interacting governance challenges. This article studies three of these (the need for new administrative arrangements, public participation, and the enforced strict time frame). It questions how these interacting governance challenges were addressed in implementing the WFD in the Netherlands – a particularly interesting country since the European Commission assesses its implementation process in relatively positive terms, while an in-depth study reported on in this article tells a contrasting story. Based on this study, the article concludes that especially the interaction effects between the governance challenges may help us to better understand the outcome of the WFD-implementation process, and to provide more suitable advice as to how to improve the implementation process in future rounds.