Rens Kortmann

The research and evaluation of serious games: Toward a comprehensive methodology

The authors present the methodological background to and underlying research design of an ongoing research project on the scientific evaluation of serious games and/or computer-based simulation games (SGs) for advanced learning. The main research questions are: (1) what are the requirements and design principles for a comprehensive social scientific methodology for the evaluation of SGs?; (2) to what extent do SGs contribute to advanced learning?; (3) what factors contribute to or determine this learning?; and (4) to what extent and under what conditions can SG-based learning be transferred to the real world? In the Netherlands between 2005 and 2012, several hundred SG sessions with 12 SGs were evaluated systematically, uniformly and quantitatively to create a dataset, which comprises data on 2488 respondents in higher education or work organizations. The authors present the research model, the quasi-experimental design and the evaluation instruments. This focus in this paper is on the methodology and dataset, which form a sound foundation for forthcoming publications on the empirical results. [ABSTRACT FROM AUTHOR]

Evolution of visual resolution constrained by a trade-off

The evolution of visual systems is constrained by a trade-off between spatial and temporal resolution. In this article we aim at identifying the causes of the trade-off at the retinal level in both artificial and natural visual systems. We start by selecting two factors that limit the values of spatial and temporal resolution. Then we show in two experiments on the evolution of an artificial system that the two factors induce trade-off curves connecting the evolved values of spatial and temporal resolution. A comparison of the experimental results with the resolution evolved in natural visual systems leads us to the conclusion that in natural systems the same factors are responsible for the observed trade-off.

Studying Animals through Artificial Evolution: The Cricket Case

In this paper we introduce the notion of historical evidence - the ability to replicate biologically realistic evolutionary scenarios - for hypothesised mechanisms for control of sensorimotor behaviour. We apply the idea to the phonotaxis mechanism proposed by Webb and her collaborators to account for the abilities of Gryllus bimaculatus. To do this, we tested whether the proposed control mechanism, when implemented in a robot model of the animal, could account for evolutionary adaptations observed in the natural system. We describe and discuss the experiment and its results, but start by explaining the methodology used, which is an extension of Webbs existing methodology for obtaining behavioural evidence for a hypothesised mechanism. We conclude there is historical evidence for the neural control mechanism investigated.

In pursuit of evidence: a design and empirical study of a gamified online marketplace

Gamification seems a promising method to engage people and stimulate their activity, both offline and online, but has a dearth of empirical evidence in scientific literature. This study investigated whether gamification can have a positive effect on the amount of user-generated content on online marketplaces. Two similar gamification treatments were designed for the mobile website of OLX India, a marketplace for used goods. The treatments were quantitatively evaluated in a double-blind controlled experiment on 51,103 OLX users, who were randomly selected and evenly assigned to a control group and two treatment groups. The users who actively engaged with either of the treatments created more than 6 times as much content as users who did not engage with the treatments, resulting in an overall content increase of over 18% for the treatment groups compared to the control group. Gamification on online marketplaces is effective for users who actively interact with a treatment, not for all users per se.

The Culture Driven Game Design Method: Adapting Serious Games to the Players’ Culture

Players of serious games are culturally sensitive agents; by interacting with the game and other players they bring their own culture into the game. This can result in conflicting behaviour that hampers the players to reach the objectives of the game. It is therefore necessary that the design of the game architecture is adjusted to the players’ culture. Currently, game designers typically adjust serious games to their players’ culture by playtesting with their target group. However, since playtesting demands a lot of time, incurs high costs and may spoil the client’s first impression of the game, playtesting is not always possible or desirable. This chapter presents an alternative to playtesting which we call the Culture Driven Game Design Method. This method provides a tool to assess and represent the players’ culture as well as a set of guidelines to process this assessment and avoid conflicts between the players’ culture and the architecture of the game.

Veerkracht 2.0

Veerkracht is a game for servant-leadership development. Although the first version of the game already provided a rather authentic environment for professional training, some players reported that it did not sufficiently provide the rich and meaningful interaction required to practice leadership skills. To revise the game we took inspiration from the literature on embodied cognition and added embodied interactions with non-player characters. An evaluation of the revised game, Veerkracht 2.0, yielded significant increases in some of the learning effects when compared to the original game. Further analysis showed that the increased learning was to a large extent due to the revised (embodied) interactions in the game. We concluded that, although some questions remain unanswered, embodied cognition seems a promising area of research for improving game designs and game design theories.

Addressing Challenges of Planning in Multimodal Transportation Nodes with Simulation Games

Global transportation knows many different modalities – goods arrive from faraway places by ship, plane, railway, or truck. Airports and seaports both represent important nodes within the global transportation network. Both show distinct characteristics, but also similarities when it comes to challenges like required flexibility, robustness, reliability and situational awareness of the stakeholders involved. In this article, we introduce two different simulation games addressing some of these challenges in two complex transportation nodes and discuss the qualitative results of user tests with the games. Within a comparative section, we show how simulation games can be used to address the challenges of multimodal transportation.

Parsifal a Game Opera

Richard Wagner’s Parsifal was recently rewritten and performed as a ‘game opera’. We used observations, questionnaires, and interviews to study how the 700+ audience were facilitated to experientially learn about the show’s main themes: compassion and collaboration. This case study contributed to our understanding how performance art may improve games for learning and training purposes, many of which now are notoriously ‘boring’. We concluded that performance art’s main contribution, in particular to games discussing fundamental values such as compassion, is to captivate players and ‘lure’ them into their natural behaviour. Thus the Parsifal game opera emotionally confronted its audience with their – callous and selfish – behaviour and intensified their learning through embodied experiences. However, some players lacked time and support to (collectively) reflect on their experiences and lacked catharsis. Therefore, we recommend using gameful performance art for learning and training purposes, provided that all activities in experiential learning are sufficiently facilitated.