Christian Sebastian Loh

Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics

The behavioral differences between expert and novice performance is a well-studied area in training literature. Advances in technology have made it possible to trace players’ actions and behaviors within an online gaming environment as user-generated data for performance assessment. In this study, we introduce the use of string similarity to differentiate likely-experts from a group of unknown performers (mixture of novices and experts) based on how similar their in-game actions are to that of experts. Our findings indicate that string similarity is viable as an empirical assessment method to differentiate likely-experts from novices and potentially useful as the first performance metric for Serious Games Analytics (SEGA).

Information Trails: In-Process Assessment of Game-Based Learning

Assessment is an issue that is important to educators because without it, there is no way of telling if learners have arrived at the intended destination. Conducting assessments with game-based learning requires new tools and research methodologies because traditional face-to-face techniques do not transfer well into the multi-user virtual environments found in game worlds. Formative assessment is more useful to instructors and learners because it provides multi-point feedback to help them self-reflect and improve on what they are doing. This chapter describes an “all-rounded” assessment system for game-based learning, which take into consideration the needs of multiple consumers (of information), namely, the Administrators, the Trainers or Educators, and the Learners. The assessment system made use of Information Trails ©—a viable assessment methodology to collect user-generated action data as the game-based learning is occurring (hence, an in-process assessment). The data collected are then remotely transmitted to an external data storage using telemetry, and displayed in real-time via a data visualization application known as Performance Tracing Report Assistant

Serious Games Analytics: Theoretical Framework

“Serious Games” is a unique industry that is concerned with the training/learning performance assessment of its clients. It is one of three digital technology industries (along with digital games, and online learning) that are rapidly advancing into the arena of analytics. The analytics from these industries all came from the tracing of user-generated data as they interacted with the systems, but differed from one another in the primary purposes for such analytics. For example, the purpose of game analytics is to support the growth of digital (entertainment) games, while that of learning analytics is to support the online learning industries. Although some game and learning analytics can indeed be used in serious games, they lack specific metrics and methods that outline the effectiveness of serious games—an important feature to stakeholders. Serious Games Analytics need to provide (actionable) insights that are of values to the stakeholders—specific strategies/policies to improve the serious games, and to (re)train or remediate play-learners for performance improvement. Since the performance metrics from one industry are unlikely to transfer well into another industry, those that are optimal for use in the Serious Games industry must be properly identified as Serious Games Analytics—to properly measure, assess, and improve performance with serious games.

researching and developing Serious Games as Interactive Learning Instructions

As serious games gain momentum in the academic arena, no doubt more educators and instructional technologists will begin considering the possibility of making their own games for instruction. As developers of instructional resources, instructional technologists need to steer clear of producing more ‘video’ games, and instead, developing more ‘serious’ games that incorporate both learning and assessment. The research community needs to learn from tested processes and best practices to avoid repeating old mistakes. The model for serious game making presented in this article has been used successfully for the creation of an award winning project, and will now be shared for the benefits of fellow researchers, educators, and instructional technologists.

Improving the Impact and Return of Investment of Game-Based Learning

ABSTRACT Today’s economic situation demands that learning organizations become more diligent in their business deal-ings to reduce cost and increase bottom line for survival. While there are many champions and proponents claiming that game-based learning (GBL) is sure to improve learning, researchers have, thus far, been unable to (re)produce concrete, empirical evidence supporting this claim. Lacking appropriate assessment methodolo-gies to showcase the effectiveness of the learning technology and to convince stakeholders that GBLs could really work, it is no wonder that many learning organizations regard GBL training as a potentially high-risk technology investment. This paper describes a GBL assessment methodology designed specifically to collect player-generated data in an in situ manner (i.e., within the game environment itself) through telemetry. This methodology further incorporate data visualization to translate the data collected into meaningful information and actionable intelligence for various stakeholders.DOI: 10.4018/jvple.2013010101

Measuring Expert Performance for Serious Games Analytics: From Data to Insights

Advances in technology have made it possible to trace players’ actions and behaviors (as user-generated data) within online serious gaming environments for performance measurement and improvement purposes. Instead of a Black box approach (such as pretest/posttest), we can approach serious games as a White box, assessing performance of play-learners by manipulating the performance variables directly. In this chapter, we describe the processes to obtain user-generated gameplay data in situ using serious games for training—i.e., data tracing, cleaning, mining, and visualization. We also examine ways to differentiate expert-novice performances in serious games, including behavior profiling. We introduce a new Expertise Performance Index, based on string similarities that take into account the “course of actions” chosen by experts and compare that to those of the novices. The Expertise Performance Index can be useful as a metric for serious games analytics because it can rank play-learners according to their competency levels in the serious games.

Confronting the Dark Side of Video Games

New video games have increasingly allowed players to choose their own destiny (or path of character development) by presenting them with choice of actions from both light and dark sides. Some studies suggested that players who have been exposed to the “dark side” of video games may act out unacceptable social behaviors based on their learning experiences. This issue of the effect of unintentional learning in video games deserves much scrutiny and honest discussion among researcher. Unfortunately, the issue has so far, either been bashed or glossed over by experts on both side of the effect-of-video-games chasm.

Once Upon a Game

In view of the recent interest in using videogames for learning, many teachers and parents have begun to question the place of videogames in the classrooms. In this chapter, we attempt to explore the very idea of playing and learning by trying to rediscover the hidden meanings in usual words, like “game,” “play,” “school” and “education” through a lexical and conceptual analysis within the Western culture, roaming among ancient and modern languages. It is through the rediscovery of our roots that we as educators can be better informed to either embrace or discard the call to integrate play into education for game-based instruction.

Choice and effects of instrument sound in aural training

A musical note produced through the vibration of a single string is psychoacoustically simpler/purer than that produced via multiple-strings vibration. Does the psychoacoustics of instrument sound have any effect on learning outcomes in music instruction? This study investigated the effect of two psychoacoustically distinct instrument sounds on pitch discrimination achievement of college music students. Repeated-measures analysis revealed a significant difference between achievement scores of students by instrument sounds, which indicated that psychoacoustic complexity of instrument sound could affect learning outcome in music instruction.A musical note produced through the vibration of a single string is psychoacoustically simpler/purer than that produced via multiple-strings vibration. Does the psychoacoustics of instrument sound have any effect on learning outcomes in music instruction? This study investigated the effect of two psychoacoustically distinct instrument sounds on pitch discrimination achievement of college music students. Repeated-measures analysis revealed a significant difference between achievement scores of students by instrument sounds, which indicated that psychoacoustic complexity of instrument sound could affect learning outcome in music instruction.

Lost my way: an educational geometry game for young children

Playful activities are regarded to be a promising way for fostering childrens geometric thinking and understanding. In this paper we introduce Lost My Way, an educational game about geometric transformations targeted at middle school children of about 10 years of age. We describe the design of the game and report preliminary results and experiences gained from the first two stages of our iterative development process which included a playtesting session and an on-site evaluation at a local middle school.