Francesco Bellotti

Assessment in and of serious games: an overview

There is a consensus that serious games have a significant potential as a tool for instruction. However, their effectiveness in terms of learning outcomes is still understudied mainly due to the complexity involved in assessing intangible measures. A systematic approach--based on established principles and guidelines--is necessary to enhance the design of serious games, and many studies lack a rigorous assessment. An important aspect in the evaluation of serious games, like other educational tools, is user performance assessment. This is an important area of exploration because serious games are intended to evaluate the learning progress as well as the outcomes. This also emphasizes the importance of providing appropriate feedback to the player. Moreover, performance assessment enables adaptivity and personalization to meet individual needs in various aspects, such as learning styles, information provision rates, feedback, and so forth. This paper first reviews related literature regarding the educational effectiveness of serious games. It then discusses how to assess the learning impact of serious games and methods for competence and skill assessment. Finally, it suggests two major directions for future research: characterization of the players activity and better integration of assessment in games.

Mapping learning and game mechanics for serious games analysis

Although there is a consensus on the instructional potential of Serious Games (SGs), there is still a lack of methodologies and tools not only for design but also to support analysis and assessment. Filling this gap is one of the main aims of the Games and Learning Alliance (http://www.galanoe.eu) European Network of Excellence on Serious Games, which has a focus upon pedagogy-driven SGs. This paper relies on the assumption that the fundamental aspect of SG design consists in the translation of learning goals/practices into mechanical element of gameplay, serving to an instructional purpose beside that of play and fun. This paper proposes the Learning Mechanics–Game Mechanics (LM-GM) model, which supports SG analysis and design by allowing reflection on the various pedagogical and game elements in an SG. The LM-GM model includes a set of pre-defined game mechanics and pedagogical elements that we have abstracted from literature on game studies and learning theories. Designers and analysts can exploit these mechanics to draw the LM-GM map for a game, so as to identify and highlight its main pedagogical and entertainment features, and their interrelations. The tool may also be useful for teachers to evaluate the effectiveness of a given game and better understand how to implement it in educational settings. A case study is reported to illustrate the frameworks support in determining how gameplay and pedagogy intertwine in an SG. Finally, the paper presents the results of two comparative user tests demonstrating the advantages of the proposed model with respect to a similar state-of-the-art framework.

Platform for distributed 3D gaming

Video games are typically executed on Windows platforms with DirectX API and require high performance CPUs and graphics hardware. For pervasive gaming in various environments like at home, hotels, or internet cafes, it is beneficial to run games also on mobile devices and modest performance CE devices avoiding the necessity of placing a noisy workstation in the living room or costly computers/consoles in each room of a hotel. This paper presents a new cross-platform approach for distributed 3D gaming in wired/wireless local networks. We introduce the novel system architecture and protocols used to transfer the game graphics data across the network to end devices. Simultaneous execution of video games on a central server and a novel streaming approach of the 3D graphics output to multiple end devices enable the access of games on low cost set top boxes and handheld devices that natively lack the power of executing a game with high-quality graphical output.

Designing Effective Serious Games: Opportunities and Challenges for Research

Serious Games represent an acknowledged potential for instruction, because they are able to strongly motivate learners. They can also provide immersive environments where advanced users can practice knowledge and skills, also exploiting multimodal interaction. They can combine the effectiveness of computer processing and data storage, with high levels of attractiveness. Our work has investigated the state of the art research on SGs, starting from the cognitive aspects, that are necessary in order to root technological development and applications in sound theoretical foundations. The paper discusses some key aspects about SG design and exploitation: choice of components-off-the-shelf or from-scratch design, tools and methodologies for development or adaptation, intelligent tutoring, virtual coaches and affective learning, living worlds, game mechanics, Human-Computer Interaction. While several SGs have been developed, still the literature stresses a lack of significant, extensive user tests. Further research is necessary to investigate in greater detail the real effectiveness of the various types of SGs. The paper proposes several research questions - that range from requirements elicitation to design and from deployment to use and evaluation - to be answered in order to avoid technology pushing and drive technological research according to the requirements of the end-users and stakeholders. We believe that deepening the analysis about these issues is key to strengthen the foundations of SG research, for which we identify four major directions: definition of metrics and learning progress evaluation tools; methodologies and tools for designing games from various topics and for various users; computing and communication architectures; technologies that can enhance the overall system performance.

Enhancing the educational value of video games

Lowering the barrier between education and real entertainment is an important challenge in order to better exploit the potential of computers and reach a demographic that is traditionally averse to learning. To this end, it is important to investigate how to exploit the appeal of video games (VGs) to also favor and induce learning via playing video games. Achieving this goal is not only a matter of content, since simply “superimposed” educational content risks being perceived as boring. Hence we believe that the game should feature mechanisms for acquiring knowledge and skill that are smoothly embedded in a meaningful, homogeneous, and compelling whole. Thus, there is a need to compartmentalize components of a game engine so that it becomes easy and efficient to integrate the graphics/interface—which has already been done very well by state-of-the-art successful video games and the educational aspect which is typically poor in those same games. Hence we have defined a general set of mechanisms and modules that can be inserted in state-of-the-art VG environments and are aimed at promoting various kinds of knowledge and procedural skill acquisition. In order to investigate and validate this concept, we have built an educational game, SeaGame, using a state-of-the-art commercial game development approach, and enriched the environment with instances of developed educational modules. Analyzing user test results, we conclude that SeaGame is perceived quite similarly to commercial VGs, which suggests that the proposed mechanisms do not compromise the overall enjoyability of the game, which is key to attracting a wide demographic that is not currently involved in educational activities during their leisure time. The results of this research can be generalized, since the standards of commercial games and the proposed educational enhancements can be instantiated in a variety of educational contexts and applied to different types of content.

An activity theory-based model for serious games analysis and-conceptual design

There are currently a number of models, frameworks and methodologies for serious games analysis and design that provide useful interpretations of the possibilities and limitations offered by serious games. However, these tools focus mostly on high-level aspects and requirements and do not help understand how such high-level requirements can be concretely satisfied. In this paper, we present a conceptual model, called Activity Theory-based Model of Serious Games (ATMSG), that aims to fill this gap. ATMSG supports a systematic and detailed representation of educational serious games, depicting the ways that game elements are connected to each other throughout the game, and how these elements contribute to the achievement of the desired pedagogical goals. Three evaluation studies indicate that ATMSG helped participants, particularly those with gaming experience, identify and understand the roles of each component in the game and recognize the games educational objectives. We present a model for serious games analysis and conceptual design, called ATMSG.Activity theory is used as the underlying theoretical framework.ATMSG supports a systematic and detailed representation of educational serious games.ATMSG depicts how the combination of serious games elements supports pedagogical goals.A taxonomy of serious games elements is presented to assist the analysis of SGs.

Supporting authors in the development of task-based learning in serious virtual worlds

Serious virtual worlds (SVWs) represent a great opportunity for learning and should embed a lot of high-quality contextualised information so that the player can take the most from his or her exploration of the environment. This requires defining new methodologies and tools for effective production. We have investigated this issue in implementing a couple of SVWs and abstracted a conceptual framework relying on the task-based learning pedagogical theory. The model defines games set in realistic SVWs enriched with embedded educational tasks. Tasks are simple trial activities that embody units of knowledge. The player can discover them in his or her SVW exploration and interact with them in order to construct meaning, build lasting memories and deepen understanding of the featured item(s). The model involves pedagogical task annotation—which allows decoupling the tasks, which can be reused in different VWs from the definition of their delivery strategy in the context of a specific VW, which is specified by the VW designer and automatically managed by the run-time engine. This approach simplifies the authoring work. The visual Creative Toolkit (CT) we have developed is being used for producing contents for the 15 reconstructed cultural cities that will be featured in the first version of the Travel in Europe VW. The paper described the overall framework and the details of the CT. It also presented an implemented example, briefly discussing the methodology we have followed for developing cultural heritage content. [ABSTRACT FROM AUTHOR]

Towards the Automotive HMI of the Future: Overview of the AIDE-Integrated Project Results

The Adaptive Integrated Driver-vehicle interfacE (AIDE) is an integrated project funded by the European Commission in the Sixth Framework Programme. The project, which involves 31 partners from the European automotive industry and academia, deals with behavioral and technical issues related to automotive human-machine interface (HMI) design, with a particular focus on integration and adaptation. The project involves tightly integrated empirical research, driver-behavior modeling, and methodological and technological development. This paper provides an overview of the AIDE Sub-Project 3 results dealing with the design, development, and integration of the AIDE system in three prototype vehicles, together with the evaluation results of the trials.

Designing a Course for Stimulating Entrepreneurship in Higher Education through Serious Games

Enhancing the offer for entrepreneurship education is an important challenge for the nowadays knowledge societies. The eSG project is addressing this issue by analysing the added value that could be contributed by employing serious games (SGs) as a tool for allowing students – in particular technology students - to become familiar, mainly through practice, with basic concepts of entrepreneurship and company management. This paper presents the main requirements for the course and SGs obtained by surveying literature, entrepreneurs, students and teachers. We represented the requirements in a table template keeping into account usability, pedagogy, the entrepreneurship skills expressed by state of the art models and three major axes for entrepreneurship education at universities. These table descriptors were then used to assess validity of SGs and choose an appropriate mix for the courses. We have also defined a set of metrics to evaluate the advancement of students during the course. Based on these tools and knowledge, the next steps of the project will involve extensive user testing in the actual courses that are being performed in Genoa, Delft and Barcelona.

User Assessment in Serious Games and Technology-Enhanced Learning

1 Department of Naval, Electric, Electronic and Telecommunications Engineering, University of Genoa, Via all’Opera Pia 11/a, 16145 Genoa, Italy 2 Faculty of Business and Information Technology, University of Ontario Institute of Technology, 2000 Simcoe Street North, Oshawa, Canada L1H 7K4 3Department of Mechanical and Aerospace Engineering, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106, USA 4 Faculty of Computer Science, Universidad Complutense de Madrid, Ciudad Universitaria Universidad Complutense de Madrid, 28040 Madrid, Spain