Giani Petri

How games for computing education are evaluated? A systematic literature review

Educational games are assumed to be an effective and efficient instructional strategy for computing education. However, it is essential to systematically evaluate such games in order to obtain sound evidence of their impact. Thus, the objective of this article is to present the state of the art on how games for computing education are evaluated. Therefore, we performed a systematic literature review of a sample of 3617 articles from which 112 relevant articles have been identified, describing 117 studies on the evaluation of games for computing education. Based on these studies we analyzed how evaluations are defined (the analysis factors evaluated, research designs, evaluation models/methods used, kind of data collection instruments, etc.), how they have been executed (sample size and replications) and analyzed (data analysis methods used). As a result, we can confirm that most evaluations use a simple research design in which, typically, the game is used and afterwards subjective feedback is collected via questionnaires from the learners. The majority of the evaluations are run with small samples, without replication, using mostly qualitative methods for data analysis. We also observed that most studies do not use a well-defined evaluation model or method. This shows that there is a need for more rigorous evaluations as well as methodological support in order to assist game creators and instructors to improve such games as well as to systematically support decisions on when or how to include them within instructional units. Exploring how games for computing education are evaluated.No consensus on which quality factors are evaluated.Most evaluations are performed in an ad-hoc manner lacking scientific rigor.Models developed for evaluating educational games: MEEGA and EGameFlow.Need for more rigorous evaluations as well as methodological support.

Quality of games for teaching software engineering: an analysis of empirical evidences of digital and non-digital games

Educational games have been used as an innovative instructional strategy in order to achieve learning more effectively in Software Engineering (SE) education. However, it is essential to systematically evaluate such games in order to obtain sound evidence on their benefits. And, although, several SE games have been evaluated adopting diverse research designs and measurements, so far no larger scale study across different games has been conducted. In this respect, this article presents a comprehensive analysis in order to summarize empirical evidence on the benefits of digital and non-digital games used for SE education. The analysis is based on data collected from 43 case studies that use MEEGA, the most commonly used model for educational game evaluation, evaluating 20 different SE games, involving a total population of 723 students. Our analysis indicate evidence that digital and non-digital games can yield a positive effect on the learning of SE, providing a pleasant and engaging experience to the students and motivate them. Our analysis also points out that non-digital games more easily seem to promote a more positive experience, principally in terms of fun and social interaction than the digital ones used for teaching SE. These results may guide SE instructors in the selection of educational games as instructional methods and guide game creators with respect to the development of new games.

A large-scale evaluation of a model for the evaluation of games for teaching software engineering

In order to adopt games for Software Engineering (SE) education effectively it is essential to obtain sound evidence on their quality. A prominent evaluation model is MEEGA (Model for the Evaluation of Educational Games), which provides a systematic support to evaluate the games quality in terms of motivation, user experience and learning. To facilitate its application, the model provides a questionnaire for collecting data on the perception of the students after they played an educational game in a case study with a one-shot post-test design. However, in order to assure a valid feedback on the games quality an important issue is the reliability and validity of the questionnaire. In this respect, this article presents a large-scale evaluation of the MEEGA questionnaire in terms of reliability and construct validity. The analysis is based on data collected in 43 case studies, evaluating 20 different SE games, involving a population of 723 students. Our analysis indicates that the MEEGA questionnaire can be considered reliable (Cronbachs alpha α=.915). In terms of construct validity, there exists evidence of convergent validity through an acceptable degree of correlation of almost all item pairs within each dimension. Yet, we identified a need for the re-grouping of items based on the results of a factor analysis, mainly with respect to items related to motivation and user experience. These results allow SE researchers and instructors to rely on the MEEGA questionnaire in order to evaluate SE games and, thus, contribute to their improvement and to direct an effective and efficient adoption for SE education.

Qualidade de jogos digitais e não digitais utilizados para o ensino de engenharia de software no Brasil

Educational games have been used as an innovative strategy for Software Engineering (SE) education in Brazil. However, it is essential to evaluate such games in order to obtain sound evidence on their quality. Thus, the objective of this article is to summarize empirical evidence on the quality of digital and non-digital games used for teaching SE in Brazil. The analysis is based on data collected from 41 case studies that used Meega, the most used model for the evaluation of educational games, evaluating 19 games, involving a population of 704 students. The results indicate that digital and non-digital games can have a positive effect on learning by providing a pleasant and engaging experience for students and motivating them to study. These results can guide instructors in the selection of games as instructional methods and guide game creators toward developing new educational games. Key words: quality evaluation, educational games, software engineering.

UM ROLE PLAYING GAME PARA O ENSINO DE ELICITAÇÃO E ANÁLISE DE REQUISITOS

A engenharia de requisitos objetiva estabelecer o entendimento do problema e as necessidades do cliente. No entanto, o processo de ensino desta competencia nem sempre e satisfatorio, por ser um conteudo teorico, muitos professores utilizam somente aulas expositivas, onde acaba limitando o aprendizado dos alunos. Assim, surge a necessidade de explorar novos metodos de ensino, como os jogos educacionais, de modo a criar um ambiente que desperte o interesse dos alunos. O objetivo deste trabalho e desenvolver e avaliar um jogo educacional do tipo RPG no qual o jogador atuara como um engenheiro de requisitos coletando e priorizando os requisitos em um cenario simulado. Uma avaliacao sistematica realizada aponta resultados satisfatorios quanto a motivacao, experiencia do usuario e aprendizagem dos alunos.