Tiffany Barnes

Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data

We have proposed a novel application of Markov decision processes (MDPs), a reinforcement learning technique, to automatically generate hints for an intelligent tutor that learns. We demonstrate the feasibility of this approach by extracting MDPs from four semesters of student solutions in a logic proof tutor, and calculating the probability that we will be able to generate hints at any point in a given problem. Our results indicate that extracted MDPs and our proposed hint-generating functions will be able to provide hints over 80% of the time. Our results also indicate that we can provide valuable tradeoffs between hint specificity and the amount of data used to create an MDP.

Experimental evaluation of an educational game for improved learning in introductory computing

We are developing games to increase student learning and attitudes in introductory CS courses. Wus Castle is a game where students program changes in loops and arrays in an interactive, visual way. The game provides immediate feedback and helps students visualize code execution in a safe environment. We compared the game to a traditional programming assignment in an introductory CS course. In our study, half of the students were randomly selected to play the learning game first and half to write a program first. Our results show that students who play our learning game first outperform those who write a program before playing the game. Students in the game-first group felt they spent less time on the assignments, and all students preferred the learning game over the program. These results suggest that games like Wus Castle can help prepare students to create deeper, more robust understanding of computing concepts while improving their perceptions of computing homework assignments.

Game2Learn: building CS1 learning games for retention

This paper presents Game2Learn, an innovative project designed to leverage games in retaining students in computer science (CS). In our two-pronged approach, students in integrative final-year capstone courses and summer research experiences develop games to teach computer science, which, in turn, will be used to improve introductory computing courses. Our successful model for summer undergraduate research and capstone projects engages students in solving the computing retention problem, allows them to quickly create games, and instructs students in user- and learner-centered design and research methods. Results show that this method of building games to teach engages students at multiple levels, inspiring newer students that one day their homework may all be games, and encouraging advanced students to continue on into graduate studies in computing.

An integrated scenario management strategy

Scenarios have proven effective for eliciting, describing and validating software requirements; however, scenario management continues to be a significant challenge to practitioners. One reason for this difficulty is that the number of possible relations among scenarios grows exponentially with the number of scenarios. If these relations are formalized, they can be more easily identified and supported. To provide this support, we extend the benefits of project-wide glossaries with two complementary approaches. The first approach employs shared scenario elements to identify and maintain common episodes among scenarios. The resulting episodes impose consistency across related scenarios and provide a way to visualize their interdependencies. The second approach quantifies similarity between scenarios. The resulting similarity measures serve as heuristics for finding duplicate scenarios, scenarios needing further elaboration, and scenarios which have not yet been identified yielding valuable information about how well the scenarios provide coverage of the requirements. These two approaches, integrated with a scenario database, project glossaries, configuration management, and coverage analysis, form the basis of a useful and effective strategy for scenario management and evolution.

Experimental Evaluation of Automatic Hint Generation for a Logic Tutor

We have augmented the Deep Thought logic tutor with a Hint Factory that generates data-driven, context-specific hints for an existing computer aided instructional tool. We investigate the impact of the Hint Factorys automatically generated hints on educational outcomes in a switching replications experiment that shows that hints help students persist in a deductive logic proofs tutor. Three instructors taught two semester-long courses, each teaching one semester using a logic tutor with hints, and one semester using the tutor without hints, controlling for the impact of different instructors on course outcomes. Our results show that students in the courses using a logic tutor augmented with automatically generated hints attempted and completed significantly more logic proof problems, were less likely to abandon the tutor, performed significantly better on a post-test implemented within the tutor, and achieved higher grades in the course.

Game2Learn: improving the motivation of CS1 students

Games are increasingly being used for education and training in a variety of areas. We are developing a game to teach introductory computer science concepts, called Game2Learn, to increase student motivation and engagement in learning CS1, which are critical for recruiting students into computer science. We evaluated student feedback and performance of initial prototypes to examine the Game2Learn concept and provide design guidelines for ongoing game development. In this paper, we present the results of this study, which demonstrate that students can have fun programming within a game, and that in-game rewards and punishments are vital to the motivation and potential learning of students.

Experimental evaluation of teaching recursion in a video game

We present EleMental: The Recurrence, a novel game that provides computer science students the opportunity to write code and perform interactive visualizations to learn about recursion through depth-first search of a binary tree. We designed the game to facilitate maximum transfer of learning to writing real programs, while also providing for interactive visualizations. We conducted a study with computer science majors to measure the impact of the game on learning and on attitudes toward educational games. Our results demonstrate the enthusiasm students have for learning games and provide insight into how such games should be constructed.

Program representation for automatic hint generation for a data-driven novice programming tutor

We describe a new technique to represent, classify, and use programs written by novices as a base for automatic hint generation for programming tutors. The proposed linkage graph representation is used to record and reuse student work as a domain model, and we use an overlay comparison to compare in-progress work with complete solutions in a twist on the classic approach to hint generation. Hint annotation is a time consuming component of developing intelligent tutoring systems. Our approach uses educational data mining and machine learning techniques to automate the creation of a domain model and hints from student problem-solving data. We evaluate the approach with a sample of partial and complete, novice programs and show that our algorithms can be used to generate hints over 80 percent of the time. This promising rate shows that the approach has potential to be a source for automatically generated hints for novice programmers.

Can Immersive Virtual Humans Teach Social Conversational Protocols

We investigated the effects of using immersive virtual humans to teach users social conversational verbal and non-verbal protocols in south Indian culture. The study was conducted using a between-subjects experimental design, and compared instruction and interactive feedback from immersive virtual humans against instruction based on a written study guide with illustrations of the social protocols. Participants were then tested on how well they learned the social conversational protocols by exercising the social conventions in front of videos of real people. The results of our study suggest that participants who trained with the virtual humans performed significantly better than the participants who studied from literature.

Wu's castle: teaching arrays and loops in a game

We are developing games to teach introductory computer science concepts to increase student motivation and engagement in learning to program. Wus Castle is a two-dimensional role playing game that teaches loops and arrays in an interactive, visual way. In this game, the player interactively programs magical creatures to create armies of snowmen. The game provides immediate feedback and helps students visualize the execution of their code in a safe environment. We tested the game in a CS1 course, where students could earn extra credit to play Wus Castle. Our results show learning gains for game players, compared both through pre- and post-tests differences and improved performance on relevant final exam questions when compared to students who did not play the game. The results of this study suggest that Wus Castle implements good practices for teaching programming within a game.