Behrooz Mostafavi

Towards data-driven mastery learning

We have developed a novel data-driven mastery learning system to improve learning in complex procedural problem solving domains. This new system was integrated into an existing logic proof tool, and assigned as homework in a deductive logic course. Student performance and dropout were compared across three systems: The Deep Thought logic tutor, Deep Thought with integrated hints, and Deep Thought with our data-driven mastery learning system. Results show that the data-driven mastery learning system increases mastery of target tutor-actions, improves tutor scores, and lowers the rate of tutor dropout over Deep Thought, with or without provided hints.

Evolution of an Intelligent Deductive Logic Tutor Using Data-Driven Elements

Deductive logic is essential to a complete understanding of computer science concepts, and is thus fundamental to computer science education. Intelligent tutoring systems with individualized instruction have been shown to increase learning gains. We seek to improve the way deductive logic is taught in computer science by developing an intelligent, data-driven logic tutor. We have augmented Deep Thought, an existing computer-based logic tutor, by adding data-driven methods, specifically; intelligent problem selection based on the student’s current proficiency, automatically generated on-demand hints, and determination of student problem solving strategies based on clustering previous students. As a result, student tutor completion (the amount of the tutor the students completed) steadily improved as data-driven methods were added to Deep Thought, allowing students to be exposed to more logic concepts. We also gained additional insights into the effects of different course work and teaching methods on tutor effectiveness.

Data-Driven Worked Examples Improve Retention and Completion in a Logic Tutor

Research shows that expert-crafted worked examples can have a positive effect on student performance. To investigate the potential for data-driven worked examples to achieve similar results, we generated worked examples for the Deep Thought logic tutor, and conducted an experiment to assess their impact on performance. Students who received data-driven worked examples were much more likely to complete the tutor, and completed the tutor in less time. This study demonstrates that worked examples, automatically generated from student data, can be used to improve student learning in tutoring systems.

Combining Worked Examples and Problem Solving in a Data-Driven Logic Tutor

Previous research has shown that worked examples can increase learning efficiency during computer-aided instruction, especially when alternatively offered with problem solving opportunities. In this study, we investigate whether these results are consistent in a complex, open-ended problem solving domain, where students are presented with randomly ordered sets of worked examples and required problem solving. Our results show that worked examples benefits students early in tutoring sessions, but are comparable to hint-based systems for scaffolding domain concepts. Later in tutoring sessions, worked examples are less beneficial, and can decrease performance for lower-proficiency students.

Data-driven proficiency profiling: proof of concept

Data-driven methods have previously been used in intelligent tutoring systems to improve student learning outcomes and predict student learning methods. We have been incorporating data-driven methods for feedback and problem selection into Deep Thought, a logic tutor where students practice constructing deductive logic proofs. In this latest study we have implemented our data-driven proficiency profiler (DDPP) into Deep Thought as a proof of concept. The DDPP determines student proficiency without expert involvement by comparing relevant student rule scores to previous students who behaved similarly in the tutor and successfully completed it. The results show that the DDPP did improve in performance with additional data and proved to be an effective proof of concept.

Towards the creation of a data-driven programming tutor

Educational data mining methods are being used to automatically generate hints to students in intelligent tutoring systems Using these methods, we hope to create a system that can give individualized instruction By analyzing time snapshot data from exams in an introductory programming course, we will write a program to construct state graphs for each students performance, eventually resulting in a Markov decision process that represents different approaches to writing the target program, and providing feedback to students Once this system is sufficiently tested and refined, it will then be applied to subsequent semesters students in the programming course.

Investigation of the Influence of Hint Type on Problem Solving Behavior in a Logic Proof Tutor

Within intelligent tutoring systems, hint policies are needed to determine when and how to give hints and what type of hint is most beneficial. In this study, we focus on discovering whether certain hint types influence problem solving behavior. We investigate the influence of two hint types (next-step hints and more abstract high-level hints) on students’ behavior in a college-level logic proof tutor, Deep Thought. The results suggest that hint types can affect student behavior, including hint usage, rule applications, and time in-tutor.

Empirically Evaluating the Effectiveness of POMDP vs. MDP Towards the Pedagogical Strategies Induction.

The effectiveness of Intelligent Tutoring Systems (ITSs) often depends upon their pedagogical strategies, the policies used to decide what action to take next in the face of alternatives. We induce policies based on two general Reinforcement Learning (RL) frameworks: POMDP&. MDP, given the limited feature space. We conduct an empirical study where the RL-induced policies are compared against a random yet reasonable policy. Results show that when the contents are controlled to be equal, the MDP-based policy can improve students’ learning significantly more than the random baseline while the POMDP-based policy cannot outperform the later. The possible reason is that the features selected for the MDP framework may not be the optimal feature space for POMDP.