Min Chi

Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies

For many forms of e-learning environments, the system’s behavior can be viewed as a sequential decision process wherein, at each discrete step, the system is responsible for selecting the next action to take. Pedagogical strategies are policies to decide the next system action when there are multiple ones available. In this project we present a Reinforcement Learning (RL) approach for inducing effective pedagogical strategies and empirical evaluations of the induced strategies. This paper addresses the technical challenges in applying RL to Cordillera, a Natural Language Tutoring System teaching students introductory college physics. The algorithm chosen for this project is a model-based RL approach, Policy Iteration, and the training corpus for the RL approach is an exploratory corpus, which was collected by letting the system make random decisions when interacting with real students. Overall, our results show that by using a rather small training corpus, the RL-induced strategies indeed measurably improved the effectiveness of Cordillera in that the RL-induced policies improved students’ learning gains significantly.

Implicit versus explicit learning of strategies in a non-procedural cognitive skill

University physics is typical of many cognitive skills in that there is no standard procedure for solving problems, and yet a few students still master the skill. This suggests that their learning of problem solving strategies is implicit, and that an effective tutoring system need not teach problem solving strategies as explicitly as model-tracing tutors do. In order to compare implicit vs. explicit learning of problem solving strategies, we developed two physics tutoring systems, Andes and Pyrenees. Pyrenees is a model-tracing tutor that teaches a problem solving strategy explicitly, whereas Andes uses a novel pedagogy, developed over many years of use in the field, that provides virtually no explicit strategic instruction. Preliminary results from an experiment comparing the two systems are reported.

Detecting Opinion Spammer Groups Through Community Discovery and Sentiment Analysis

In this paper we investigate on detection of opinion spammer groups in review systems. Most existing approaches typically build pure content-based classifiers, using various features extracted from review contents; however, spammers can superficially alter their review contents to avoid detections. In our approach, we focus on user relationships built through interactions to identify spammers. Previously, we revealed the existence of implicit communities among users based upon their interaction patterns [3]. In this work we further explore the community structures to distinguish spam communities from non-spam ones with sentiment analysis on user interactions. Through extensive experiments over a dataset collected from Amazon, we found that the discovered strong positive communities are more likely to be opinion spammer groups. In fact, our results show that our approach is comparable to the existing state-of-art content-based classifier, meaning that our approach can identify spammer groups reliably even if spammers alter their contents.

Inducing effective pedagogical strategies using learning context features

Effective pedagogical strategies are important for e-learning environments While it is assumed that an effective learning environment should craft and adapt its actions to the users needs, it is often not clear how to do so In this paper, we used a Natural Language Tutoring System named Cordillera and applied Reinforcement Learning (RL) to induce pedagogical strategies directly from pre-existing human user interaction corpora 50 features were explored to model the learning context Of these features, domain-oriented and system performance features were the most influential while user performance and background features were rarely selected The induced pedagogical strategies were then evaluated on real users and results were compared with pre-existing human user interaction corpora Overall, our results show that RL is a feasible approach to induce effective, adaptive pedagogical strategies by using a relatively small training corpus Moreover, we believe that our approach can be used to develop other adaptive and personalized learning environments.

Can Diagrams Predict Essay Grades

Diagrammatic models of argument have grown in prominence in recent years. While they have been applied in a number of tutoring contexts, it has not yet been shown that student-produced diagrams can be used to effectively grade students or predict their future performance. We show that manually-assigned diagram grades and automatic structural features of argument diagrams can be used to predict students’ future essay grades, thus supporting the use of argument diagrams for instruction. We also show that the automatic features are competitive with expert human grading despite the fact that semantic content was ignored in automatic processing.

When Is Tutorial Dialogue More Effective Than Step-Based Tutoring?

It is often assumed that one-on-one dialogue with a tutor, which involves micro-steps, is more effective than conventional step-based tutoring. Although earlier research often has not supported this hypothesis, it may be because tutors often are not good at making micro-step decisions. In this paper, we compare a micro-step based NL-tutoring system that employs induced pedagogical policies, Cordillera, to a well-evaluated step-based ITS, Andes. Our overall conclusion is that the pairing of effective policies with a micro-step based system does significantly outperform a step-based system; however, there is no significant difference in the absence of effective policies. Moreover, while micro-step tutoring is more time-consuming, the findings still hold for five out of six learning performance measures when time on task is factored out.

Data-Driven Worked Examples Improve Retention and Completion in a Logic Tutor

Research shows that expert-crafted worked examples can have a positive effect on student performance. To investigate the potential for data-driven worked examples to achieve similar results, we generated worked examples for the Deep Thought logic tutor, and conducted an experiment to assess their impact on performance. Students who received data-driven worked examples were much more likely to complete the tutor, and completed the tutor in less time. This study demonstrates that worked examples, automatically generated from student data, can be used to improve student learning in tutoring systems.

Adaptive Expertise as Acceleration of Future Learning A Case Study

This chapter begins with an extensive examination of the various ways that adaptive expertise can be measured. Most of them have fairly well-known theoretical explanations, which are reviewed briefl y. On the other hand, theoretical explanations are not easily found for one particularly valuable manifestation of adaptive expertise: acceleration of future learning. Acceleration of future learning is valuable because the growth of knowledge anticipated for the twenty-fi rst-century demands that experts be able to learn new task domains quickly. That is, their training now should raise their learning rates later : It accelerates their future learning. We present a case study where accelerated future learning was achieved. The trick was to use an intelligent tutoring system that focused students on learning domain principles. Students in this condition of the experiment apparently realized that principles were more easily learned and more effective than problem schemas, analogies, and so forth. Thus, when given the freedom to choose their own learning strategy while learning a second task domain, they seem to have focused on the principles of the new task domain. This caused them to learn faster than the control group, who were not focused on principles during their instruction on the initial task domain. In short, the metacognitive learning strategy/policy of focusing on principles seems to have transferred from one domain (probability) to another (physics), thus causing accelerated future learning of the second task domain (physics).

Intervention-BKT: Incorporating Instructional Interventions into Bayesian Knowledge Tracing

Bayesian Knowledge Tracing BKT is one of the most widely adopted student modeling methods in Intelligent Tutoring Systems ITSs. Conventional BKT mainly leverages sequences of observations e.g. correct, incorrect from student-system interaction log files to infer student latent knowledge states e.g. unlearned, learned. However, the model does not take into account the instructional interventions that generate those observations. On the other hand, we hypothesized that various types of instructional interventions can impact students latent states differently. Therefore, we proposed a new student model called Intervention-Bayesian Knowledge Tracing Intervention-BKT. Our results showed the new model outperforms conventional BKT and two factor analysis based alternatives: Additive Factor Model AFM and Instructional Factor Model IFM; moreover, the learned parameters of Intervention-BKT can recommend adaptive pedagogical policies.

Applying cognitive developmental psychology to middle school physics learning: The rule assessment method

Cognitive developmental psychology often describes children’s growing qualitative understanding of the physical world. Physics educators may be able to use the relevant methods to advantage for characterizing changes in students’ qualitative reasoning. Siegler developed the “rule assessment” method for characterizing levels of qualitative understanding for two factor situations (e.g., volume and mass for density). The method assigns children to rule levels that correspond to the degree they notice and coordinate the two factors. Here, we provide a brief tutorial plus a demonstration of how we have used this method to evaluate instructional outcomes with middle-school students who learned about torque, projectile motion, and collisions using different instructional methods with simulations.