Leena M. Razzaq

A Comparison of Traditional Homework to Computer-Supported Homework

Abstract This study compared learning for fifth grade students in two math homework conditions. The paper-and-pencil condition represented traditional homework, with review of problems in class the following day. The Web-based homework condition provided immediate feedback in the form of hints on demand and step-by-step scaffolding. We analyzed the results for students who completed both the paper-and-pencil and the Web-based conditions. In this group of 28 students, students learned significantly more when given computer feedback than when doing traditional paper-and-pencil homework, with an effect size of .61. The implications of this study are that, given the large effect size, it may be worth the cost and effort to give Web-based homework when students have access to the needed equipment, such as in schools that have implemented one-to-one computing programs.

A Web-based Authoring Tool for Intelligent Tutors: Blending Assessment and Instructional Assistance

Middle school mathematics teachers are often forced to choose between assisting students’ development and assessing students’ abilities because of limited classroom time available. To help teachers make better use of their time, a web-based system, called the Assistment system, was created to integrate assistance and assessment by offering instruction to students while providing a more detailed evaluation of their abilities to the teacher than is possible under current approaches. An initial version of the Assistment system was created and used in May, 2004 with approximately 200 students and over 1000 students currently use it once every two weeks. The hypothesis is that Assistments can assist students while also assessing them. This chapter describes the Assistment system and some preliminary results.

Feedback during web-based homework: the role of hints

Prior work has shown that computer-supported homework can lead to better results over traditional paper-and-pencil homework. This study about learning from homework involved the comparison of immediate-feedback with tutoring versus a control condition where students got feedback the next day in math class. After analyzing eighth grade students who participated in both conditions, it was found that they gained significantly more (effect size 0.40) with computer-supported homework. This result has practical significance as it suggests an effective improvement over the widely used paper-and-pencil homework. The main result is followed with a second set of studies to better understand this result: is it due to the timeliness of feedback or quality tutoring?

Hints: is it better to give or wait to be asked?

Many tutoring systems allow students to ask for hints when they need help solving problems, and this has been shown to be helpful. However, many students have trouble knowing when to ask for help or they prefer to guess rather than ask for and read a hint. Is it better to give a hint when a student makes an error or wait until the student asks for a hint? This paper describes a study that compares giving hints proactively when students make errors to requiring students to ask for a hint when they want one. We found that students learned reliably more with hints-on-demand than proactive hints. This effect was especially evident for students who tend to ask for a high number of hints. There was not a significant difference between the two conditions for students who did not ask for many hints.

Towards designing a user-adaptive web-based e-learning system

This work-in-progress report presents the groundwork for the design of a user-adaptive web-based e-learning system. A survey and two randomized controlled experiments were carried out to compare the effects of active versus passive interaction on attitude and learning and to compare user vs. system initiated control of information presentation. Results showed that the more time-consuming active interaction was indeed more helpful to less-proficient students, but it was not as helpful to more-proficient students. Results also indicate that both more- and less-proficient students learn more from system initiated information presentation. These results will help to design a user-adaptive e-learning system that can determine which kind of interactivity and information presentation works best for which students and when.

Tutorial Dialog in an Equation Solving Intelligent Tutoring System

A new intelligent tutoring system is presented for the domain of solving equations. This system is novel, because it is an intelligent equation-solving tutor that combines a cognitive model of the domain with a model of dialog-based tutoring. The tutorial model is based on the observation of an experienced human tutor and captures tutorial strategies specific to the domain of equation-solving. In this context, a tutorial dialog is the equivalent of breaking down problems into simpler steps and asking new questions before proceeding to the next step. The resulting system, named E-tutor, was compared, via a randomized controlled experiment, to a traditional model-tracing tutor that does not engage students in dialog. Preliminary results using a very small sample size showed that E-tutor capabilities performed better than the control. This set of preliminary results, though not statistically significant, shows promising opportunities to improve learning performance by adding tutorial dialog capabilities to ITSs. The system is available at www.wpi.edu/~leenar/E-tutor.

Open Content Authoring Tools

Education researchers often disagree about the best ways to improve student achievement. The difficulty of designing, conducting, and analyzing experiments means that there is often a dearth of empirical data to support or refute ideas. To design and conduct a simple randomized controlled experiment to compare two different ways of teaching requires a great deal of effort by a teacher or a researcher. The difficulty of conducting such experiments, and then later analyzing the results, may be why so few randomized controlled experiments are conducted in education. One of the goals of the ASSISTment System is to reduce some of those difficulties. We have built web-based tools that allow researchers to easily design, build and then compare different ways to teach children. These tools can administer randomized controlled experiments to large numbers of students. This paper describes these tools and describes a randomized controlled study that was conducted using them.

4MALITY: coaching students with different problem-solving strategies using an online tutoring system

4-coach Mathematics Active Learning Intelligent Tutoring sYstem (4MALITY) is a web-based intelligent tutoring system for 3rd, 4th, and 5th grade students who are learning math content from the state of Massachusetts (USA) required curriculum framework. The goal of 4MALITY is to personalize help for students by offering them problem-solving strategies authored from multiple points of view. Four virtual coaches (Estella Explainer, Chef Math Bear, How-to Hound, and Visual Vicuna) are designed to capture the character and content of these different problem-solving approaches with language, computation, strategy, and visual hints. A preliminary study was run with 102 students in fourth and fifth grade math classrooms over a period of two months. The results showed that the effect of using 4MALITY produced a statistically significant increase in post-test scores. We explored student performance, help-seeking behavior and meta-cognitive strategies by gender and math ability and report these results.