Brent Martin

A comparative analysis of cognitive tutoring and constraint-based modeling

Numerous approaches to student modeling have been proposed since the inception of the field more than three decades ago. What the field is lacking completely is comparative analyses of different student modeling approaches. In this paper we compare Cognitive Tutoring to Constraint-Based Modeling (CBM). We present our experiences in implementing a database design tutor using both methodologies and highlight their strengths and weaknesses. We compare their characteristics and argue the differences are often more apparent than real: for specific domains one approach may be favoured over the other, making them viable complementary methods for supporting learning.

Constraint-Based Tutors: A Success Story

Student modeling (SM) is recognized as one of the central problems in the area of Intelligent Tutoring Systems. Numerous SM approaches have been proposed and used with more or less success. Constraint-based modeling is a new approach, which has been used successfully in three tutors developed in our group. The approach is extremely efficient, and it overcomes many problems that other student modelling approaches suffer from. We present the advantages of CBM over other similar approaches, describe three constraint-based tutors and present our future research plans.

Intelligent Tutors for All: The Constraint-Based Approach

This paper presents a new type of intelligent tutoring systems, called constraint-based tutors. The system have been thoroughly evaluated and proven to achieve significant learning gains.

Evaluating and improving adaptive educational systems with learning curves

Personalised environments such as adaptive educational systems can be evaluated and compared using performance curves. Such summative studies are useful for determining whether or not new modifications enhance or degrade performance. Performance curves also have the potential to be utilised in formative studies that can shape adaptive model design at a much finer level of granularity. We describe the use of learning curves for evaluating personalised educational systems and outline some of the potential pitfalls and how they may be overcome. We then describe three studies in which we demonstrate how learning curves can be used to drive changes in the user model. First, we show how using learning curves for subsets of the domain model can yield insight into the appropriateness of the model’s structure. In the second study we use this method to experiment with model granularity. Finally, we use learning curves to analyse a large volume of user data to explore the feasibility of using them as a reliable method for fine-tuning a system’s model. The results of these experiments demonstrate the successful use of performance curves in formative studies of adaptive educational systems.

Authoring constraint-based tutors in ASPIRE

This paper presents a project the goal of which is to develop ASPIRE, a complete authoring and deployment environment for constraint-based intelligent tutoring systems (ITSs). ASPIRE is based on our previous work on constraint-based tutors and WETAS, the tutoring shell. ASPIRE consists of the authoring server (ASPIRE-Author), which enables domain experts to easily develop new constraint-base tutors, and a tutoring server (ASPIRE-Tutor), which deploys the developed systems. Preliminary evaluation shows that ASPIRE is successful in producing domain models, but more thorough evaluation is planned.

Evaluating the effectiveness of feedback in SQL-Tutor

We present an evaluation of various kinds of feedback in SQL-Tutor. Our initial hypothesis was that low-level feedback, containing all the details of a correct solution would be contra-productive, and that high-level feedback referring to the general principles of the domain that the students solution violates would be highly effective. The evaluation study performed in 1999 confirmed our hypothesis.

The Role of Domain Ontology in Knowledge Acquisition for ITSs

There have been several attempts to automate knowledge acquisition for ITSs that teach procedural tasks. The goal of our project is to automate the acquisition of domain models for constraint-based tutors for both procedural and non-procedural tasks. We propose a three-phase approach: building a domain ontology, acquiring syntactic constraints directly from the ontology, and engaging the author in a dialog, in order to induce semantic constraints using machine learning techniques. An ontology is arguably easier to create than the domain model. Our hypothesis is that the domain ontology is also useful for reflecting on the domain, so would be of great importance for building constraints manually. This paper reports on an experiment performed in order to test this hypothesis. The results show that constraints sets built using a domain ontology are superior, and the authors who developed the ontology before constraints acknowledge the usefulness of an ontology in the knowledge acquisition process.

WETAS: A Web-Based Authoring System for Constraint-Based ITS

Constraint-Based Modelling (CBM) is a student modelling technique for Intelligent Tutoring Systems (ITS) that is rapidly maturing. We have implemented several tutors using CBM, and demonstrated their suitability to, in particular, open-ended domains. It is easier to build tutors in some domains (e.g. open-ended) using CBM than other common approaches. We present WETAS (Web-Enabled Tutor Authoring System), a tutoring engine that facilitates the rapid implementation of ITS in new domains. We describe the architecture of WETAS, and give examples of two domains we have implemented.

Authoring web-based tutoring systems with WETAS

Constraint-based modelling (CBM) is a student modelling technique for intelligent tutoring systems (ITS) that is especially suited to complex, open-ended domains. It is easier to build tutors in such domains using CBM than other common approaches. The authors present WETAS (Web-Enabled Tutor Authoring System), a tutoring engine that facilitates the rapid implementation of ITS in new domains using CBM. They describe the architecture of WETAS and give examples of two domains they have implemented. They also present the results of an evaluation of a tutoring system built using WETAS in a New Zealand school.

Evaluating Adaptive Problem Selection

This paper presents an evaluation study that compares two different problem selection strategies for an Intelligent Tutoring System (ITS). The first strategy uses static problem complexities specified by the teacher to select problems that are appropriate for a student based on his/her current level of ability. The other strategy is more adaptive: individual problem difficulties are calculated for each student based on the student’s specific knowledge, and the appropriate problem is then selected based on these dynamic difficulty measures. The study was performed in the context of the SQL-Tutor system. The results show that adaptive problem selection based on dynamically generated problem difficulties can have a positive effect on student learning performance.