Moffat Mathews

Do your eyes give it away? using eye tracking data to understand students' attitudes towards open student model representations

There is sufficient evidence to show that allowing students to see their own student model is an effective learning and metacognitive strategy. Different tutors have different representations of these open student models, all varying in complexity and detail. EER-Tutor has a number of open student model representations available to the student at any particular time. These include skill meters, kiviat graphs, tag clouds, concept hierarchies, concept lists, and treemaps. Finding out which representation best helps the student at their level of expertise is a difficult task. Do they really understand the representation they are looking at? This paper looks at a novel way of using eye gaze tracking data to see if such data provides us with any clues as to how students use these representations and if they understand them.

Semantic Integration of Adaptive Educational Systems

With the growth of adaptive educational systems available to students, integration of these systems is evolving from an interesting research problem into an important practical task. One of the challenges that needs to be addressed is the development of mechanisms for student model integration. The architectural principles and representation technologies employed by adaptive educational systems define the applicability of a particular integration approach. This chapter reviews the existing mechanisms and details one of them: the evidence integration.

Detecting gaming the system in constraint-based tutors

Recently, detectors of gaming the system have been developed for several intelligent tutoring systems where the problem-solving process is reified, and gaming consists of systematic guessing and help abuse Constraint-based tutors differ from the tutors where gaming detectors have previously been developed on several dimensions: in particular, higher-level answers are assessed according to a larger number of finer-grained constraints, and feedback is split into levels rather than an entire help sequence being available at any time Correspondingly, help abuse behaviors differ, including behaviors such as rapidly repeating the same answer or blank answers to elicit answers We use text replay labeling in combination with educational data mining methods to create a gaming detector for SQL-Tutor, a popular constraint-based tutor This detector assesses gaming at the level of multiple-submission sequences and is accurate both at identifying gaming within submission sequences and at identifying how much each student games the system It achieves only limited success, however, at distinguishing different types of gaming behavior from each other.

Thermo-Tutor: An Intelligent Tutoring System for thermodynamics

We present the design and an evaluation of Thermo-Tutor, an Intelligent Tutoring System (ITS) that teaches thermodynamic cycles in closed systems. Thermo-Tutor provides opportunities for students to practice their skills by solving problems. When a student submits a solution, Thermo-Tutor analyzes it and provides appropriate feedback. We discuss the support for problem solving, and the student model the ITS maintains. An initial evaluation of Thermo-Tutor was performed at the University of Canterbury. The findings show that the ITS supports student learning effectively.

Modeling and studying gaming the system with educational data mining

In this chapter, we will discuss our work to understand why students game the system. This work leverages models of student gaming, termed “detectors”, which can infer student gaming in log files of student interaction with educational software. These detectors are developed using a combination of human observation and annotation, and educational data mining. We then apply the detectors to large data sets, and analyze the detectors’ predictions, using discovery with models methods, to study the factors associated with gaming behavior. Within this chapter, we will discuss the work to develop these detectors, and what we have discovered through these analyses based on these detectors. We will discuss evidence for how gaming the system impacts learning and evidence for why students choose to game. We will also discuss attempts to address gaming the system through adaptive scaffolding.

Evaluation of spatial abilities through tabletop AR

Research has been done into improving the means by which we organise and manage information. The usefulness of 2D versus 3D interfaces and environments has also been debated and evaluated. Human spatial abilities can be used to store more information about particular objects including their position in space. Our hypothesis states that as 3D objects contain more information about themselves and their relative position in space than 2D objects, although users take longer to process this information, they should be more accurate when searching and retrieving 3D objects. The evaluation study conducted compared spatial abilities between a 2D version of a memory game and an Augmented Reality (AR) version. Results showed that participants took significantly longer to complete the AR 3D version of the game than the 2D version, but did so with significantly fewer attempts i.e. they were more accurate. These results are specifically relevant for the design and development process of interfaces for AR applications.

Supporting Constructive Video-Based Learning: Requirements Elicitation from Exploratory Studies

Although videos are a highly popular digital medium for learning, video watching can be a passive activity and results in limited learning. This calls for interactive means to support engagement and active video watching. However, there is limited insight into what engagement challenges have to be overcome and what intelligent features are needed. This paper presents an empirical way to elicit requirements for innovative functionality to support constructive video-based learning. We present two user studies with an active video watching system instantiated for soft skill learning (pitch presentations). Based on the studies, we identify whether learning is happening and what kind of interaction contributes to learning, what difficulties participants face and how these can be overcome with additional intelligent support. Our findings show that participants who engaged in constructive learning have improved their conceptual understanding of presentation skills, while those who exhibited more passive ways of learning have not improved as much as constructive learners. Analysis of participants’ profiles and experiences led to requirements for intelligent support with active video watching. Based on this, we propose intelligent nudging in the form of signposting and prompts to further promote constructive learning.

Do Erroneous Examples Improve Learning in Addition to Problem Solving and Worked Examples

Learning from Problem Solving PS, Worked Examples WE and Erroneous Examples ErrEx have all proven to be effective learning strategies. However, there is still no agreement on what kind of assistance in terms of different learning activities should be provided to students in Intelligent Tutoring Systems ITSs to optimize learning. A previous study [1] found that alternating worked examples and problem solving AEP was superior to using just one type of learning tasks. In this paper, we compare AEP to a new instructional strategy which, in addition to PS and WEs, additionally offers erroneous examples to students. The results indicate that erroneous examples prepare students better for problem solving in comparison to worked examples. Explaining and correcting erroneous examples also leads to improved debugging and problem-solving skills.

The effects of domain and collaboration feedback on learning in a collaborative intelligent tutoring system

We present initial results from a study comparing the effects of domain and collaboration feedback on learning within COLLECT-UML, a collaborative problem-solving ITS. Using COLLECT-UML, two students in separate physical locations (a collaborative pair) construct UML class diagrams to solve problems together. In the default version, COLLECT-UML provides both domain and collaboration feedback. In this study however, collaborative pairs were randomly assigned to one of four modes (treatment conditions) which varied the feedback presented by the system: no feedback (NF), domain feedback only (DF), collaborative feedback only (CF), and both domain and collaborative feedback (DCF). All conditions improved significantly between pre- and post-test, showing that practicing within COLLECT-UML helps learning. At a surface level, collaborative pairs in all modes had similar amounts of collaboration. The DCF mode had significantly higher learning gains than the other modes, indicating the value of receiving both domain and collaborative feedback. Surprisingly, the CF mode had the lowest learning gains (lower than NF), suggesting that, in this case, good collaboration without domain feedback could have simply reinforced erroneous domain knowledge.

A Virtual Reality Environment for Rehabilitation of Prospective Memory in Stroke Patients

Prospective Memory (PM), or remembering to perform actions in the future, is of crucial importance for everyday life. This kind of memory is often impaired in stroke survivors and can interfere with independent living. We have developed a computer-based treatment which uses visual imagery to teach participants how to remember time- and event-based prospective memory tasks better. After the treatment, participants practiced their PM skills using videos first, and later in a Virtual Reality (VR) environment. The VR environment uses Constraint-Based Modeling (CBM) to track the user actions and provide individual feedback. We report on a study with 15 stroke survivors, which shows that our treatment is highly effective.