Jonathan Sewall

The Beginning of a Beautiful Friendship? Intelligent Tutoring Systems and MOOCs

A key challenge in ITS research and development is to support tutoring at scale, for example by embedding tutors in MOOCs. An obstacle to at-scale deployment is that ITS architectures tend to be complex, not easily deployed in browsers without significant server-side processing, and not easily embedded in a learning management system (LMS). We present a case study in which a widely used ITS authoring tool suite, CTAT/TutorShop, was modified so that tutors can be embedded in MOOCs. Specifically, the inner loop (the example-tracing tutor engine) was moved to the client by reimplementing it in JavaScript, and the tutors were made compatible with the LTI e-learning standard. The feasibility of this general approach to ITS/MOOC integration was demonstrated with simple tutors in an edX MOOC “Data Analytics and Learning.”

The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains

Intelligent Tutoring Systems have been shown to be effective in a number of domains, but they remain hard to build, with estimates of 200-300 hours of development per hour of instruction. Two goals of the Cognitive Tutor Authoring Tools (CTAT) project are to (a) make tutor development more efficient for both programmers and non-programmers and (b) produce scientific evidence indicating which tool features lead to improved efficiency. CTAT supports development of two types of tutors, Cognitive Tutors and Example-Tracing Tutors, which represent different trade-offs in terms of ease of authoring and generality. In preliminary small-scale controlled experiments involving basic Cognitive Tutor development tasks, we found efficiency gains due to CTAT of 1.4 to 2 times faster. We expect that continued development of CTAT, informed by repeated evaluations involving increasingly complex authoring tasks, will lead to further efficiency gains.

Creating cognitive tutors for collaborative learning: steps toward realization

Our long-term research goal is to provide cognitive tutoring of collaboration within a collaborative software environment. This is a challenging goal, as intelligent tutors have traditionally focused on cognitive skills, rather than on the skills necessary to collaborate successfully. In this paper, we describe progress we have made toward this goal. Our first step was to devise a process known as bootstrapping novice data (BND), in which student problem-solving actions are collected and used to begin the development of a tutor. Next, we implemented BND by integrating a collaborative software tool, Cool Modes, with software designed to develop cognitive tutors (i.e., the cognitive tutor authoring tools). Our initial implementation of BND provides a means to directly capture data as a foundation for a collaboration tutor but does not yet fully support tutoring. Our next step was to perform two exploratory studies in which dyads of students used our integrated BND software to collaborate in solving modeling tasks. The data collected from these studies led us to identify five dimensions of collaborative and problem-solving behavior that point to the need for abstraction of student actions to better recognize, analyze, and provide feedback on collaboration. We also interviewed a domain expert who provided evidence for the advantage of bootstrapping over manual creation of a collaboration tutor. We discuss plans to use these analyses to inform and extend our tools so that we can eventually reach our goal of tutoring collaboration.

Cognitive tutoring of collaboration: developmental and empirical steps towards realization

In this paper, we describe developmental and empirical steps we have taken toward providing Cognitive Tutoring to students within a collaborative software environment. We have taken two important steps toward realizing this goal. First, we have integrated a collaborative software tool, Cool Modes, with software designed to develop Cognitive Tutors (the Cognitive Tutor Authoring Tool). Our initial integration does not provide tutoring per se but rather acts as a means to capture data that provides the beginnings of a tutor for collaboration. Second, we have performed an initial study in which dyads of students used our software to collaborate in solving a classification / composition problem. This study uncovered five dimensions of analysis that our approach must use to help us better understand student collaborative behavior and lead to the eventual development of a Cognitive Tutor for collaboration. We discuss our plans to incorporate such analysis into our approach and to run further studies.

Authoring Tools for Collaborative Intelligent Tutoring System Environments.

Authoring tools have been shown to decrease the amount of time and resources needed for the development of Intelligent Tutoring Systems (ITSs). Although collaborative learning has been shown to be beneficial to learning, most of the current authoring tools do not support the development of collaborative ITSs. In this paper, we discuss an extension to the Cognitive Tutor Authoring Tools to allow for development of collaborative ITSs through multiple synchronized tutor engines. Using this tool, an author can combine collaboration with the type of problem solving support typically offered by an ITS. Different phases of collaboration scripts can be tied to particular problem states in a flexible, problem-specific way. We illustrate the tool’s capabilities by presenting examples of collaborative tutors used in recent studies that showed learning gains. The work is a step forward in blending computer-supported collaborative learning and ITS technologies in an effort to combine their strengths.

Beanstalk: A unity game addressing balance principles, socio-emotional learning and scientific inquiry

Beanstalk is an educational game for children ages 6-10 teaching balance-fulcrum principles while folding in scientific inquiry and socio-emotional learning. This paper explores the incorporation of these additional dimensions using intrinsic motivation and a framing narrative. Four versions of the game are detailed, along with preliminary player data in a 2×2 pilot test with 64 children shaping the modifications of Beanstalk for much broader testing.

Embedding Intelligent Tutoring Systems in MOOCs and e-Learning Platforms

Intelligent tutoring systems ITS and MOOCs tend to have complementary pedagogical approaches, but their combination is rarely if ever seen. A key obstacle may be technical integration. We present a generalizable case study of extending ITS authoring technology to make tutors easily embeddable into a variety of MOOC/e-learning platforms and run on a range of web-enabled devices. We enhanced the domain-independent Cognitive Tutor Authoring Tools CTAT to enable integration of CTAT tutors into multiple environments. A salient lesson learned is that use of widely-used web-based technologies HTML and JavaScript may be a major factor in ITS uptake. Also, we found that embedding tutors into existing LMS is challenging, but environment-specific changes can be isolated in a generalizable manner.

The Frequency of Tutor Behaviors: A Case Study

For cross-pollination between ITS authoring tools, it may be useful to know the prevalence of tutoring behaviors crafted with these tools. As a case study, we analyze the problem units of Mathtutor, a web-based intelligent tutor for middle-school mathematics built, as an example-tracing tutor, with the Cognitive Tutor Authoring Tools CTAT. We focus on tutoring behaviors that are relevant to a wide range of tutoring systems, not just example-tracing tutors, including behaviors not found in VanLehns 2006 taxonomy of tutor behaviors. Our analysis reveals that several tutor behaviors not typically highlighted in the ITS literature were used extensively, sometimes in unanticipated ways. Others were less prevalent than expected. This novel insight into the prevalence of tutor behaviors may provide practical guidance to ITS authoring tool developers. At a theoretical level, it extends VanLehns taxonomy of tutor behavior, potentially expanding how the field conceptualizes ITS behavior.

Tutorial on Rapid Development of Intelligent Tutors using the Cognitive Tutor Authoring Tools (CTAT)

Intelligent Tutoring Systems (ITS) can both help improve student learning and serve as useful platforms for experiments in learning science [1,2]. But the difficulty of building or customizing ITSs has hindered their acceptance among educators and researchers [3]. The Cognitive Tutor Authoring Tools (CTAT) project aims to provide a suite of authoring tools that make tutor development more affordable by leveraging human-computer interaction and artificial intelligence techniques. Previous efforts on CTAT added the capability for nonprogrammers to create exampletracing tutors via a programming-by-demonstration technique that requires no coding [4]. While exampletracing tutors provide a student experience similar to that of the more general cognitive tutors, they also require that an author demonstrate and fully annotate each individual problem to be presented.

ITS authoring through programming-by-demonstration

The Cognitive Tutor Authoring Tools (CTAT) [1] are a suite of software programs meant to make the creation of web-based ITS practical for non-programmers CTAT supports a relatively novel type of tutors, called example-tracing tutors, which use examples of problem-solving approaches to assess and guide students as they practice solving problems CTAT employs a programming-by-demonstration paradigm that relies on creating examples of how problems are to be solved, rather than defining general rules or constraints that characterize solutions or solution processes The result is an editable behavior graph that contains the tutors intelligence about how to react to student actions and what hints to give for next steps Although relatively easy to build, example-tracing tutors support the key behaviors identified by VanLehn [2] as characteristic of ITS Data from over 26 research studies using CTAT indicate that these tools lower the cost of ITS development by a factor of 4-8.