Benjamin D. Nye

AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring

AutoTutor is a natural language tutoring system that has produced learning gains across multiple domains (e.g., computer literacy, physics, critical thinking). In this paper, we review the development, key research findings, and systems that have evolved from AutoTutor. First, the rationale for developing AutoTutor is outlined and the advantages of natural language tutoring are presented. Next, we review three central themes in AutoTutor’s development: human-inspired tutoring strategies, pedagogical agents, and technologies that support natural-language tutoring. Research on early versions of AutoTutor documented the impact on deep learning by co-constructed explanations, feedback, conversational scaffolding, and subject matter content. Systems that evolved from AutoTutor added additional components that have been evaluated with respect to learning and motivation. The latter findings include the effectiveness of deep reasoning questions for tutoring multiple domains, of adapting to the affect of low-knowledge learners, of content over surface features such as voices and persona of animated agents, and of alternative tutoring strategies such as collaborative lecturing and vicarious tutoring demonstrations. The paper also considers advances in pedagogical agent roles (such as trialogs) and in tutoring technologies, such semantic processing and tutoring delivery platforms. This paper summarizes and integrates significant findings produced by studies using AutoTutor and related systems.

Intelligent Tutoring Systems by and for the Developing World: A Review of Trends and Approaches for Educational Technology in a Global Context

As information and communication technology access expands in the developing world, learning technologies have the opportunity to play a growing role to enhance and supplement strained educational systems. Intelligent tutoring systems (ITS) offer strong learning gains, but are a class of technology traditionally designed for most-developed countries. Recently, closer consideration has been made to ITS targeting the developing world and to culturally-adapted ITS. This paper presents findings from a systematic literature review that focused on barriers to ITS adoption in the developing world. While ITS were the primary focus of the review, the implications likely apply to a broader range of educational technology as well. The geographical and economic landscape of tutoring publications is mapped out, to determine where tutoring systems research occurs. Next, the paper discusses challenges and promising solutions for barriers to ITS within both formal and informal settings. These barriers include student basic computing skills, hardware sharing, mobile-dominant computing, data costs, electrical reliability, internet infrastructure, language, and culture. Differences and similarities between externally-developed and locally-developed tutoring system research for the developing world are then considered. Finally, this paper concludes with some potential future directions and opportunities for research on tutoring systems and other educational technologies on the global stage.

ITS and the Digital Divide: Trends, Challenges, and Opportunities

This paper analyzes the state of current intelligent tutoring systems (ITS) research for applications in the developing world. Recent data shows a rapidly narrowing digital divide, with internet and computing device access rising sharply in less developed countries. Tutoring systems could be a transformative technology in these areas, where shortages of teachers and materials are persistent problems. However, the unique challenges and opportunities for ITS in this context are not well-explored. This paper identifies barriers to adoption distinct to the developing world, then presents the results of a systematic mapping study of recent ITS literature (2009-2012) that looks at the level of focus given to each barrier. This study finds that only a small percentage of peer-reviewed publications and architectures address even one of the barriers preventing adoption in these contexts. Implications and strategies being used to target these barriers are discussed.

Semantic Representation Analysis: A General Framework for Individualized, Domain-Specific and Context-Sensitive Semantic Processing

Language agnostic methods for semantic extraction, encoding, and applications are an increasingly active research area in computational linguistics. This paper introduces an analytic framework for vector-based semantic representation called semantic representation analysis (SRA). The rationale for this framework is considered, as well as some successes and future challenges that must be addressed. A cloud-based implementation of SRA as a domain-specific semantic processing portal has been developed. Applications of SRA in three different areas are discussed: analysis of online text streams, analysis of the impression formation over time, and a virtual learning environment called V-CAEST that is enhanced by a conversation-based intelligent tutoring system. These use-cases show the flexibility of this approach across domains, applications, and languages.

Barriers to ITS Adoption: A Systematic Mapping Study

Despite leading to strong learning outcomes, intelligent tutoring systems (ITS) have struggled to reach widescale adoption. However, recent increases in educational technology adoption are slowly leading to larger user bases. Such order-of-magnitude increases have significant research implications for the number and diversity of users. To better understand the problems and solutions that impact this transition, a review of barriers to ITS adoption was performed. This paper significantly extends a prior systematic mapping study of recent ITS literature (2009-2012) focusing on barriers in the developing world. The present study examines research published on possible barriers to adoption related to students, teachers, and school systems. The results indicate that while barriers related to students have received extensive attention, less attention has been given to barriers related to teachers and schools. Successful and innovative approaches to integrating ITS with teacher and school needs are reviewed, with consideration given to both published research papers and successful commercial systems.

ITS, The End of the World as We Know It: Transitioning AIED into a Service-Oriented Ecosystem.

Advanced learning technologies are reaching a new phase of their evolution where they are finally entering mainstream educational contexts, with persistent user bases. However, as AIED scales, it will need to follow recent trends in service-oriented and ubiquitous computing: breaking AIED platforms into distinct services that can be composed for different platforms (web, mobile, etc.) and distributed across multiple systems. This will represent a move from learning platforms to an ecosystem of interacting learning tools. Such tools will enable new opportunities for both user-adaptation and experimentation. Traditional macro-adaptation (problem selection) and step-based adaptation (hints and feedback) will be extended by meta-adaptation (adaptive system selection) and micro-adaptation (event-level optimization). The existence of persistent and widely-used systems will also support new paradigms for experimentation in education, allowing researchers to understand interactions and boundary conditions for learning principles. New central research questions for the field will also need to be answered due to these changes in the AIED landscape.

ElectronixTutor: An Intelligent Tutoring System with Multiple Learning Resources for Electronics.

BackgroundThe Office of Naval Research (ONR) organized a STEM Challenge initiative to explore how intelligent tutoring systems (ITSs) can be developed in a reasonable amount of time to help students learn STEM topics. This competitive initiative sponsored four teams that separately developed systems that covered topics in mathematics, electronics, and dynamical systems. After the teams shared their progress at the conclusion of an 18-month period, the ONR decided to fund a joint applied project in the Navy that integrated those systems on the subject matter of electronic circuits. The University of Memphis took the lead in integrating these systems in an intelligent tutoring system called ElectronixTutor. This article describes the architecture of ElectronixTutor, the learning resources that feed into it, and the empirical findings that support the effectiveness of its constituent ITS learning resources.ResultsA fully integrated ElectronixTutor was developed that included several intelligent learning resources (AutoTutor, Dragoon, LearnForm, ASSISTments, BEETLE-II) as well as texts and videos. The architecture includes a student model that has (a) a common set of knowledge components on electronic circuits to which individual learning resources contribute and (b) a record of student performance on the knowledge components as well as a set of cognitive and non-cognitive attributes. There is a recommender system that uses the student model to guide the student on a small set of sensible next steps in their training. The individual components of ElectronixTutor have shown learning gains in previous decades of research.ConclusionsThe ElectronixTutor system successfully combines multiple empirically based components into one system to teach a STEM topic (electronics) to students. A prototype of this intelligent tutoring system has been developed and is currently being tested. ElectronixTutor is unique in its assembling a group of well-tested intelligent tutoring systems into a single integrated learning environment.

SKOPE-IT (Shareable Knowledge Objects as Portable Intelligent Tutors): overlaying natural language tutoring on an adaptive learning system for mathematics

BackgroundThis study investigated learning outcomes and user perceptions from interactions with a hybrid intelligent tutoring system created by combining the AutoTutor conversational tutoring system with the Assessment and Learning in Knowledge Spaces (ALEKS) adaptive learning system for mathematics. This hybrid intelligent tutoring system (ITS) uses a service-oriented architecture to combine these two web-based systems. Self-explanation tutoring dialogs were used to talk students through step-by-step worked examples to algebra problems. These worked examples presented an isomorphic problem to the preceding algebra problem that the student could not solve in the adaptive learning system.ResultsDue to crossover issues between conditions, experimental versus control condition assignment did not show significant differences in learning gains. However, strong dose-dependent learning gains were observed that could not be otherwise explained by either initial mastery or time-on-task. User perceptions of the dialog-based tutoring were mixed, and survey results indicate that this may be due to the pacing of dialog-based tutoring using voice, students judging the agents based on their own performance (i.e., the quality of their answers to agent questions), and the students’ expectations about mathematics pedagogy (i.e., expecting to solving problems rather than talking about concepts). Across all users, learning was most strongly influenced by time spent studying, which correlated with students’ self-reported tendencies toward effort avoidance, effective study habits, and beliefs about their ability to improve in mathematics with effort.ConclusionsIntegrating multiple adaptive tutoring systems with complementary strengths shows some potential to improve learning. However, managing learner expectations during transitions between systems remains an open research area. Finally, while personalized adaptation can improve learning efficiency, effort and time-on-task for learning remains a dominant factor that must be considered by interventions.

Evaluating the Effectiveness of Integrating Natural Language Tutoring into an Existing Adaptive Learning System

This paper reports initial results of an evaluation for an ITS that follows service-oriented principles to integrate natural language tutoring into an existing adaptive learning system for mathematics. Self-explanation tutoring dialogs were used to talk students through step-by-step worked solutions to Algebra problems. These worked solutions presented an isomorphic problem to a preceding Algebra problem that the student could not solve in an adaptive learning system. Due to crossover issues between conditions, experimental versus control condition assignment did not show significant differences in learning gains. However, strong dose-dependent learning gains were observed that could not be otherwise explained by either initial mastery or time-on-task.

Tutorial Dialogue Modes in a Large Corpus of Online Tutoring Transcripts

Building on previous work in this area, we provide a description and justification for a new way of identifying modes and mode switches in tutorial dialogues, part of a coding scheme involving 16 modes and 125 distinct dialogue acts. We also present preliminary results from an analysis of 1,438 human-annotated transcripts, consisting of more than 90,000 turns. Among other findings, this analysis shows subtle differences in the “mode architecture” of successful vs. less successful sessions, as judged by expert tutors.