Bruce M. McLaren

The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains

Intelligent Tutoring Systems have been shown to be effective in a number of domains, but they remain hard to build, with estimates of 200-300 hours of development per hour of instruction. Two goals of the Cognitive Tutor Authoring Tools (CTAT) project are to (a) make tutor development more efficient for both programmers and non-programmers and (b) produce scientific evidence indicating which tool features lead to improved efficiency. CTAT supports development of two types of tutors, Cognitive Tutors and Example-Tracing Tutors, which represent different trade-offs in terms of ease of authoring and generality. In preliminary small-scale controlled experiments involving basic Cognitive Tutor development tasks, we found efficiency gains due to CTAT of 1.4 to 2 times faster. We expect that continued development of CTAT, informed by repeated evaluations involving increasingly complex authoring tasks, will lead to further efficiency gains.

Automated, Unobtrusive, Action-by-Action Assessment of Self-Regulation During Learning With an Intelligent Tutoring System

Assessment of students’ self-regulated learning (SRL) requires a method for evaluating whether observed actions are appropriate acts of self-regulation in the specific learning context in which they occur. We review research that has resulted in an automated method for context-sensitive assessment of a specific SRL strategy, help seeking while working with an intelligent tutoring system. The method relies on a computer-executable model of the targeted SRL strategy. The method was validated by showing that it converges with other measures of help seeking. Automated feedback on help seeking driven by this method led to a lasting improvement in students’ help-seeking behavior, although not in domain-specific learning. The method is unobtrusive, is temporally fine-grained, and can be applied on a large scale and over extended periods. The approach could be applied to other SRL strategies besides help seeking.

User-Controllable Security and Privacy for Pervasive Computing

We describe our current work in developing novel mechanisms for managing security and privacy in pervasive computing environments. More specifically, we have developed and evaluated three different applications, including a contextual instant messenger, a people finder application, and a phone-based application for access control. We also draw out some themes we have learned thus far for user-controllable security and privacy.

Computational Models of Ethical Reasoning: Challenges, Initial Steps, and Future Directions

How can machines support or, even more significantly, replace humans in performing ethical reasoning? This question greatly interests machine ethics researchers. Imbuing a computer with the ability to reason about ethical problems and dilemmas is as difficult a task as there is for AI scientists and engineers. The author briefly describes a few of the programs and discusses in detail two programs, both of which employ techniques from the area of AI known as case-based reasoning and implement aspects of the ethical approach known as casuistry. One of these programs, Truth-Teller, accepts a pair of ethical dilemmas and describes the salient similarities and differences between them, from both an ethical and a pragmatic perspective. The other program, SIROCCO, accepts a single ethical dilemma and retrieves other cases and ethical principles that might be relevant

Extensionally defining principles and cases in ethics: an AI model

Principles are abstract rules intended to guide decision-makers in making normative judgments in domains like the law, politics, and ethics. It is difficult, however, if not impossible to define principles in an intensional manner so that they may be applied deductively. The problem is the gap between the abstract, open-textured principles and concrete facts. On the other hand, when expert decision-makers rationalize their conclusions in specific cases, they often link principles to the specific facts of the cases. In effect, these expert-defined associations between principles and facts provide extensional definitions of the principles. The experts operationalize the abstract principles by linking them to the facts.This paper discusses research in which the following hypothesis was empirically tested: extensionally defined principles, as well as cited past cases, can help in predicting the principles and cases that might be relevant in the analysis of new cases. To investigate this phenomenon computationally, a large set of professional ethics cases was analyzed and a computational model called SIROCCO, a system for retrieving principles and past cases, was constructed. Empirical evidence is presented that the operationalization information contained in extensionally defined principles can be leveraged to predict the principles and past cases that are relevant to new problem situations. This is shown through an ablation experiment, comparing SIROCCO to a version of itself that does not employ operationalization information. Further, it is shown that SIROCCOs extensionally defined principles and case citations help it to outperform a full-text retrieval program that does not employ such information.

Computer supported moderation of e-discussions: the ARGUNAUT approach

Despite their potential value for learning purposes, e-discussions do not necessarily lead to desirable results, even when moderated. The study of the moderators role, especially in synchronous, graphical e-discussions, and the development of appropriate tools to assist moderators are the objectives of the ARGUNAUT project. This project aims at unifying awareness and feedback mechanisms in e-discussion environments, presently implemented on two existing platforms. This system is primarily directed to a human moderator and facilitating moderation, but might also help the students monitor their own interactions. At the heart of system are the inter-relations between an off-line AI analysis mechanism and an on-line monitoring module. This is done through a collaboration of technological and pedagogical teams, showing promising preliminary results.

Creating cognitive tutors for collaborative learning: steps toward realization

Our long-term research goal is to provide cognitive tutoring of collaboration within a collaborative software environment. This is a challenging goal, as intelligent tutors have traditionally focused on cognitive skills, rather than on the skills necessary to collaborate successfully. In this paper, we describe progress we have made toward this goal. Our first step was to devise a process known as bootstrapping novice data (BND), in which student problem-solving actions are collected and used to begin the development of a tutor. Next, we implemented BND by integrating a collaborative software tool, Cool Modes, with software designed to develop cognitive tutors (i.e., the cognitive tutor authoring tools). Our initial implementation of BND provides a means to directly capture data as a foundation for a collaboration tutor but does not yet fully support tutoring. Our next step was to perform two exploratory studies in which dyads of students used our integrated BND software to collaborate in solving modeling tasks. The data collected from these studies led us to identify five dimensions of collaborative and problem-solving behavior that point to the need for abstraction of student actions to better recognize, analyze, and provide feedback on collaboration. We also interviewed a domain expert who provided evidence for the advantage of bootstrapping over manual creation of a collaboration tutor. We discuss plans to use these analyses to inform and extend our tools so that we can eventually reach our goal of tutoring collaboration.

A politeness effect in learning with web-based intelligent tutors

College students learned to solve chemistry stoichiometry problems with a web-based intelligent tutor that provided hints and feedback, using either polite or direct language. There was a pattern in which students with low prior knowledge of chemistry performed better on subsequent problem-solving tests if they learned from the polite tutor rather than the direct tutor (d=.78 on an immediate test, d=.51 on a delayed test), whereas students with high prior knowledge showed the reverse trend (d=-.47 for an immediate test; d=-.13 for a delayed test). These results point to a boundary condition for the politeness principle-the idea that people learn more deeply when words are in polite style. At least for low-knowledge learners, the results are consistent with social agency theory-the idea that social cues, such as politeness, can prime learners to accept a web-based tutor as a social partner and therefore try harder to make sense of the tutors messages.

Using erroneous examples to improve mathematics learning with a web-based tutoring system

Middle school students learned to solve decimal problems with a web-based tutoring system.ExErr group received erroneous examples to correct and explain.PS group received problems to solve and explain.ExErr group outperformed PS group on a delayed test and on judging answer correctness.PS group reported liking the lessons better than the ExErr group. This study examines whether asking students to critique incorrect solutions to decimal problems based on common misconceptions can help them learn about decimals better than asking them to solve the same problems and receive feedback. In a web-based tutoring system, 208 middle school students either had to identify, explain, and correct errors made by a fictional student (erroneous examples group) or solve isomorphic versions of the problems with feedback (problem-solving group). Although the two groups did not differ significantly on an immediate posttest, students in the erroneous examples group performed significantly better on a delayed posttest administered one week later (d=.62). Students in the erroneous examples group also were more accurate at judging whether their posttest answers were correct (d=.49). Students in the problem-solving group reported higher satisfaction with the materials than those in the erroneous examples group, indicating that liking instructional materials does not equate to learning from them. Overall, practice in identifying, explaining, and correcting errors may help students process decimal problems at a deeper level, and thereby help them overcome misconceptions and build a lasting understanding of decimals.

Studying the effects of personalized language and worked examples in the context of a web-based intelligent tutor

Previous studies have demonstrated the learning benefit of personalized language and worked examples. However, previous investigators have primarily been interested in how these interventions support students as they problem solve with no other cognitive support. We hypothesized that personalized language added to a web-based intelligent tutor and worked examples provided as complements to the tutor would improve student (e-)learning. However, in a 2 x 2 factorial study, we found that personalization and worked examples had no significant effects on learning. On the other hand, there was a significant difference between the pretest and posttest across all conditions, suggesting that the online intelligent tutor present in all conditions did make a difference in learning. We conjecture why personalization and, especially, the worked examples did not have the hypothesized effect in this preliminary experiment, and discuss a new study we have begun to further investigate these effects.