Ilya M. Goldin

Introducing PETE: computer support for teaching ethics

In this paper, we discuss the challenges in providing computer support for teaching professional ethics using a case-based approach. We describe our tutoring software, PETE, which helps students prepare cases for class discussion. PETE enables students to practice methods of moral reasoning. It also encourages them to compare their work to a range of other peer responses. We discuss how the program could incorporate AI techniques and how to evaluate its effectiveness.

Peering inside peer review with Bayesian models

Instructors and students would benefit more from computer-supported peer review, if instructors received information on how well students have understood the conceptual issues underlying the writing assignment. Our aim is to provide instructors with an evaluation of both the students and the criteria that students used to assess each other. Here we develop and evaluate several hierarchical Bayesian models relating instructor scores of student essays to peer scores based on two peer assessment rubrics. We examine model fit and show how pooling across students and different representations of rating criteria affect model fit and how they reveal information about student writing and assessment criteria. Finally, we suggest how our Bayesian models may be used by an instructor or an ITS.

Learner Differences and Hint Content

Because feedback affects learning, it is central to many educational technologies. We analyze properties of hint feedback in an intelligent tutoring system for high school geometry. First, we examine whether feedback content or feedback sequence is a better predictor of student performance after feedback. Second, we investigate whether linguistic features of hints affect performance. We find that students respond to different hint types differently even after accounting for student proficiency, skill difficulty, and prior practice. We also find that hint content, but not linguistic features affects performance. The findings suggest that tutoring system developers should focus on individual learner differences and feedback content.

Validity and Reliability of an Instrument for Assessing Case Analyses in Bioengineering Ethics Education

Assessment in ethics education faces a challenge. From the perspectives of teachers, students, and third-party evaluators like the Accreditation Board for Engineering and Technology and the National Institutes of Health, assessment of student performance is essential. Because of the complexity of ethical case analysis, however, it is difficult to formulate assessment criteria, and to recognize when students fulfill them. Improvement in students’ moral reasoning skills can serve as the focus of assessment. In previous work, Rosa Lynn Pinkus and Claire Gloeckner developed a novel instrument for assessing moral reasoning skills in bioengineering ethics. In this paper, we compare that approach to existing assessment techniques, and evaluate its validity and reliability. We find that it is sensitive to knowledge gain and that independent coders agree on how to apply it.

Viz-R: Using Recency to Improve Student and Domain Models

We describe a new method to troubleshoot and improve domain and student models from interactive learning environments. The method applies as long as the models can generate predictions of student behavior. The method is a visualization of model predictions, categorized using a metric of recent performance. We describe the method, its application in prior work to student models, and a proposed extension to domain models.

New Directions in Formative Feedback in Interactive Learning Environments

Formative feedback is well known as a key factor in influencing learning. Modern interactive learning environments provide a broad range of ways to provide feedback to students as well as new tools to understand feedback and its relation to various learning outcomes. This issue focuses on the role of formative feedback through a lens of how technologies both support student learning and enhance our understanding of the mechanisms of feedback. The papers in the issue span a variety of feedback strategies, instructional domains, AI techniques, and educational use cases in order to improve and understand formative feedback in interactive learning environments. The issue encompasses three primary themes critical to understanding formative feedback: 1) the role of human information processing and individual learner characteristics for feedback efficiency, 2) how to deliver meaningful feedback to learners in domains of study where student work is difficult to assess, and 3) how human feedback sources (e.g., peer students) can be supported by user interfaces and technology-generated feedback.

Evaluating Simplicial Mixtures of Markov Chains for Modeling Student Metacognitive Strategies

Modeling and discovery of the strategies that students use, both cognitive and metacognitive, is important for building accurate models of student knowledge and learning. We present a simulation study to examine whether simplicial mixtures of Markov chains (SM-MC) can be used to model student metacognitive strategies. We find that SM-MC models cannot be estimated on the moderately sized data sets common in education, and must be adapted to be useful for strategy modeling.

Formative Feedback in Interactive Learning Environments

Educators and researchers have long recognized the importance of formative feedback for learning. Formative feedback helps learners understand where they are in a learning process, what the goal is, and how to reach that goal. While experimental and observational research has illuminated many aspects of feedback, modern interactive learning environments provide new tools to understand feedback and its relation to various learning outcomes.