Collin Lynch

Toward legal argument instruction with graph grammars and collaborative filtering techniques

This paper presents an approach for intelligent tutoring in the field of legal argumentation. In this approach, students study transcripts of US Supreme Court oral argument and create a graphical representation of argument flow as tests offered by attorneys being challenged by hypotheticals posed by Justices. The proposed system, which is based on the collaborative modeling framework Cool Modes, is capable of detecting three types of weaknesses in arguments; when it does, it presents the student with a self explanation prompt. This kind of feedback seems more appropriate than the “strong connective feedback” typically offered by model-tracing or constraint-based tutors. Structural and context weaknesses in arguments are handled by graph grammars, and the critical problem of detecting and dealing with content weaknesses in student contributions is addressed through a collaborative filtering approach, thereby avoiding the critical problem of natural language processing in legal argumentation. An early version of the system was pilot tested with two students.

Minimally Invasive Tutoring of Complex Physics Problem Solving

Solving complex physics problems requires some kind of knowledge for selecting appropriate applications of physics principles. This knowledge is tacit, in that it is not explicitly taught in textbooks, existing tutoring systems or anywhere else. Experts seem to have acquired it via implicit learning and may not be aware of it. Andes is a coach for physics problem solving that has had good evaluations, but still does not teach complex problem solving as well as we would like. The conventional ITS approach to increasing its effectiveness requires teaching the tacit knowledge explicitly, and yet this would cause Andes to be more invasive. In particular, the textbooks and instructors would have to make space in an already packed curriculum for teaching the tacit knowledge. This paper discusses our attempts to teach the tacit knowledge without making Andes more invasive.

Learning by diagramming Supreme Court oral arguments

This paper describes an intelligent tutoring system, LARGO, that helps students learn skills of legal reasoning with hypotheticals by analyzing oral arguments before the US Supreme Court. The skills involve proposing a rule-like test for deciding a case, posing hypotheticals to challenge the rule, and responding by analogizing or distinguishing the hypotheticals and/or modifying the proposed test. Students diagram arguments in a special-purpose graphical language and receive feedback in the form of reflection questions.

Implicit versus explicit learning of strategies in a non-procedural cognitive skill

University physics is typical of many cognitive skills in that there is no standard procedure for solving problems, and yet a few students still master the skill. This suggests that their learning of problem solving strategies is implicit, and that an effective tutoring system need not teach problem solving strategies as explicitly as model-tracing tutors do. In order to compare implicit vs. explicit learning of problem solving strategies, we developed two physics tutoring systems, Andes and Pyrenees. Pyrenees is a model-tracing tutor that teaches a problem solving strategy explicitly, whereas Andes uses a novel pedagogy, developed over many years of use in the field, that provides virtually no explicit strategic instruction. Preliminary results from an experiment comparing the two systems are reported.

How Early Does the CS Gender Gap Emerge?: A Study of Collaborative Problem Solving in 5th Grade Computer Science

Elementary computer science has gained increasing attention within the computer science education research community. We have only recently begun to explore the many unanswered questions about how young students learn computer science, how they interact with each other, and how their skill levels and backgrounds vary. One set of unanswered questions focuses on gender equality for young computer science learners. This paper examines how the gender composition of collaborative groups in elementary computer science relates to student achievement. We report on data collected from an in-school 5th grade computer science elective offered over four quarters in 2014-2015. We found a significant difference in the quality of artifacts produced by learner groups depending upon their gender composition, with groups of all female students performing significantly lower than other groups. Our analyses suggest important factors that are influential as these learners begin to solve computer science problems. This new evidence of gender disparities in computer science achievement as young as ten years of age highlights the importance of future study of these factors in order to provide effective, equitable computer science education to learners of all ages.

Can Diagrams Predict Essay Grades

Diagrammatic models of argument have grown in prominence in recent years. While they have been applied in a number of tutoring contexts, it has not yet been shown that student-produced diagrams can be used to effectively grade students or predict their future performance. We show that manually-assigned diagram grades and automatic structural features of argument diagrams can be used to predict students’ future essay grades, thus supporting the use of argument diagrams for instruction. We also show that the automatic features are competitive with expert human grading despite the fact that semantic content was ignored in automatic processing.

Data-Driven Worked Examples Improve Retention and Completion in a Logic Tutor

Research shows that expert-crafted worked examples can have a positive effect on student performance. To investigate the potential for data-driven worked examples to achieve similar results, we generated worked examples for the Deep Thought logic tutor, and conducted an experiment to assess their impact on performance. Students who received data-driven worked examples were much more likely to complete the tutor, and completed the tutor in less time. This study demonstrates that worked examples, automatically generated from student data, can be used to improve student learning in tutoring systems.

Toward assessing law students' argument diagrams

The development of graphical argument models is an active and growing area of research in Artificial Intelligence and Law. The aim is to develop models which may be readily used by legal professionals and novices to produce and parse arguments. If this goal is to be realized it is important to develop models that human reasoners can manipulate and assess consistently. We report on an ongoing study of graph agreement in the context of the LARGO system.

Exploring Online Course Sociograms Using Cohesion Network Analysis.

Massive Open Online Courses (MOOCs) have become an important platform for teaching and learning because of their ability to deliver educational accessibility across time and distance. Online learning environments have also provided new research opportunities to examine learning success at a large scale. One data tool that has been proven effective in exploring student success in on-line courses has been Cohesion Network Analysis (CNA), which offers the ability to analyze discourse structure in collaborative learning environments and facilitate the identification of learner interaction patterns. These patterns can be used to predict students’ behaviors such as dropout rates and performance. The focus of the current paper is to identify sociograms (i.e., interaction graphs among participants) generated through CNA on course forum discussions and to identify temporal trends among students. Here, we introduce extended CNA visualizations available in the ReaderBench framework. These visualizations can be used to convey information about interactions between participants in online forums, as well as corresponding student clusters within specific timeframes.

Using Serious Game Analytics to Inform Digital Curricular Sequencing: What Math Objective Should Students Play Next?

This paper applied serious game analytics to inform digital curricular sequencing in a longitude, curriculum-integrated math game, ST Math. When integrating serious games into classrooms, teachers may have the flexibility to change the order of math objectives for student groups to play. However, it is unclear how teacher decisions, as well as the sequencing of the original curricular order affect students. Moreover, few researchers have applied data-driven methods to inform content ordering in educational games, where the nature of educational content and student behaviors are different from many e-learning platforms. In this paper, we present a novel method that suggests curricular sequencing based on the prediction relationship between math objectives. Our results include specific design recommendations for ST Math, and general data-driven insights for digital curricular design, such as the pacing of objectives and the ordering of math concepts. Our method can potentially be applied to data from a wide range of games and digital learning platforms, enabling developers to better understand how to sequence educational content.