Martina A. Rau

How Should Intelligent Tutoring Systems Sequence Multiple Graphical Representations of Fractions? A Multi-Methods Study

Providing learners with multiple representations of learning content has been shown to enhance learning outcomes. When multiple representations are presented across consecutive problems, we have to decide in what sequence to present them. Prior research has demonstrated that interleaving tasks types (as opposed to blocking them) can foster learning. Do the same advantages apply to interleaving representations? We addressed this question using a variety of research methods. First, we conducted a classroom experiment with an intelligent tutoring system for fractions. We compared four practice schedules of multiple graphical representations: blocked, fully interleaved, moderately interleaved, and increasingly interleaved. Based on data from 230 4th and 5th-grade students, we found that interleaved practice leads to better learning outcomes than blocked practice on a number of measures. Second, we conducted a think-aloud study to gain insights into the learning mechanisms underlying the advantage of interleaved practice. Results show that students make connections between representations only when explicitly prompted to do so (and not spontaneously). This finding suggests that reactivation, rather than abstraction, is the main mechanism to account for the advantage of interleaved practice. Third, we used methods derived from Bayesian knowledge tracing to analyze tutor log data from the classroom experiment. Modeling latent measures of students’ learning rates, we find higher learning rates for interleaved practice than for blocked practice. This finding extends prior research on practice schedules, which shows that interleaved practice (compared to blocked practice) impairs students’ problem-solving performance during the practice phase when using raw performance measures such as error rates. Our findings have implications for the design of multi-representational learning materials and for research on adaptive practice schedules in intelligent tutoring systems.

Why interactive learning environments can have it all: resolving design conflicts between competing goals

Designing interactive learning environments (ILEs; e.g., intelligent tutoring systems, educational games, etc.) is a challenging interdisciplinary process that needs to satisfy multiple stakeholders. ILEs need to function in real educational settings (e.g., schools) in which a number of goals interact. Several instructional design methodologies exist to help developers address these goals. However, they often lead to conflicting recommendations. Due to the lack of an established methodology to resolve such conflicts, developers of ILEs have to rely on ad-hoc solutions. We present a principled methodology to resolve such conflicts. We build on a well-established design process for creating Cognitive Tutors, a highly effective type of ILE. We extend this process by integrating methods from multiple disciplines to resolve design conflicts. We illustrate our methodologys effectiveness by describing the iterative development of the Fractions Tutor, which has proven to be effective in classroom studies with 3,000 4th-6th graders.

Complementary Effects of Sense-Making and Fluency-Building Support for Connection Making: A Matter of Sequence?

Multiple graphical representations can significantly improve students’ learning. To acquire robust knowledge of the domain, students need to make connections between the different graphical representations. In doing so, students need to engage in two crucial learning processes: sense-making processes to build up conceptual understanding of the connections, and fluency-building processes to fast and effortlessly make use of perceptual properties in making connections. We present an experimental study which contrasts two hypotheses on how these learning processes interact. Does understanding facilitate fluency-building processes, or does fluency enhance sense-making processes? And consequently, which learning process should intelligent tutoring systems support first? Our results based on test data and tutor logs show an advantage for providing support for sense-making processes before fluency-building processes. To enhance students’ robust learning of domain knowledge, ITSs should ensure that students have adequate conceptual understanding of connections between graphical representations before providing fluency-building support for connection making.

Supporting Students in Making Sense of Connections and in Becoming Perceptually Fluent in Making Connections Among Multiple Graphical Representations.

Prior research shows that multiple representations can enhance learning, provided that students make connections among them. We hypothesized that support for connection making is most effective in enhancing learning of domain knowledge if it helps students both in making sense of these connections and in becoming perceptually fluent in making connections. We tested this hypothesis in an experiment with 428 4th- and 5th-grade students who worked with different versions of an intelligent tutoring system for fractions learning. Results did not show main effects for sense-making or fluency-building support but an interaction effect, such that a combination of sense-making and fluency-building support is most effective in enhancing fractions knowledge. Causal path analysis of log data from the system shows that sense-making support enhances students’ benefit from fluency-building support, but fluency-building support does not enhance their benefit from sense-making support. Our results suggest that both understanding of connections and perceptual fluency in connection making are critical aspects of learning of domain knowledge with multiple graphical representations. Findings from the causal path analysis lead to the testable prediction that instruction should provide sense-making support and fluency-building support for connection making.

Understanding Student Success in Chemistry Using Gaze Tracking and Pupillometry

Eye tracking allows us to identify visual strategies through gaze behavior, which can help us understand how students process content. Furthermore, understanding which visual strategies are successful can help us improve educational materials that foster successful use of these visual strategies. Previous studies have demonstrated the predictive value of eye tracking for student performance. Chemistry is a highly visual domain, making it particularly appropriate to study visual strategies. Eye tracking also provides measures of pupil dilation that correlate with cognitive processes important to learning, but have not yet been assessed in any realistic learning environments. We examined the gaze behavior and pupil dilation of undergraduate students working with a specialized ITS for chemistry: Chem Tutor. Chem Tutor emphasizes visual learning by focusing specifically on graphical representations. We assessed the value of over 40 high-level gaze features along with measures of pupil diameter to predict student performance and learning gains across an entire chemistry problem set. We found that certain gaze features are strong predictors of performance, but less so of learning gains, while pupil diameter is marginally predictive of learning gains, but not performance. Further studies that assess pupil dilation with higher temporal precision will be necessary to draw conclusions about the limits of its predictive power.

A Framework for Educational Technologies that Support Representational Competencies

Visual representations are ubiquitous in STEM disciplines. Yet, students’ difficulties in learning with visual representations are well documented. Therefore, to succeed in STEM, students need representational competencies—the ability to use visual representations for problem solving and learning. Educational technologies that support students’ acquisition of representational competencies can significantly enhance their success in STEM disciplines. Current design frameworks for educational technologies do not offer sufficient guidance to develop supports for representational competencies. This paper presents a new design framework that describes an iterative, step-by-step approach for the design of educational technologies that support representational competencies (SUREC) in a way that aligns with the demands specific to the target discipline. The paper illustrates how this framework was used to inform the design of an intelligent tutoring system for undergraduate chemistry. An evaluation study suggests that the SUREC framework yielded an effective educational technology that enhances students’ learning of content knowledge.

Multi-methods Approach for Domain-Specific Grounding: An ITS for Connection Making in Chemistry

Making connections between graphical representations is integral to learning in science, technology, engineering, and mathematical (STEM) fields. However, students often fail to make these connections spontaneously. ITSs are suitable tools to support connection making. Yet, when designing an ITS for connection making, we need to investigate what learning processes and concepts play a role within the specific domain. We describe a multi-methods approach for grounding ITS design in the specific requirements of the target domain. Specifically, we applied this approach to an ITS for connection making in chemistry. We used a theoretical framework that describes potential target learning processes and conducted two empirical studies – using tests, eye tracking, and interviews – to investigate how these learning processes play out in the chemistry domain. We illustrate how our findings inform the design of a chemistry tutor. Initial pilot study results suggest that the ITS promotes learning processes that are productive in chemistry.

How to Use Multiple Graphical Representations to Support Conceptual Learning? Research-Based Principles in the Fractions Tutor

Multiple graphical representations are ubiquitous in educational materials because they serve complementary roles in emphasizing conceptual aspects of the domain. Yet, to benefit robust learning, students have to understand each representation and make connections between them. We describe research-based principles for the use of multiple graphical representations within intelligent tutoring systems (ITSs). These principles are the outcome of a series of iterative classroom experiments with the Fractions Tutor with over 3,000 students. The implementation of these principles into the Fractions Tutor results in robust conceptual learning. To our knowledge, the Fractions Tutor is the first ITS to use multiple graphical representations by implementing research-based principles to support conceptual learning. The instructional design principles we established apply to ITSs across domains.

ITS Support for Conceptual and Perceptual Connection Making Between Multiple Graphical Representations

Connection making between representations is crucial to learning in STEM domains, but it is a difficult task for students. Prior research shows that supporting connection making enhances students’ learning of domain knowledge. Most prior research has focused on supporting one type of connection-making process: conceptual reasoning about connections between representations. Yet, recent research suggests that a second type of connection-making process plays a role in students’ learning: perceptual translation between representations. We hypothesized that combining support for both conceptual and perceptual connection-making processes leads to higher learning gains on a domain-knowledge test. We tested this hypothesis in a lab experiment with 117 undergraduate students using an intelligent tutoring system for chemistry. Results show that the combination of conceptual and perceptual connection-making supports leads to higher learning outcomes. This finding suggests that the effectiveness of educational technologies can be enhanced if they combine support for conceptual and perceptual connection-making processes.

Thinking with your hands: interactive graphical representations in a tutor for fractions learning

Learning with multiple graphical representations is effective in many instructional activities, including fractions. However, students need to be supported in understanding the individual representations and in how the representations relate to one another. We investigated (1) whether interactive manipulations of graphical representation support a deeper understanding of the representations compared to static graphics and (2) whether connection-making activities help students better understand the relations between representations. In a study with 312 4th and 5th grade students we found that interactive representations were indeed more effective in improving student fraction learning compared to static fraction graphics, especially for students yet unfamiliar with the topics being taught. We found no effect for connection-making activities. The results suggest that domains with (multiple) representations are best taught with tutor-guided student manipulation of these graphics rather than with static pictures.