Manu Kapur

Designing for Productive Failure

In this article, we describe the design principles undergirding productive failure (PF; M. Kapur, 2008). We then report findings from an ongoing program of research on PF in mathematical problem solving in 3 Singapore public schools with significantly different mathematical ability profiles, ranging from average to lower ability. In the 1st study, 7th-grade mathematics students from intact classes experienced 1 of 2 conditions: (a) PF, in which students collaboratively solved complex problems on average speed without any instructional support or scaffolds up until a teacher-led consolidation; or (b) direct instruction (DI), in which the teacher provided strong instructional support, scaffolding, and feedback. Findings suggested that although PF students generated a diversity of linked representations and methods for solving the complex problems, they were ultimately unsuccessful in their problem-solving efforts. Yet despite seemingly failing in their problem-solving efforts, PF students significantly outperformed DI students on the well-structured and complex problems on the posttest. They also demonstrated greater representation flexibility in solving average speed problems involving graphical representations, a representation that was not targeted during instruction. The 2nd and 3rd studies, conducted in schools with students of significantly lower mathematical ability, largely replicated the findings of the 1st study. Findings and implications of PF for theory, design of learning, and future research are discussed.

Productive failure in CSCL groups

This study was designed as a confirmatory study of work on productive failure (Kapur, Cognition and Instruction, 26(3), 379–424, 2008). N = 177, 11th-grade science students were randomly assigned to solve either well- or ill-structured problems in a computer-supported collaborative learning (CSCL) environment without the provision of any external support structures or scaffolds. After group problem solving, all students individually solved well-structured problems followed by ill-structured problems. Compared to groups who solved well-structured problems, groups who solved ill-structured problems expectedly struggled with defining, analyzing, and solving the problems. However, despite failing in their collaborative problem-solving efforts, these students outperformed their counterparts from the well-structured condition on the individual near and far transfer measures subsequently, thereby confirming the productive failure hypothesis. Building on the previous study, additional analyses revealed that neither preexisting differences in prior knowledge nor the variation in group outcomes (quality of solutions produced) seemed to have had any significant effect on individual near and far transfer measures, lending support to the idea that it was the nature of the collaborative process that explained productive failure.

Productive Failure in Learning Math

When learning a new math concept, should learners be first taught the concept and its associated procedures and then solve problems, or solve problems first even if it leads to failure and then be taught the concept and the procedures? Two randomized-controlled studies found that both methods lead to high levels of procedural knowledge. However, students who engaged in problem solving before being taught demonstrated significantly greater conceptual understanding and ability to transfer to novel problems than those who were taught first. The second study further showed that when given an opportunity to learn from the failed problem-solving attempts of their peers, students outperformed those who were taught first, but not those who engaged in problem solving first. Process findings showed that the number of student-generated solutions significantly predicted learning outcomes. These results challenge the conventional practice of direct instruction to teach new math concepts and procedures, and propose the possibility of learning from ones own failed problem-solving attempts or those of others before receiving instruction as alternatives for better math learning.

Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment

This paper argues for a need to develop methods for examining temporal patterns in computer-supported collaborative learning (CSCL) groups. It advances one such quantitative method—Lag-sequential Analysis (LsA)—and instantiates it in a study of problem-solving interactions of collaborative groups in an online, synchronous environment. LsA revealed significant temporal patterns in CSCL group discussions that the commonly used “coding and counting” method could not reveal. More importantly, analysis demonstrated how variation in temporal patterns was significantly related to variation in group performance, thereby bolstering the case for developing and testing temporal methods and measures in CSCL research. Findings are discussed, including issues of reliability, validity, and limitations of the proposed method.

Sensitivities to early exchange in synchronous computer-supported collaborative learning (CSCL) groups

This study reports the impact of high sensitivity to early exchange in 11th-grade, CSCL triads solving well- and ill-structured problems in Newtonian Kinematics. A mixed-method analysis of the evolution of participation inequity (PI) in group discussions suggested that participation levels tended to get locked-in relatively early on in the discussion. Similarly, high (low) quality member contributions made earlier in a discussion did more good (harm) than those made later on. Both PI and differential impact of member contributions suggest a high sensitivity to early exchange; both significantly predicting the eventual group performance, as measured by solution quality. Consequently, eventual group performance could be predicted based on what happened in the first 30-40% of a discussion. In addition to drawing theoretical and methodological implications, implications for scaffolding CSCL groups are also discussed.

Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning

Learning and performance are not always commensurable. Conditions that maximize performance in the initial learning may not maximize learning in the longer term. I exploit this incommensurability to theoretically and empirically interrogate four possibilities for design: productive success, productive failure, unproductive success, and unproductive failure. Instead of only looking at extreme comparisons between discovery learning and direct instruction, an analysis of the four design possibilities suggests a vast design space in between the two extremes that may be more productive for learning than the extremes. I show that even though direct instruction can be conceived as a productive success compared to discovery learning, theoretical and empirical analyses suggests that it may well be an unproductive success compared with examples of productive failure and productive success. Implications for theory and the design of instruction are discussed.

Comparing Learning from Productive Failure and Vicarious Failure.

A total of 136 eighth-grade math students from 2 Singapore schools learned from either productive failure (PF) or vicarious failure (VF). PF students generated solutions to a complex problem targeting the concept of variance that they had not learned yet before receiving instruction on the targeted concept. VF students evaluated the solutions generated by PF students before receiving the same instruction. Student-generated solutions were either suboptimal or incorrect, and in this sense can be conceived as failed problem-solving attempts. Although there was no difference on self-reported engagement, PF students reported significantly greater mental effort and interest in knowing the canonical solution to the problem than VF students. When preexisting differences in general ability, math ability, and prior knowledge were controlled, PF students outperformed VF students on conceptual understanding and transfer without compromising procedural fluency. These results suggest that when learning a new math concept, people learn better from their own failed solutions than those of others provided appropriate instruction on the targeted concept is given after the generation or evaluation activity.

The assistance dilemma in CSCL

How to design structure for supporting collaborative learning is a fundamental theoretical and design issue in CSCL research. At the center of this issue lies an assistance dilemma: when to provide support structures and when to withhold them (at least temporarily) to optimize student learning. On the one hand, providing support structures right from the start has the advantage of reducing cognitive load, avoiding floundering and potential frustration. It may well lead to productive success, but there is also the danger of unproductive success--an illusion of performance without learning. On the other hand, withholding support may well lead to productive failure as students persist in active sense-making and problem-solving activities, but there is the danger of unproductive failure in students being overwhelmed. This symposium aims to interrogate issues pertinent to the assistance dilemma continuum by bringing together an eclectic group of CSCL researchers with commitments on various points on the continuum.

Learning from Productive Failure

I advance a theoretically and empirically-grounded case for designing for and learning from failure, and instantiate it in a learning design called Productive Failure (PF). I describe the key mechanisms and the design principles of PF. The PF learning design comprises a generation and exploration phase followed by a consolidation and knowledge assembly phase. Findings show that the PF learning design is more effective in developing conceptual understanding and transfer than a direct instruction design. Follow-up studies are described wherein key aspects of the productive failure design were tested over multiple classroom-based studies as well as controlled experiments, and how these studies helped us interrogate and understand the criticality of key mechanisms embodied in the PF design. Implications for the learning theory and the design of instruction are discussed by situating findings in the long-standing instructivist-constructivist debate.

Conceptualizing Debates in Learning and Educational Research: Toward a Complex Systems Conceptual Framework of Learning

This article proposes a conceptual framework of learning based on perspectives and methodologies being employed in the study of complex physical and social systems to inform educational research. We argue that the contexts in which learning occurs are complex systems with elements or agents at different levels—including neuronal, cognitive, intrapersonal, interpersonal, cultural—in which there are feedback interactions within and across levels of the systems so that collective properties arise (i.e., emerge) from the behaviors of the parts, often with properties that are not individually exhibited by those parts. We analyze the long-running cognitive versus situative learning debate and propose that a complex systems conceptual framework of learning (CSCFL) provides a principled way to achieve a theoretical rapprochement. We conclude with a consideration of more general implications of the CSCFL for educational research.