Nikol Rummel

Learning to Collaborate: An Instructional Approach to Promoting Collaborative Problem Solving in Computer-Mediated Settings

Effective collaboration in computer-mediated settings among spatially distributed people is a precondition for success in many new learning and working contexts but it is hard to achieve. We have developed two instructional approaches to improve collaboration in such settings by promoting peoples capabilities to collaborate in a fruitful way and furthering their understanding of what characterizes good collaboration. The rationale is that strategies necessary for a good and effective computer-mediated collaboration may be conveyed to people by exposing them to an elaborated worked-out collaboration example (observational learning) or by giving them the opportunity to learn from scripted collaborative problem-solving. An experimental study was conducted that compared learning from observing a worked-out collaboration example with the learning effects of scripted collaborative problem-solving, the effects of unscripted collaborative problem-solving, and a control condition without a learning phase. The experimental design provided clearly separated phases for the instructional treatments (learning phase) and for applying and testing the acquired skills (application phase). Both observing a worked-out collaboration example and collaborating with a script during the learning phase showed positive effects on process and outcome of the second collaboration in the application phase.

A rating scheme for assessing the quality of computer-supported collaboration processes

The analysis of the process of collaboration is a central topic in current CSCL research. However, defining process characteristics relevant for collaboration quality and developing instruments capable of assessing these characteristics are no trivial tasks. In the assessment method presented in this paper, nine qualitatively defined dimensions of collaboration are rated quantitatively: sustaining mutual understanding, dialogue management, information pooling, reaching consensus, task division, time management, technical coordination, reciprocal interaction, and individual task orientation. The data basis for the development of these dimensions was taken from a study in which students of psychology and medicine collaborated on a complex patient case via a desktop-videoconferencing system. A qualitative content analysis was performed on a sample of transcribed collaboration dialogue. The insights from this analysis were then integrated with theoretical considerations about the roles of communication, joint information processing, coordination, interpersonal relationship, and motivation in the collaboration process. The resulting rating scheme was applied to process data from a new sample of 40 collaborating dyads. Based on positive findings on inter-rater reliability, consistency, and validity from this evaluation, we argue that the new method can be recommended for use in different areas of CSCL.

Learning to collaborate while being scripted or by observing a model

In an earlier study, we had tested if observing a collaboration model, or alternatively, following a collaboration script could improve students’ subsequent collaboration in a computer-mediated setting and promote their knowledge of good collaboration. Both model and script showed positive effects. The current study was designed to further probe the effects of model and script by comparing them to conditions in which the learning was supported by providing elaboration support (instructional prompts and a reflective self-explanation phase). In addition, we applied a newly developed, innovative rating scheme to analyze the collaborative process: The rating scheme combines qualitative evaluation with quantitative assessment. Forty dyads were tested, eight in each of the following conditions: model plus elaboration, model, script plus elaboration, script, and control. Observing a collaboration model with elaboration support yielded the best results over all other conditions on measures of the quality of collaborative process and on outcome variables. Model without elaboration was second best. The results for the script conditions were mixed; on some variables, even below those of the control condition. The results of the current study lead us to challenge the positive view on collaboration scripts prevalent in CSCL research. We propose adaptive scripting as a possible solution.

Are two heads always better than one? Differential effects of collaboration on students’ computer-supported learning in mathematics

While some studies found positive effects of collaboration on student learning in mathematics, others found none or even negative effects. This study evaluates whether the varying impact of collaboration can be explained by differences in the type of knowledge that is promoted by the instruction. If the instructional material requires students to reason with mathematical concepts, collaboration may increase students’ learning outcome as it promotes mutual elaboration. If, however, the instructional material is focused on practicing procedures, collaboration may result in task distribution and thus reduce practice opportunities necessary for procedural skill fluency. To evaluate differential influences of collaboration, we compared four conditions: individual vs. collaborative learning with conceptual instructional material, and individual vs. collaborative learning with procedural instructional material. The instruction was computer-supported and provided adaptive feedback. We analyzed the effect of the conditions on several levels: Logfiles of students’ problem-solving actions and video-recordings enabled a detailed analysis of performance and learning processes during instruction. In addition, a post-test assessed individual knowledge acquisition. We found that collaboration improved performance during the learning phase in both the conceptual and the procedural condition; however, conceptual and procedural material had a differential effect on the quality of student collaboration: Conceptual material promoted mutual elaboration; procedural material promoted task distribution and ineffective learning behaviors. Consequently, collaboration positively influenced conceptual knowledge acquisition, while no positive effect on procedural knowledge acquisition was found. We discuss limitations of our study, address methodological implications, and suggest practical implications for the school context.

A new method to assess the quality of collaborative process in CSCL

In CSCL research, the collaborative process - the way people collaborate while working on tasks and learning -- is of central importance. Instructional measures are being developed to improve the quality of the collaboration which itself determines to a great extent the results of working and learning in groups. However, assessing collaborative process is not easy. We have developed a new assessment method by quantitatively rating nine qualitatively defined characteristic dimensions of collaboration. In this paper, we first describe how these dimensions were extracted from video-recordings of dyads collaborating to solve interdisciplinary tasks. Then we explain how the resulting rating system was applied to and tested on another sample. Based on positive findings from this application, we argue that the new method can be recommended for different areas of CSCL research.

Can People Learn Computer-Mediated Collaboration by Following A Script?

Our central hypothesis is that partners who jointly work on a task in a computer-mediated setting following a collaboration script, can acquire collaborative skills that will help to improve the collaboration in subsequent tasks as well as their outcome. In an experimental study, a collaboration script was provided for a first computer-mediated collaboration in one experimental condition. Meantime, in a different experimental condition, the collaborators observed a model-collaboration. Learning effects of script and model were expected to become evident in the process and outcome of a second, unscripted computer-mediated collaboration. Compared to two control conditions (a condition with unsupported collaboration during the learning phase and a condition without a learning phase) both the script condition and the model condition showed positive effects on process and outcome during the application phase. This leads to the conclusion that collaboration scripts can indeed constitute a promising instructional method to promote collaborative competences and to improve subsequent computer-mediated collaboration.

Designing Automated Adaptive Support to Improve Student Helping Behaviors in a Peer Tutoring Activity

Adaptive collaborative learning support systems analyze student collaboration as it occurs and provide targeted assistance to the collaborators. Too little is known about how to design adaptive support to have a positive effect on interaction and learning. We investigated this problem in a reciprocal peer tutoring scenario, where two students take turns tutoring each other, so that both may benefit from giving help. We used a social design process to generate three principles for adaptive collaboration assistance. Following these principles, we designed adaptive assistance for improving peer tutor help-giving, and deployed it in a classroom, comparing it to traditional fixed support. We found that the assistance improved the conceptual content of help and the use of interface features. We qualitatively examined how each design principle contributed to the effect, finding that peer tutors responded best to assistance that made them feel accountable for help they gave.

CTRL: A research framework for providing adaptive collaborative learning support

There is evidence suggesting that providing adaptive assistance to collaborative interactions might be a good way of improving the effectiveness of collaborative activities. In this paper, we introduce the Collaborative Tutoring Research Lab (CTRL), a research-oriented framework for adaptive collaborative learning support that enables researchers to combine different types of adaptive support, particularly by using domain-specific models as input to domain-general components in order to create more complex tutoring functionality. Additionally, the framework allows researchers to implement comparison conditions by making it easier to vary single factors of the adaptive intervention. We evaluated CTRL by designing adaptive and fixed support for a peer tutoring setting, and instantiating the framework using those two collaborative scenarios and an individual tutoring scenario. As part of the implementation, we integrated pre-existing components from the Cognitive Tutor Algebra (CTA) with custom-built components. The three conditions were then compared in a controlled classroom study, and the results helped us to contribute to learning sciences research in peer tutoring. CTRL can be generalized to other collaborative scenarios, but the ease of implementation relates to the complexity of the existing components used. CTRL as a framework has yielded a full implementation of an adaptive support system and a controlled evaluation in the classroom.

Computer Supported Collaborative Learning and Intelligent Tutoring Systems

In this chapter we discuss how recent advances in the field of Computer Supported Collaborative Learning (CSCL) have created the opportunity for new synergies between CSCL and ITS research. Three “hot” CSCL research topics are used as examples: analyzing individual’s and group’s interactions, providing students with adaptive intelligent support, and providing students with adaptive technological means.

Adaptive Intelligent Support to Improve Peer Tutoring in Algebra

Adaptive collaborative learning support (ACLS) involves collaborative learning environments that adapt their characteristics, and sometimes provide intelligent hints and feedback, to improve individual students’ collaborative interactions. ACLS often involves a system that can automatically assess student dialogue, model effective and ineffective collaboration, and provide relevant support. While there is evidence that ACLS can improve student learning, little is known about why systems that incorporate ACLS are effective. Does relevant support improve student interactions by providing just-in-time feedback, or do students who believe they are receiving relevant support feel more accountable for the collaboration, and thus more motivated to improve their interactions? In this paper, we describe an adaptive system we have developed to support help-giving during peer tutoring in high school algebra: the Adaptive Peer Tutoring Assistant (APTA). To validate our approach, we conducted a controlled study that demonstrated that our system provided students with more relevant support and was more effective at improving student learning than parallel nonadaptive conditions. Our contributions involve generalizable techniques for implementing ACLS that can function adaptively and effectively, and the finding that adaptive support does indeed improve student learning because of the relevance of the support.