Nafisa Awwal

Collaborative Problem Solving Tasks

This chapter outlines two distinct types of collaborative problem solving tasks – content-free and content-dependent – each allowing students to apply different strategies to solve problems collaboratively. Content-free tasks were developed to emphasise the enhancement of inductive and deductive thinking skills. Content-dependent tasks allow students to draw on knowledge gained through traditional learning areas or subjects within the curriculum. The collaborative problem solving framework emphasises communication for the purpose of information gathering, identification of available and required information, identification and analysis of patterns in the data, formulation of contingencies or rules, generalisation of rules, and test hypotheses. Characteristics of tasks which were identified as appropriate for eliciting collaborative problem solving processes are reported and illustrated by exemplar items.

Automatic Coding Procedures for Collaborative Problem Solving

This chapter examines the procedure followed in defi ning a scoring process to enable the reporting of individual student results for teachers to use in the classroom. The procedure begins with the identifi cation of task features that match elements of the skills frameworks, and is followed by the generation of simple rules to collect data points to represent these elements. The data points are extracted from log fi les generated by students engaged in the assessment tasks and consist of the documentation of each event, chat and action from each student. The chapter includes examples of the process for defi ning and generating global and local (task specifi c) indicators, and examples of how the indicators are coded, scored and interpreted. The development of coding and scoring of data generated when students engage in collaborative problem solving tasks is described. The data generated are captured in a process stream data fi le. Patterns of these data are coded as indicators of ele- ments defi ned in the conceptual framework outlined in Hesse et al. ( 2015 ; Chap. 2 ) and the relative complexity of indicators is used in a scoring process. The scored data are then used to calibrate the tasks. The calibrations form the basis of interpre- tation and these are used in forming reports for students and teachers. Figure 6.1 summarises the entire process from task development to the reporting of student ability based on a developmental framework.

Platforms for Delivery of Collaborative Tasks

Prior to the inception of the ATC21STM project, no single platform existed for the implementation of human-to-human internet interactive tasks. (The acronym ATC21STM has been globally trademarked. For purposes of simplicity the acronym is presented throughout the chapter as ATC21S.) Given the global goals of the project, the assessment tasks and the professional development materials for administration and teaching interventions were constructed on several platforms. The characteristics and capacities, and the advantages and disadvantages, of these platforms are described in this chapter. The mathematics-based collaborative tasks were developed on a Hong Kong-based platform and migrated to the online testing framework of the Assessment Research Centre (ARCOTS) at the University of Melbourne. The reasoning-based tasks were developed directly on this platform. Another platform (FADS) was used for the development of the Learning in Digital Networks or ICT (LDN-ICT) tasks. An attempt was made to modify a serious game product but the platform requirements of the developer (Pixelearning) led to this attempt being abandoned. Issues such as automating assignment of login codes, report generation, and real-time synchrony are discussed in detail in this chapter. The chapter highlights factors that discriminate between the maintenance and management of a local system of assessment and teaching materials and the issues that need to be considered when maintaining and managing such a system on a global scale.

Visualising and examining sequential actions as behavioural paths that can be interpreted as markers of complex behaviours

Abstract Visualisation of pathways taken by students through problem solving environments provides the potential to identify patterns of exploratory behaviour that may relate to different levels of proficiency or different cognitive approaches. This is particularly the case when problems are situated in complex online environments and are programmed to generate large scale data for analysis. Using process data from an online collaborative problem solving task, we visualised behavioural paths of 607 pairs of students as directed graphs. The empirical paths were then examined using exploratory network analysis based on four main aspects of exploration (prominence, branches, clusters, and shortest paths). The primary purpose of such analysis is to detect sequences that are potentially relevant for establishing particular paths as meaningful markers of complex behaviours. This visualisation approach enabled us to capture all actual (as opposed to possible ) pathways from the data. Although there may be an optimal pathway through a complex task according to criteria such as efficiency or correct solution, the added human factors bring a dynamic to the activity that provides a richer environment for understanding students’ collaborative processes, both cognitive and social. The method adopted in this study provides a prototype for exploration of other complex skillsets.

Measuring Collaborative Problem Solving Using Mathematics-Based Tasks

This study describes an online method of measuring individual students’ collaborative problem-solving abilities using four interactive mathematics-based tasks, with students working in pairs. Process stream data were captured from 3,000 students who completed the tasks in the United States, Australia, Canada, Costa Rica, Singapore, and Finland. The data were transformed into indicators of collaborative problem-solving ability and were analyzed using item response modeling. The assessments employed in this study can be used as a teaching tool for introduction to algebraic concepts and as a measurement instrument for collaborative problem-solving ability. The paper describes the construction, calibration, and reliability of the tasks and considers validation issues, such as fairness between assessments for both partners and avoidance of cultural biases. Investigations into the dependencies between student scores provide evidence for convergent and discriminant validity.

Sequential actions as markers of behavioural and cognitive processes

This paper presents a novel exploratory method to extract information about behavioural and cognitive processes that occur during the performance of complex tasks. The methodology is framed in a structure where process data on interactions between the task performer and the task space environment are captured in a data stream for big data analytics.We begin by describing a reductionist approach that involves deconstructing complex processes into constituent parts. We extend this approach by looking at sequences of these constituent parts with the aim of detecting patterns in a persons action pathways. We propose the exploration of these pathways as indicators of underlying behavioural and cognitive processes. Current sequence-based approaches are presented and a proposed approach in dealing with sequences as empirical pathways is demonstrated using data stream from an interactive and collaborative online-administered complex task. These empirical pathways were visualised as directed graphs and an exploratory network analysis is proposed to examine specific features of the emergent paths.General challenges encountered during the development of such a method are described and specific future challenges that we foresee in transitioning from an exploratory phase to a confirmatory phase are examined. Finally, possibilities for further research as well as future applications are discussed. An exploratory method to extract information on behavioural and cognitive processes.Performance in complex tasks can be systematically measured and analysed.Behavioural indicators are visualised as a network of directed graph.Exploring dominant paths in the network can yield process information.

Pair interactions in online assessments of collaborative problem solving: case-based portraits

This exploratory case study focuses on how pairs of students can build a shared understanding and acquire collaborative problem-solving (CPS) practices during an online assessment of CPS skills, which is seen in the context of the CPS construct, in a symmetrical and asymmetrical task type. Even though CPS is widely recognised as a core twenty-first-century competency, its nature is not yet well understood. Also, until recently, most of studies have focused on the individual’s solution to a problem or on the skills individuals bring into a problem-solving space. This study extends from an individual- to group-level focus in CPS, emphasising the role and quality of the social aspects in CPS processes and outcomes. Focusing on the group level because it mediates multiple levels of learning, including individual cognition and socio-cultural practices, may provide us with a better understanding of how pairs establish CPS practices. Because of the complexity of CPS and the general challenges of remote collaboration in an online context, the study relies on the triangulation of multiple data sources and phases of analysis. In this paper, the aim is to explore and visualise through contrasting case-based portraits of two pairs how micro-interaction processes evolve at the pair level. The results show that despite students’ similar CPS performance outcome scores and task designs aimed to facilitate collaboration, variations in micro-interactions occur across pairs, for example as individual and joint solution endeavours and as balanced and unbalanced dynamics of group interactions. Studying these patterns at the pair level may provide new insights into CPS and support strategies for acquiring these practices.

Generic log files and algorithms developed for educational multiplayer games

There is potential for online multiplayer educational games to play a vital role to measure cognitive and social abilities among students. Educational games can be developed to capture student responses or actions, both shared and unshared, within the game environment. This paper presents a data capturing process in online games with a focus on the structural components required to measure sufficiently. The first part of this paper introduces a suggested structure of the log files and the second part describes generic algorithms used to extract information from the log files to indicate any observable cognitive and non-cognitive processes of the students. The term generic algorithm refers the common applicability of the algorithms to log files across games and students to provide researchers with rich and meaningful data about student behaviours. A considerable number of generic algorithms were developed to extract salient student behaviour from student activity log files, including intent to achieve efficiency in both present and future development of such complex educational endeavours.