Roos Van Gasse

The impact of collaboration on teachers’ individual data use

ABSTRACT Research considers collaboration to be a significant factor in terms of how teachers use data to improve their practice. Nevertheless, the effects of teacher collaboration with regard to teachers’ individual data use has remained largely underexplored. Moreover, little attention has been paid to the interplay between collaboration and the personal factors that influence teachers’ data use. This paper addresses this research gap by defining factors that affect collaboration, and by investigating the impact of collaboration on teachers’ individual data use. The resulting research questions were answered by drawing on questionnaire data from 1,472 primary and secondary school teachers in Flanders. The findings indicate that collaboration is the main explanatory factor for teachers’ individual data use compared to teachers’ self-efficacy and attitude. Therefore, this study demonstrates the value of collaboration for future research and for creating a supportive environment for teachers’ individual data use.

Feedback opportunities of comparative judgement: an overview of possible features and acceptance at different user levels

Given the increasing criticism on common assessment practices (e.g. assessments using rubrics), the method of Comparative Judgement (CJ) in assessments is on the rise due to its opportunities for reliable and valid competence assessment. However, up to now the emphasis in digital tools making use of CJ has lied primarily on efficient algorithms for CJ rather than on providing valuable feedback. Digital Platform for the Assessment of Competences (D-PAC) investigates the opportunities and constraints of CJ-based feedback and aims to examine the potential of CJ-based feedback for learning. Reporting on design based research, this paper describes the features of D-PAC feedback available at different user levels: the user being assessed (assesse), the user assessing others (assessor) and the user who coordinates the assessment (Performance Assessment Manager (PAM)). Interviews conducted with different users in diverse organizations show that both the characteristics of D-PAC feedback and the acceptance at user level is promising for future use of D-PAC. Despite that further investigations are needed with regard to the contribution of D-PAC feedback for user learning, the characteristics and user acceptance of D-PAC feedback are promising to enlarge the summative scope of CJ to formative assessment and professionalization.

Oranges and Apples? Using Comparative Judgement for Reliable Briefing Paper Assessment in Simulation Games

Achieving a fair and rigorous assessment of participants in simulation games represents a major challenge. Not only does the difficulty apply to the actual negotiation part, but it also extends to the written assignments that typically accompany a simulation. For one thing, if different raters are involved, it is important to assure that differences in severity do not affect the grades. Recently, comparative judgement (CJ) has been introduced as a method allowing for a team-based grading. This chapter discusses in particular the potential of comparative judgement for assessing briefing papers from 84 students. Four assessors completed 622 comparisons in the Digital Platform for the Assessment of Competences (D-PAC) tool. Results indicate a reliability level of 0.71 for the final rank order, which had demanded a time investment around 10.5 h from the team of assessors. Next to this, there was no evidence of bias towards the most important roles in the simulation game. The study also details how the obtained rank orders were translated into grades, ranging from 11 to 17 out of 20. These elements showcase CJ’s advantage in reaching adequate reliability levels for briefing papers in an efficient manner.

Instrumental, conceptual and symbolic effects of data use: the impact of collaboration and expectations

Abstract The contribution of data use in schools has been proven via visible changes in policy and practice in schools (instrumental effects), changes in practitioners learning or cognition (conceptual effects) and changes in opinions or attitudes regarding teaching or policy-making (symbolic effects). Nevertheless, limited research is available on the extent to which data use in schools results in the aforementioned effects and how they can be explained by data use expectations and collaboration. This paper addresses both issues by describing and explaining data use effects via a large-scale study in Flanders. Data collected from 1472 teachers indicate that, although teachers are moderately positive about the extent to which data use results in different types of effects, data use effects cannot be taken for granted. Structural Equation Modelling (SEM) shows that explicating data use expectations and data use collaboration are essential in order to facilitate data use effects in schools.

The impact of school culture on schools’ pupil well-being policy-making capacities

Abstract Pupil well-being has been an important topic in educational research for some time. Differences between schools in their influence on the well-being of their pupils are attributed to the policy-making capacities of the school. Little is known about schools’ policy-making capacities with regard to pupil well-being, and the impact of school culture on these policy-making capacities. This study addresses these questions using a quantitative research approach. The results show that teachers rate the policy-making capacities of their school, with regard to pupil well-being, in largely positive terms and that these policy-making capacities are mainly affected by flexible perspectives of effectiveness within the culture of schools.