Combining surveys and sensors to explore student behaviour
AArticle
Combining surveys and sensors to explore studentbehaviour
Inkeri Kontro * and Mathieu Génois University of Helsinki, Helsinki, Finland CNRS, CPT, Aix Marseille Univ, Université de Toulon, Marseille, France GESIS, Leibniz Institute for the Social Sciences, Köln, Germany * Correspondence: inkeri.kontro@helsinki.fi† These authors contributed equally to this work.Received: date; Accepted: date; Published: date
Abstract:
Student belongingness is important for successful study paths, and group work forms animportant part of modern university physics education. To study the group dynamics of introductoryphysics students at the University of Helsinki, we collected network data from seven laboratory coursesections of approximately 20 students each for seven consecutive weeks. The data was collected via theSocioPatterns platform, and supplemented with students’ major subject, year of study and gender. Wealso collected the Mechanics Baseline Test to measure physics knowledge and the Colorado LearningAttitudes about Science Survey to measure attitudes. We developed metrics for studying the smallnetworks of the laboratory sessions by using connections of the teaching assistant as a constant. In thenetwork, we found both demographically homogeneous and heterogeneous groups that are stable. Whilesome students are consistently loosely connected to their networks, we were not able to identify riskfactors. Based on our results, the physics laboratory course is equally successful in building stronglyconnected groups regardless of student demographics in the sections or the formed small groups.SocioPatterns supplemented with surveys thus provides an opportunity to look into the dynamics ofstudents’ social networks.
Keywords: small group; undergraduate laboratory; physics education; network; student retention
1. Introduction
Laboratory skills are an essential part of university physics curriculum, and recently much attentionhas been devoted to what laboratory courses should teach to students [1]. Traditionally, physics laboratorycourses have been attempting to deepen physics knowledge by introducing physics concepts in a newsetting, and often they have offered little freedom of experimental procedure. These “cookbook labs” havebeen heavily criticized, as they neither build conceptual knowledge, nor expertise in laboratory skills [2,3].Many curriculums for investigative laboratory activities exist. Some combine lecture and laboratorycourses into a single curriculum. For example, the Investigative Science Learning Environment [5,6],Studio Physics [7] and SCALE-UP [8] use an investigative, open process for developing both conceptualand experimental skills.Another approach is to separate the teaching of physics knowledge and laboratory skills completely.The American Association of Physics Teachers (AAPT) has given out guidelines for physics laboratorycourses, which emphasize the role of acquired laboratory skills [4]. In this approach, the laboratorycourses are constructed around the desired laboratory skills, rather than physics content [9,10]. Laboratory a r X i v : . [ phy s i c s . e d - ph ] M a r of 22 skills include e.g. designing experiments, modelling of physical systems and reporting results, but alsointerpersonal skills such as collaboration [1,4]. The role of collaboration in learning is naturally not limited to a learning goal in experimental physics.Collaborative assignments improve learning and induce conceptual change in students. In physics,discussion-based strategies such as peer instruction are popular. Peer discussions increase the learning ofconcepts [11,12]. Guided peer reflection increases the use of diagrams and lead to improved learning gains[13]. The use of peer discussions also improve student attrition [14].The quality of the students’ discussion is important for individual learning outcomes [15]. Students,who have a tool to test a hypothesis and guidance to accommodate divergent views did well both indiscussions and individually [16]. Peer discussions can even lead to co-construction of knowledge, wherestudents after a discussion can answer a question neither could answer before [12]. Hence, collaboration isimportant in students’ construction of knowledge.An important question is how to form the groups. Currently, no consensus exists for whether groupsshould be mixed (heterogeneous) or matched by ability (homogeneous). For example, Webb et al. foundthat high-achievers performed more uniformly in homogeneous groups, but a better predictor than groupcomposition was the quality of the group functioning [17]. Cheng et al. similarly found that groupcomposition did not matter, whereas the quality of group processes did [18]. In any case, in mathematicsand natural science, the effect seems small [19]. Also results from college level education vary. A recentstudy Harlow et al. [20] found no link between group composition and learning gains in introductoryphysics, but another study in biology classes found that students with low initial knowledge benefitedfrom homogeneous groups [21].Another important factor influencing group work is that when allowed to form freely, subgroups tendto be less diverse than the group they are drawn from [35]. Interactions within a group are naturally notalways equal. In a computational study Koponen et al. [36] found that variations in only two interactionpotentials (“competivity” and “cooperativity”) led to various group structures. In this study, certaininteraction potentials led to group members being excluded from the group. The (lack of) diversity in asmall-group may be due to apparent factors, such as gender, but also the underlying factors (differences inknowledge) discussed above [35].However, the effect of collaboration is not limited to learning and constructing knowledge. Socialnetworks can be characterized through measurements of the importance of a node (person) to the network.For example, betweenness centrality is a measure of how many shortest paths between different nodesgo through a certain node, and thus how important this node is for contacts. These kinds of networkcentrality measures can be combined with student information to predict student performance. Forexample, when examining networks built from student surveys, network centrality measures and gradescorrelate significantly, and network centrality measures even predict future grades [22]. Similarly, networksbuilt from survey data seem to predict student retention [23].However, network analysis in physics largely relies student answers on whom they remember talkingto. Monitoring the evolution of a network is time-consuming and difficult, and the method is less suitedfor monitoring student activity in a single course, such as a laboratory course. While the laboratorycould be monitored by video or audio recording or observations, the analysis of this kind of data is verytime-consuming. Following students in large courses can be prohibitively costly. of 22
Networks have proven to be a very useful framework to study social structures [47], as they allow tonaturally encode the relations (links) between individuals (nodes). By studying the network structure, onecan then extract relevant information about the context it models.One example is the analysis of the position of an individual in a context. In particular, detectingindividuals that are central can be of high interest: these individuals can constitute bottlenecks forinformation flows or spreading processes such as rumor or epidemic spreading. There exist many centrality measures [48], but the most used are degree (the number of nodes to which one node is connected),the strength (the sum of the weights of the links surrounding one node) and the betweenness centrality (thefraction of shortest paths passing through one node). Centrality measures also exists for links, with similardefinitions, when one wants to assess which connections play a significant role in a network. However,centrality measures are useful only when one considers networks that are large enough, in order to havea range of values that allows to rank the nodes/links. In small, dense networks, all nodes and links aremore or less equivalent with respect to these measures.Another example is the identification of groups in networks, known as community detection .Community detection is a vast topic, and many methods exist to identify relevant groups from thestructure of a network [49]. One of the standard ones is the maximisation of modularity [50]. Again, thesemethods are usually tailored for sufficiently large networks, and are not useful when dealing with small,dense networks.Network analysis provides many tools to extract information from social systems. It is a frameworkthat allows to reduce the complexity of the original situation while keeping all its relevant structures. Asa conclusion on its usefulness, let us note that while network analysis usually consists in abstracting asocial system as a static network that models the connections between agents, a newer framework hasbeen emerging in the last decades, which allows to take into account the dynamics of such structures. Theanalysis of such temporal networks has and will also provide many new insights to social sciences[51,52].
In the past few decades, the development of physics teaching has led to the adoption of new tools.More and more, students are given diagnostic tests which are used to assess their initial knowledge andwhat they learned in instruction. These diagnostic tests can be focused solely on physics concepts (e.g. theForce Concept Inventory, FCI [24]) or they can combine concepts and general physics problem solving (e.g.Mechanics Baseline Test, MBT [25]).Conceptual learning, as measured by conceptual tests, does not necessarily correlate with problemsolving ability. In fact, the development of these tests was sparked by the notion that beginning physicsstudents hold many common misconceptions in physics, and that instruction is often not able to addressthese misconceptions and to induce conceptual change in student reasoning [26].In addition to the conceptual knowledge and conceptual gains of physics students, the expert-likeattitudes of students have become a more important topic of study, and several instruments have beendeveloped to study the evolution of student attitudes. These instruments relate student attitudes to thoseof experts. The Colorado Learning Attitudes about Science Survey (CLASS) is an instrument which hasquestions that relate to physics studies broadly, rather than to individual courses [27]. CLASS consists of42 statements, which are scored by a five point Likert scale.These expert-like beliefs are often seen as a desired learning goal as such. They have also beenshown to correlate weakly with learning [28–30]. They also correlate with experiencing high levels ofchallenge, interest and skill at the same time (optimal learning moments) [31]. However, the developmentof expert-like attitudes is not straightforward. Generally, expert-like beliefs decline with instruction [27,32]. of 22
Exceptions are mainly courses that focus on modeling [33]. In many cases, students also know whatexperts think, but they do not agree when it comes to their own learning [34].
In this study, we examine the social dynamics of an introductory laboratory course. This laboratorycourse is a course with higher than average student attrition. A common criticism from introductoryphysics students is that it can be hard to get to know other students, and the laboratory, with its groupactivities, is an important place for building up social contacts. However, we do not know what factorsinfluence the development of social contacts in the laboratory.Our research questions were as follows: • Can we identify risk factors for being exluded out of laboratory practice? • Do students’ pre-existing knowledge influence their risk of dropping out of the course?To do this, we built a network of student contacts in the lab course, and collected information on thestudent demographics: major, year of study and gender. In this paper, we will show that we can use therole of the teaching assistant in the network to identify the groups working together and hence the stronglyand weakly connected students. We also show that on this course, students of all studied demographicshave equal roles in the network.
2. Materials and Methods
The introductory laboratory course at the University of Helsinki runs concurrent with the lecturecourses in introductory physics, and is mandatory for all students who complete at least 25 credits(European Credit Transfer and Accumulation System, ECTS) of physics. The majority of students on thiscourse are first-year physics majors, and the laboratory is one important place to learn to know otherstudents. Six laboratory assignments are completed each term, and each assignment spans 2-3 weeks,giving students freedom to experiment and iterate their measurement set-up. The course is set up in sevensections, with up to 20 students present at the same time. The assignments are done in small-groups of 3-5students.In total, the number of students on the laboratory course is around 140-150 in the beginning of theyear. The majority of the participants ( 60 %) are physics major students. The rest are physics minorstudents, mostly other science majors, or non-degree students participating through open university. Animportant category of students are pre-service teachers who have mathematics as their first and physics astheir second subject.We have identified the laboratory course as a course which students easily drop out of. To see whetherwe could identify risk factors in the complex social dynamics of laboratory work, we set out to explorethe social dynamics by using the SocioPattern platform. We wanted to see both how social networksevolve in an undergraduate laboratory, and whether we can find correlations between demographics,initial knowledge and being excluded out of collaborative learning in the laboratory.The students form the small-groups during the first week. They are allowed to change sections and tovisit other sections, but the recommendation is to work with the same people the whole year. Subgroupsin general are less diverse than the group they are drawn from [35], and we assumed this also to be thecase for small groups formed by the students themselves during the beginning of their physics studies.Hence, we collected information on major subject, year of study and gender. We wanted to go beyondsurface-level diversity measures and also collected a pre-test on mechanics knowledge (MBT, [25]) and of 22 the attitude survey CLASS, [27]. MBT was chosen because the incoming students at UH saturate manyconceptual surveys in mechanics, including the FCI.
The SocioPatterns platform is a tool developed by the SocioPatterns collaboration to collectinformation about human interactions in the physical world in an automated, observer-free way[37].The original goal was to detect transmission routes of airborne diseases. However, it has since been widelyused to study patterns in human interactions in order to analyse social phenomena[38–43]The system is designed to detect face-to-face contacts between individuals. It consists in sensors thatare worn by the participants (Figure 1), able to detect each other at short range (1.5 meters maximum)through the use of RFID chips and radio emitters. Furthermore, as the signal used for the detectionis blocked by the body, detection is only possible when the individuals are in their respective fronthalf-spheres. A contact as detected by this method is thus defined as a physical proximity (less than 1.5meters), where both individuals are facing each other (they are located in the front space of each other). Figure 1. SocioPatterns sensor.
Front aspect and scale.
Sensors are calibrated so that a contact lasting 20 seconds will be detected with probability ∼
100 %.Contacts lasting less than 20 seconds are detected with a probability which decreases with their duration.This calibration sets the temporal resolution of the system: contacts are recorded every 20 seconds. Theminimum contact duration is therefore set at 20 seconds. The internal detection system of the sensorslimits the number of simultaneous contacts to 25 per interval of 20 seconds.The sensors have a very limited built-in memory: antennas are used to collect and store the contactdata. As a consequence, only the areas covered by these antennas are monitored: any contact occurringoutside will not be recorded. Antennas have a theoretical range of detection of 30 meters, but it is limitedby the presence of obstacles such as walls.
The studied location consists in a single classroom, intended for practical classes in experimentalphysics. It consists in two rows of three tables. The room is 11.5 meters long and 9.6 meters wide (Figure2), allowing for the complete location to be monitored by a single antenna. CorridorTTT TTT
Sensors box Antenna
Figure 2. Setup of the study.
Tables are represented by the rectangles with a capital T. The position of theantenna is signalled by the target symbol.
Students are distributed in seven sections. The class takes place every week on Thursday or Fridaydepending on the section, and lasts two hours (Table 1). Students are assigned to a section based on theirstated preference but may on occasion come to a different one if needed.
Table 1. Student sections.Thursday Friday
Section 1 08:15 to 10:00 Section 6 08:15 to 10:00Section 2 10:15 to 12:00 Section 7 10:15 to 12:00Section 3 12:15 to 14:00Section 4 14:15 to 16:00Section 5 16:15 to 18:00
Because of equipment limitations, sensors are not assigned to students. Instead, at each beginningof a class students pick a sensor to wear in a box and writes down on a sheet the sensor number alongwith their student number. At the end of the class, sensors are put back to the box. This setup allows for abetter cleaning and separation of the contact data afterwards (see next section). The teaching assistant isassigned to a section, and follows the same procedure. Data collection is started before beginning of thefirst class of the day and stopped after the end of the last one. It is collected continuously in between.
Once pre-processed from the raw data, the data collected comes in the form of a tij file, in which eachline is a contact occurring at time t between sensors i and j (Figure 3). Figure 3. Example of a tij file.
This lists 10 contacts, all occurring on time 1536214160. The first lineindicates that the contact occurred between sensors 1037 and 1665. of 22
Since the same sensor can be used by several students during the day, we use the sheets linking sensorand student number to reconstruct the identity of the students from the data. The method is the following:1. From the sheet with sensor and student numbers, we build for each sensor the list of student whohave used it, with their section number.2. From the same sheet we extract the list of sensors that were used during the day.3. From the data, we extract the raw contact activity timelines of all used sensors. Because we imposethe students to put the sensors back in the box at the end of a class, and using the fact the there arealways sensors remaining unused in the box, we are able to automatically detect the exact times atwhich a sensor is taken out of the box and put back in. Indeed, the contact activity drops when thesensor is taken out, as it loses contact with the sensors remaining in the box, and it jumps when thesensor is put back in, as it detects the presence of the sensors in the box. The contact activity is spiky,so in order to improve detection we smooth the curve by averaging on a sliding time window, thenwe impose a detection threshold. We mark all the times the smoothed curve crosses the threshold:low activity periods thus defined are the activity windows of the sensor (see Figure 4).4. The automatic detection is not perfect, so we check by hand that all activity windows were detectedcorrectly, and update their definitions where it is necessary.5. We compare the detected activity windows to the theoretical ones extracted from the sheet, in orderto identify activity windows that might be missing due to sensor malfunctions.6. Using the corrected activity windows, we replace the sensor number in the data with an anonymousstudent ID, remove all contacts between activity windows and involving unused sensors. c o n t a c t s time Figure 4. Example of raw contact activity timeline.
The red line is the raw contact activity ( x axis is timein seconds, y axis the number of contacts recorded per time step). The green line is the smoothed contactactivity, the blue crosses mark the automatically detected times where the smoothed activity drops belowthe detection threshold, defining the activity windows of the sensor. In the present example, we see that thesensor 1052 has been used in section 1, 2, 4 or 5, but not in section 3. At the end of this procedure, we have cleaned data about contacts between students in the same formatas Figure 3, using anonymous student ID instead of the sensor numbers (a file linking the anonymous IDto the student number is kept, so that we can later link the contact data to the survey data). This is the datawe then analyse.
A network is comprised of nodes and links, which bind the nodes to each other. The contact dataforms a temporal network, in which nodes are individual students and links are contacts between them,that appear and disappear as time passes. From this rich and complex data, we compute the aggregatednetwork, in which a link exist between two nodes if they have been in contact at least once, and the weightof this link is the total contact duration between the two individuals. of 22
Figure 5. Example of aggregated contact network.
Colours code the five sections of a Thursday. Roundnodes are students, triangle nodes are teaching assistants. The width of a link is proportional to its weight.
As seen on Figure 5, teaching assistants (TAs) always adopt a central position in the networks, whilestudents are aggregated in groups around him/her. The next step is identifying which students areworking together (forming a small-group).Usually, the nodes in a network could be characterized by e.g. betweenness centrality, which is ameasure of how central a node is to the network. Betweenness centrality for a node is calculated bycalculating the shortest paths (number of links weighed by link strength) between all possible pairsof nodes, and calculating the number of shortest paths through each node. Thus, higher betweennesscentrality means a more central place in the network. However, the laboratory networks are too small forthe betweenness centrality to vary much between students. As only a 20 people are in the laboratory at thesame time, no individuals are far from each other in terms of the number of links needed to get from oneperson to another. Also, we have numerous contacts between students from different groups. Instead ofthe usual community detection algorithms, such as the modularity method[50], we use a divisive approach,in which we progressively remove links to make the groups appear.The traditional method is the Girvan-Newman algorithm [44], in which at each step we rank the linkin decreasing betweenness centrality and remove the most central one. This algorithm is based on theassumption that a network is made of densely connected groups joined by few links. However, in our casethe links between groups are too numerous for such a detection to work. Using the weighted betweennesscentrality even worsens the problem, as the strong links exist within the groups. Hence, removing linkswith high betweenness centrality breaks the subgroups, rather than makes them appear. of 22
For these reasons, we use directly the weights of the links as the criterion for link removal. Sincestrong links are within the groups, we remove links starting with the weakest and then following increasingweights. As for the usual method, we then track the depercolation steps : the steps in which the numberof connected components in the network increases, i.e. the steps where a group breaks away from thenetwork (Figure 6). In the beginning (upper left corner of Figure 6) all links, meaning all contacts betweenindividuals, are present). As the weakest links are removed, progressively, subgroups appear. Whenstronger and stronger links are removed, the number of subgroups increases, but progressively groups arebroken up to individuals as the percolation steps reach a point which removes the strong links inside thegroups.
Figure 6. Depercolation steps for the yellow group from Figure 5.
Between each step, links are removedin increasing weights. The steps shown are the one where the remaining network breaks into disconnectedparts. The link(s) responsible for the parting is(are) shown in red. The weight of the links are not shown inthis representation. Round nodes are students, triangle node is the TA.
We keep track of the groups thus formed at each step in Figure 6, and note the link which removal isresponsible for the breaking, along with its weight. This allows us to build a depercolation tree joining thesegroups (Figure 7), similar to the dendrogram what one would get from a clustering algorithm[53]. li n k w e i g h t Figure 7. Depercolation tree for the group of Figure 6.
Branches mark last connections between subgroupsas revealed by the depercolation process. The vertical axes represents the weight of the last link connectingtwo groups, and groups are positioned at the level which they break. The colours of a group code for whichnodes constitutes it. The dashed red horizontal line is the TA threshold, i.e. the point at which the TAbecomes isolated in the depercolation process (in this case individual 54).
This tree gives the whole internal structure of the network in terms of link depercolation. For example,in Figure 7, the group first breaks into a group of four students (61, 128, 79, 116) and the rest of the class.We can thus interpret that this group of four is only weakly connected to the whole group, as all linksconnecting it to the rest have low weights. However, the students in this group of four are stronglyconnected to each other.Such tree gives many possible partitions of the original network, depending on where we cut in termsof limit weight. In our case, we however have a potential natural threshold by using the TA as a reference.
At some point in the depercolation process, the TA becomes isolated in the network. By definition, theTA does not belong to any student group. We can thus consider that links weaker than the strongest linkconnecting the TA to the network are not relevant. In the tree, this is noted as the red dashed line. Allgroups that break before this point ( i.e. above the red line in Figure 7) are coined as weakly connected asthey are connected at best with a link weaker than the threshold. All groups that break after ( i.e. under thered line in Figure 7) are coined as strongly connected and are assumed to be relevant student groups. Inthe example of Figure 7, that means we have one group of six students (Group A: 102, 30, 37, 57, 74, 62),the TA (54), three isolated students (10, 86, 136), a group of four students (Group B: 80, 13, 55, 66) anotherisolated student (99) and the previously spotted group of four students (Group C: 61, 128, 79, 116).The tree also allows us to investigate the inner structure of the groups, by looking at the branchingunder the threshold line. All of them are made of one strong pair, to which the other nodes are singlyconnected: (102, 30) for Group A, (80, 13) for Group B, (116, 79) for Group C. Similarly, looking at thebranching above the threshold line, we can understand how the groups connect to form the whole network: • isolated node 10 connects to Group A; • isolated nodes 86 and 136 form a pair, which then connects to Group A; • isolated node 99 connects to Group B, which then connects to Group A; • Group C connects last to Group A.This analysis provides thus a method to interpret behaviour from the aggregated network, simplifyingthe structure by focusing on the strongest links between them. From these connecting steps, one can thenmake hypotheses about the relations between the students, and the group structure that exists within theclass.
The study takes place during several weeks. We can thus explore the way the structures identifiedthrough the depercolation method evolve with time.To do that, we define the proximity between two individuals as the number of steps necessary to gofrom one to the other in the depercolation tree. The fact that we are dealing with a tree structure ensuresthat this distance is consistent with the groups as we defined them: two individuals belonging to the samegroups will be closer than two individuals from different groups. Having this proximity measure, we canthan compute its evolution as time passes. _ _
06 18 _ _
13 18 _ _
20 18 _ _
27 18 _ _
04 18 _ _
11 18 _ _
102 62 _ _
06 18 _ _
13 18 _ _
20 18 _ _
27 18 _ _
04 18 _ _
11 18 _ _ _ _
06 18 _ _
13 18 _ _
20 18 _ _
27 18 _ _
04 18 _ _
11 18 _ _
136 79 _ _
06 18 _ _
13 18 _ _
20 18 _ _
27 18 _ _
04 18 _ _
11 18 _ _ time time time time d i s t a n c e Figure 8. Evolution of the distance between individuals 102, 62, 136 and 79 and all the other studentsof their section during seven weeks.
Horizontal axis is time, vertical axis are the students from the section.The colours code for the number of steps separating the two individuals in the tree from Figure 7 (seecolour scale on the right), green codes for missing values due to the absence of either student.
From the examples shown in Figure 8, we can see five typical evolution patterns: • stability of a group: whenever the proximity stays high along time, it signals a persistent group, forexample the trios (102, 30, 37) and (79, 116, 128), or the pair (62, 10); • stability of separation, whenever the proximity stays low, for example 62 and 13, 62 and 79, 79 and86, etc; • coming together: when the proximity increases in time, for example 62 and 86; • separation: when the proximity decreases in time, for example 136 and 86; • temporary groups: when groups from occasionally and do not persist, for example 79 and 13 whichare close for two weeks in the middle, 136 and 62 which are close only on 13/09 and 18/10, or 79and 99 which get closer then separate.The results reveal that building on the depercolation trees and the long term monitoring, one can thushave access to the internal dynamics of student groups. All students participating in the laboratory course were asked to complete a background informationform, stating their study track (Physical sciences, Mathematics, Science teacher, Chemistry or Other),study year (1, 2, higher or other, where other accounts for non-degree students) and gender (male, femaleor other), along with their consent to combine these data with data collected from sensors and surveyscollected on introductory physics courses. Students who did not wish to participate were instructed tonot wear a sensor in the laboratory and to decline their consent in the background survey. Filling in theconsent form was compulsory for unlocking the return of lab reports, meaning all students who returnedlaboratory work for grading had to give or decline consent. Hence, voluntary participation and informed consent were ensured. All data was treated anonymously. As the research also did not involve interventionin the physical integrity of the participants, deviation from informed consent, studying children under theage of 15, exposure to exceptionally strong stimuli, causing long-term mental harm beyond the risks ofdaily life, or risks to the security of the participants, the study did not require an ethics review, accordingto the guidelines of Finnish Advisory Board on Research Integrity [45].To collect information on the students’ level of physics knowledge and their attitudes towards learningscience, the MBT and CLASS surveys were administered on the physics course lectured concurrently withthe laboratory course. The MBT and CLASS surveys are administered as a part of homework exercises, andparticipation is rewarded with exercise credit equal to one homework problem. This credit was availableto students regardless of whether they gave consent to use the data for research. The data were collectedelectronically and improper data were discarded. For MBT, using less than 300 seconds and for CLASSless than 250 seconds for finishing the survey were used as cut-off. For CLASS, also having more than 4missing answers, having more than 26 same answers (out of 41), and an incorrect answer to the controlitem (31) were used to discard improper data.The MBT was administered in the first week and CLASS in the second week of studies, which alsocorresponds to the first and second week of the lab course. The students are meant to take the laboratoryand the lecture course at the same time, but many students postpone the laboratory course, and thesesurveys were not compulsory. Out of the students consenting to participate in the study, 79% answeredthe MBT and 75% the CLASS.
3. Results
The final data set includes 151 individuals, of which 144 are students and 7 are TAs. 13 of the studentsdid not return the consent form, meaning they are excluded from further analysis. The number of studentsfor whom also both CLASS and MBT scores were available was 91.
We calculated the number of times each student was present in the laboratory and the number oftimes they were strongly connected, i.e. connected to at least one other student after the percolation stepwhere the TA was detached from the group structure. From this, we calculated the strongly connectedpercentage (Table 2). The numbers are strikingly similar for all student groups: men and women, physicsmajors and other majors, and first year students and non-first year students. Non-first year students werepooled in analysis due to their small number.
Table 2. Student participation according to student type.
Average and standard error of the mean oftimes present, times in a strongly connected group and the percentage of strongly connected groups.Type N Present Strongly connected % Strongly connectedAll 131 5.02 ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± On average, students obtained 15.40 points (59%) in the MBT. Students majoring in physics had aslightly higher average than non-majors, and non-first year students had higher scores than first-year students (Table 3). There is a statistically significant difference between the scores of men and women( p = p = d : d = M − M SD pooled , (1)where M and M are the means and SD pooled is the pooled standard deviation. The pooled standarddeviation is calculated from SD pooled = (cid:115) ( n − ) SD + ( n − ) SD n + n − SD and SD are the standard deviations and n and n the numbers of observation of the samplesto be compared. The effect size for MBT between men and women is d = p = d = Table 3. MBT and CLASS scores by demographics.
Type N MBT CLASSAll 91 15.40 ± ± ± ± ± ± ± ± ± ± ± ± ± ± To study the relationship between the participation metrics (times present, times strongly connectedand percentage of times strongly connected to other students) we calculated the correlation coefficientsbetween them, MBT and CLASS. The results are presented in Table 4. The correlations are very low fornon-dependent variables except for a low to moderate (0.405) correlation between the overall CLASS scoreand the MBT score. The most likely explanation is that a favourable attitude towards learning physicscontributes to good results in a physics problem solving survey, but also that success in physics problemsolving contributes to higher self-efficacy, that is beliefs about one’s abilities in this context, which likelyreflects on the CLASS score.
Table 4. Correlation of participation metrics, MBT and CLASS scores.
Present SC % SC MBT CLASSPresent 1SC 0.589 1% SC -0.139 0.701 1MBT 0.063 0.059 0.014 1CLASS 0.106 -0.041 -0.190 0.405 1
We also wanted to see whether students who were strongly connected in the beginning of the coursewere more likely to participate at the end of the course. For this, we also calculated correlations betweentimes present and times strongly connected in the first four weeks (first two laboratory exercises) of thecourse to those of the remaining weeks (Table 5). Again, the calculated correlations were small, meaningneither the times the student was present early in the course nor the fraction of times they were stronglyconnected to other students predicted participation during the last laboratory exercise. Results weresimilar when only the first two weeks were included in the calculation.
Table 5. Correlation of participation metrics for beginning and end of the half-semester. SC = stronglyconnected.
Present beg. SC beg. Present endPresent beg. 1SC beg. -0.12 1Present end 0.28 -0.10 1
As a measure of learning, we collected the grades for the first laboratory assignment. Only 84 studentshad submitted their report, and for 65 of these students, complete data (including MBT and CLASS) wereobtained. This reflects the known problem in the laboratory course: the high drop-out rate. 30% of studentsdid not receive a grade for this assignment, and hence, did not pass the course during this year.The first laboratory assignment is to measure the mass of clump of clay without the use of an (existing)scale. Solutions that involve building of e.g. balance scales are allowed. This is an open-ended assignment,meaning that the result (mass of clay) is unknown, and that there are a variety of ways to get an estimate.The students in each small-group need to agree on a common experiment and do the measurementstogether. The TA answers their questions and helps them, if they are stuck, but the TAs are instructedto not give solutions, but to use e.g. Socrative questioning to help students along. Hence, it is difficultor impossible to perform the laboratory assignment without group work, but it is of course possible forstudents to engage more or less in their group’s experiment. The other experiments are similarly open.The work is graded through a grading rubric, and for the first assignment, only the results, the(graphical) presentation of them and the uncertainty estimates affect the grade. The learning objectives areexperiment design, presentation and rudimentary error analysis.The students who turned in their work are by most measures equal to those who did not turn intheir work. Surprisingly, the students with a grade have slightly lower MBT scores than those without(15.1 ± ± ( ± ) % and ( ± ) %, respectively).Students with and without grades are similar also in terms of attendance measures (Table 6). Table 6. Participation metrics for students who received or did not receive a grade for the firstassignment. No students received a failing grade.
Type N Present SC % SCPassed 65 5.26 ± ± ± ± ± ± We also calculated the correlations of the participation metrics, MBT, CLASS and grades for the firstassignment (Table 7.) With the exception of the low correlation between MBT and grade ( r = r = r = r = Table 7. Correlation of participation metrics at the beginning of the course, MBT, CLASS and grade forthe first assignment for 65 students. SC = strongly connected.
MBT CLASS Present beg. SC beg. % SC beg. GradeMBT 1CLASS 0.340 1Present beg. 0.096 0.067 1SC beg. 0.111 0.000 0.577 1% SC beg. 0.021 -0.053 -0.005 0.754 1Grade 0.301 0.198 0.210 0.126 -0.045 1
4. Discussion
On average, students participated in ( ± ) sessions during the seven-week period (70%participation rate), and were strongly connected to a small-group ( ± ) times (75% of times present).As can be seen from Table 2, students from all demographics participated equally in the laboratoryactivities. We did not find differences between participation rates of students of different majors, year ofstudy or genders.Physics knowledge, as measured by the MBT, or expert-like attitudes in physics, as measured byCLASS, did also not correlate with participation metrics. Additionally, being strongly connected toa small-group during the first two laboratory activities (the first four weeks) did not correlate withparticipating in the last laboratory exercise (last three weeks). These results are surprising. One wouldexpect that higher physics skills and attitudes towards physics would make the students more committedto coming to class and to work productively. Also, actively attending a class early in term usually makesstudents more likely to attend later, either because they are more committed from the beginning, becausethey have already invested time, or a combination of these.If we cannot determine risk factors from the students who are strongly connected, what about theindividuals who are not? Looking at the individual level, two students always percolated away fromthe tree structure before the TA. An additional seven students were strongly connected only once. Thesestudents were evenly distributed across sections, and their demographics followed those of the course atlarge surprisingly well, considering their small number. For example, five of them (55%) were first-yearstudents, compared to 56% of the whole sample, and 3 (33%) were female, compared to 43% of the sample.Reasons for being excluded out of group work are not naturally limited to neither surface-level nor thedeep diversity measures considered here. In a computational study, variations in interaction (“competivity”and “cooperativity”) led to the formation of different small group dynamics, including exclusion of groupmembers [36]. Our experimental set-up does not allow for monitoring of interaction type or direction, butwe can see similar patterns: some students are consistently outside the small groups.We were not able to identify risk factors for being excluded from small-group activities or droppingout of the course from this sample. This is a positive result: the set-up of the laboratory course seems toserve students of different demographics and with different initial skills equally. While we did not seecorrelations between participation, grades and retention on this single course, on a larger scale, networkcentrality measures have been used to predict grades and student retention [22,23]. Hence, it is importantthat students participate and cooperate in instruction. The participation rate on our course satisfactory, bearing in mind that a laboratory assignment may be completed in one week instead of the two or threethat are allocated to it. Generally, student groups were less diverse than randomly assigned groups would be. For example,in section 2, a particularly homogeneous section which consisted mainly of physics majors in their firstyear, the groups split along gender. In week 3, the students formed three strongly connected single-gendergroups (two female, one male group) and only one mixed-gender group (Figure 9). One student wasloosely connected to the structure.On the other hand, in the more diverse section 5, pre-service teachers formed the majority. In week3, the section had one strongly connected mixed-gender group of first-year teaching students and onesingle-gender pair of first year students, but the other strongly connected groups were heterogeneous bothin terms of major, year of study and gender (Figure 9). While there were fluctuations in group compositionover time, we can track most of these groups over the seven-week period. However, new and transientgroups form in the data.The process of forming the groups has random elements. First, students sign up for the sections bylisting three in order of preference, and sections are formed by administration, who have no knowledgeof friendships or social patterns. Also, as the students sign up to several courses during the same weekwith similar admission processes without cross-checked admissions, some students may want to changesections once all selections are finalised.During the first week, students form groups in the sections by their own accord. They are instructedto work in these groups until the end of the term, but as students’ schedules can change and some studentsdrop the course, there are natural fluctuations in the structure of the small-groups. Students may alsobe absent due to e.g. illness. Groups that end up too small are encouraged by the TA to merge and toolarge groups to split. Naturally, students tend to seek out groups in which they know other students,which may explain the split by gender in section 2 (Figure 9). This is also expected, bearing in mind thatdiversity within groups tends to be smaller than diversity between groups [35]. The data is very richand the analysis could go in several directions. However, in this experiment, we have to account for thepossibility of missing data. For example, in both sections 2 and 5, in week 3, we observe a group thatconsists of a majority of male students but also one female student (Figure 9). Physics education researchliterature has examples of female students not thriving in an otherwise male group [54,55], although thisis not always the case [20]. Indeed, in later weeks, student 72 goes on to be strongly connected to a new,transient group where she is not the only woman. On the other hand, student 121 shows repeatedly tobe strongly connected to the same group where the other students are male. Unfortunately, due to thepossibility of missing data (students not wearing sensors or students wearing sensors but not consentingto data use) we are not with this data set able to say whether these students definitely are the only womenin their respective groups.Despite the many random factors influencing group composition and dynamics, we can see that manyof the groups appear stable already in the first week and remain that way until the end of the observationperiod. For example, the trio of student IDs 102, 30 and 37 appear close whenever the whole group ispresent (Figure 8) and this can also be seen in their depercolation tree (Figure 6).
Figure 9. The strongly connected groups of sections 2 and 5 during the third laboratory week, withdemographics (major: Physics (Phys), Teaching (Tea), Mathematics (Mat), Chemistry (Chem), or other,Year: 1, 2, higher (N) or other and gender.)
The course grading is based on individual laboratory reports. The learning outcomes in terms ofgrades seem unaffected by whether the student is strongly connected to a group. Also, the subset ofstudents who turned in their work are very similar to students who did not, and even the attendance on thecourse (during the first seven weeks) does not set them apart. This is unexpected, because we cannot see acorrelation between students turning in laboratory work and students attending the laboratory sessions.Neither the number of times the students attended the laboratory sessions nor the times they were stronglyconnected to other students seem to influence whether the student will turn in their work for grading. Apossible reason for the absence of correlation between times present and learning outcomes is that thereare multiple possible reasons for absence. Being away may signal low commitment, but well-performinggroups may complete the assignment more quickly and agree to not attend some sessions.There is a low correlation between mechanics knowledge (MBT) and the grade. The grade does notmeasure mechanics knowledge directly, but a good knowledge of Newtonian mechanics and problemsolving certainly helps in obtaining good and reliable results. Despite this, the correlation coefficient isonly r = Previous research has shown that CLASS scores correlate with retention on introductory physicscourses [28]. We did not observe this in our data, but our data is from a short segment of a course. In termsof social dynamics in the laboratory, the course seems to function well. All TAs play a similar, central rolein the dynamics of their respective sections, and can be used as a treshold to separate the different groups.The number of students who are mostly loosely connected to the network is small, and most studentsattend the course regularly and while there are fluctuations in the composition of the strongly connectedgroups, most groups in each section persist and most students are strongly connected to their group inany given laboratory session. Pre-existing physics knowledge or attitudes towards learning science seemto have little impact on the group dynamics or the learning outcomes. In literature, the effects of groupcomposition on group work vary significantly, but a recent study in introductory physics found that groupcomposition had no effect on learning gains [20]. Clearly, the problems that lead to students dropping thecourse are outside the laboratory sessions.
A network study that combines surveys and sensor data in a authentic setting means that the case ofmissing data has to be accounted for. In our case, the data was collected in the first weeks of the semesterand, indeed for many students, the first weeks of university studies, which means that in the student pool,there are students who only try out physics. Their commitment is low, and they can easily drop courses.Of course, students also have a right to decline to participate in a study. The students were asked not towear a sensor if they did not participate in the study, but nevertheless we had data from students whoeither declined consent or did not return the consent form. Further, not all students who gave consent tookall surveys. Our solution to this was to only include in the analysis the data from the students who gaveinformed consent at each stage. This likely skews the analysis towards a more regular student population:students who complete all assignments and students who take courses in the recommended order (as thesurveys were collected in the concurrent lecture course).In demographic terms, the students who consented to participate in the study are very similar tothe students who start introductory physics at the University of Helsinki. 60% are physics majors and40% female. The majority are first year students and the absolute majority of the students who are not,do not have physics as a major. The sample also includes some non-degree students, who generally areeither students through the open university or teachers, who are working on the qualifications for anothersubject to teach. While the commitment and diligence of the students who participate may be higher thanaverage, the sample does cover all typical kinds of students on our laboratory courses.Another limitation that must be addressed is that in our study, we have used face-to-face time as aproxy for interactions. This naturally induces a possibility of bias, because it is not possible to trace thequality of interactions. However, we believe that in the studied setting, detecting face-to-face contacts isa suitable proxy for contacts. The laboratory assignments are open ended, meaning the students haveto decide on a common strategy (with or without the help of the TA). It is thus difficult to performexperiments without communication, and as attendance in the sections is not compulsory, students donot attend if they have no experiment to do. However, we cannot know the type of interaction betweenstudents, and not all interactions are bound to be beneficial neither for the laboratory assignment nor forsocial inclusion.
5. Conclusions
In this paper, we have shown that the combination of the SocioPatterns platform and supplementarydata provides an opportunity to follow the social patterns of students working in small groups on alaboratory course. By asking students for their background information and by combining it with the data obtained for the network analysis, we can study the group composition and evolution of the groupdynamics.We introduce a procedure in which links are removed from weakest to strongest. The TA is used forcalibration: groups that are connected at the depercolation step which detaches the TA are classified asstrongly connected. Most students are strongly connected most of the times they attend the laboratory, andthe attendance is reasonably high. The advantage of this procedure over conventional network detectionalgorithms is that we are able to identify small groups in a small, tightly connected network. We canidentify typical interaction patterns in the laboratory: stable groups, transient groups, where the proximitychanges over time, and stable separations, where certain students do not interact with each other.Students of different demographics had equal risks of dropping out (measured both by attendance atthe end of the observation period and turning in work) or not being strongly connected to a group. Weobserved that some students were rarely or never strongly connected to a group, but these students didnot share any characteristic or set of characteristics. Clearly, for this course, factors influencing droppingout are more subtle than simple demographics. This means that the social dimension of the course worksequally well for students of e.g. different major subjects. On the other hand, improving the social dynamicsin the laboratory is unlikely to change the drop-out rate.
Funding:
This research received no external funding.
Conflicts of Interest:
The authors declare no conflict of interest.
References
1. Holmes, N.G.; Olsen, J.; Thomas, J.L.; Wieman, C.E. Value added or misattributed? A multi-institution study onthe educational benefit of labs for reinforcing physics content
Phys. Rev. Phys. Educ. Res. , , 010129.2. Royuk B,; Brooks, D.W. Cookbook procedures in MBL physics exercises J. Sci. Educ. Technol. Phys. Today Am. J. Phys. Phys. Rev. Spec.Top.—Phys. Educ. Res. Am. J. Phys. S38–44.8. Beichner, R. J.; Saul, J.M.; Abbott, D.S.; Morse, J.J.; Deardorff, D.; Allain, R.J.; Bonham, S.W.; Dancy, M.H.; Risley,J.S. The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project.
In:
Research-Based Reform of University Physics vol 1 , Publisher: American Association of Physics Teachers,College Park, MD. 2–39.9. Zwickl, B.M.; Finkelstein, N.; Lewandowski, H.J. The process of transforming an advanced lab course: goals,curriculum, and assessments.
Am. J. Phys. Eur. J.Phys. Am. J. Phys. , , 970-977.12. Singh, C. Impact of peer interaction on conceptual test performance. Am. J. Phys. , 446-451.13. Mason, A.J.; Singh, C. Impact of Guided Reflection with Peers on the Development of Effective Problem SolvingStrategies and Physics Learning Phys Teach Am. J. Phys. , ,1066-1069.
15. von Boxtel, C.; van der Linden, J.; Kanselaar, G. Collaborative learning tasks and the elaboration of conceptualknowledge
Learn Instr , , 311-330.16. Schwartz, B.B.; Linchevski, L. The role of task design and argumentation in cognitive development during peerinteraction: The case of proportional reasoning Learn Instr , , 510-531.17. Webb, N.M.; Nemer, K.M.; Zuniga, S. Short Circuits or Superconductors? Effects of Group Composition onHigh-Achieving Students’ Science Assessment Performance. American Educational Research Journal Br. J. Educ. Psychol. , , , 205-221.19. Lou Y.; Abrami, P.C.; Spence, J.C.; Poulsen, C.; Chambers, B.; d’Apollonia, S. Within-Class Grouping: AMeta-Analysis. Rev Educ Res , , , 423-458.20. Harlow, J.J.B.; Harrison, D.M.; Meyertholen, A. Effective student teams for collaborative learning in anintroductory university physics course. Phys. Rev. Phys. Educ. Res. , , 010138.21. Jensen, J.L.; Lawson, A. Effects of Collaborative Group Composition and Inquiry Instruction on Reasoning Gainsand Achievement in Undergraduate Biology. CBE—Life Sciences Education Phys. Rev. ST Phys. Educ. Res. , Phys Teach Phys. Teach. Am. J. Phys. Phys. Rev. ST Phys. Educ. Res. , AIP Conf. Proc. , , 61–64.29. M. J. Cahill, M. A. McDaniel, R. F. Frey, K. M. Hynes,M. Repice, J. Zhao, and R. Trousil, Understanding therelationship between student attitudes and student learning Phys. Rev. ST Phys. Educ. Res. , 010107.30. Ding, L. Verification of causal influences of reasoning skills and epistemology on physics conceptual learning Phys. Rev. ST Phys. Educ. Res. , ; Jones, D.L.; Ding, L.; Traxler, A. Eds.; Publishing House: City, Country, 2016; pp. 144–147.32. Slaughter, K.A.; Bates, S.P.; Galloway R.K. A longitudinal study of the development of attitudes and beliefstowards physics. Physics Education Research Conference 2011 . Omaha, Nebraska:
Phys. Rev. Spec. Top.—Phys. Educ. Res. Phys. Rev. ST Phys. Educ. Res. J. Artif. Soc. Soc. Simul. PLOS ONE e11596.38. Stehlé, J.; Voirin, N.; Barrat, A.; Cattuto, C.; Isella, L.; Pinton, J.-F.; Quaggiotto, M.; Van den Broeck, W.; Régis,C.; Lina, B.; Vanhems, P. High-Resolution Measurements of Face-to-Face Contact Patterns in a Primary School.
PLOS ONE e23176.
39. Barrat, A.; Cattuto, C.; Colizza, V.; Gesualdo, F.; Isella, L.; Pandolfi, E.; Pinton, J.-F.; Rava, L.; Rizzo, C.; Romano,M.; Stehlé, J.; Tozzi, A.E.; Van den Broeck, W. Empirical temporal networks of face-to-face human interactions.
Eur. Phys. J. Special Topics
PLoS ONE e107878.41. Génois, M.; Vestergaard, C. L.; Fournet, J.; Panisson, A.; Bonmarin, I.; Barrat, A. Data on face-to-face contacts inan office building suggest a low-cost vaccination strategy based on community linkers.
Network Science BMC Inf.Dis. PLoS ONE e0198733.44. Girvan, M., and Newman, M. E. J. Community structure in social and biological networks
Proc. Natl. Acad. Sci.U.S.A. ,
99 (12)
Eur. J. Phys Journal ofInformation Science
28 (6)
Connections (Toronto, Ont.)
Physics Reports
486 (3)
IEEE Transactions on Knowledge and Data Engineering
20 (2)
Physics Reports
519 (3)
Eur. Phys. J. B Proceedings of the 2002 SIAM International Conference on Data Mining
Am. J. Phys. , , , 627.55. Heller, P.; Hollabaugh, M. Teaching problem solving through cooperative grouping. Part 2: Designing problemsand structuring groups, Am. J. Phys. , ,60