Practitioner's guide to social network analysis: Examining physics anxiety in an active-learning setting
PPractitioner’s guide to social network analysis: Examining physics anxiety in anactive-learning setting
Remy Dou
1, 2, ∗ and Justyna P. Zwolak † Department of Teaching and Learning, Florida International University, Miami, Florida 33199 STEM Transformation Institute, Florida International University, Miami, Florida 33199 Joint Center for Quantum Information and Computer Science, UMD, College Park, MD 20742, USA (Dated: September 5, 2018)The application of social network analysis (SNA) has recently grown prevalent in science, tech-nology, engineering, and mathematics education research. Research on classroom networks has ledto greater understandings of student persistence in physics majors, changes in their career-relatedbeliefs (e.g., physics interest), and their academic success. In this paper, we aim to provide apractitioner’s guide to carrying out research using SNA, including how to develop data collectioninstruments, setup protocols for gathering data, as well as identify network methodologies relevantto a wide range of research questions beyond what one might find in a typical primer. We illustratethese techniques using student anxiety data from active-learning physics classrooms. We explore therelationship between students’ physics anxiety and the social networks they participate in through-out the course of a semester. We find that students’ with greater numbers of outgoing interactionsare more likely to experience negative anxiety shifts even while we control for pre anxiety, gender,and final course grade. We also explore the evolution of student networks and find that the secondhalf of the semester is a critical period for participating in interactions associated with decreasedphysics anxiety. Our study further supports the benefits of dynamic group formation strategies thatgive students an opportunity to interact with as many peers as possible throughout a semester. Tocomplement our guide to SNA in education research, we also provide a set of tools for letting otherresearchers use this approach in their work – the
SNA toolbox – that can be accessed on GitHub.
I. INTRODUCTION
The principle that information exists within, and be-cause of, human interactions with one another anchorsmany theories of philosophy, sociology, and knowledgedevelopment [1–4]. Even the knowledge that existswithin our scientific enterprises, however objectively weapproach our research questions, has to go through a se-ries of socially constructed hurdles before finding accep-tance in our communities. The peer-review process exem-plifies that. For that reason, social scientists, includingeducation researchers, have began to study the nature ofinteractions between people and how those interactionsfacilitate (or hinder) information flow and development.The way social interactions affect learning experiencescan vary significantly between individuals. For example,some students like discussing their ideas to reaffirm theirknowledge. They may face little difficulty when reach-ing out to others for help or to offer support. As such,they truly thrive in an environment that promotes peer-to-peer and student-faculty interactions. Others dreadsharing their ideas in public, especially when these ideasare still developing. It may be because of a sense of anxi-ety, a feeling of self-consciousness, or shyness. Whateverthe reason, such students might have difficulties appreci-ating active-engagement learning strategies and even getdiscouraged from persisting in a course. Understandinghow and why students build communities, and how these ∗ redou@fiu.edu † [email protected] communities affect their educational well-being is essen-tial to improving their experiences in and beyond theclassroom.One way to approach this problem is to examine stu-dent integration using the tools of social network anal-ysis (SNA). While SNA does not directly capture thecontent of interactions, it allows us to quantify the var-ious aspects of relational structures that result fromthose interactions [5]. The application of SNA has re-cently grown prevalent in science, technology, engineer-ing, and mathematics (STEM) education research. Fromclassroom network dynamics and career persistence toschool-level group belonging and information sharing,network methodology has proven itself useful in help-ing researchers understand factors affecting students suc-cess in STEM [6–8]. However, while there are resourcesfor those interested in the application of SNA, the fewprimers that exist fail to provide enough detail to carryout nuanced education studies and the more in-depthtextbooks lack a classroom framework by which to in-terpret results from such analyses [5, 9–11]. A succinct,higher-level practical guide showing the entire processfrom designing relevant tools to collecting data to apply-ing SNA in educational contexts is (to the best of the au-thors’ knowledge) currently absent from literature. Thiswork is intended to fill in this gap.For over half a decade we have applied SNA in thefield of physics education research. That work has led togreater understandings of student persistence, changesin their career-related beliefs (e.g., physics interest), andtheir academic success [7, 8, 12–14]. In the process,we also established SNA study design in the classroom a r X i v : . [ phy s i c s . e d - ph ] S e p context, including development of data collection instru-ments, setup of protocols for gathering and digitizingdata, as well as identification of network methodologiesrelevant to a wide range of research questions. We alsobuilt a software suite – the SNA toolbox – that allows tocarry the network analysis presented in the following sec-tions. In this paper, we aim to present these approachesand techniques in SNA using the context of student anx-iety, and to discuss how outcomes and interpretationsvary based on methodological and analytical choices. Wefocus on social networks found in classrooms, i.e., net-works representing peer-to-peer and student-instructorinteractions. Our goal is to provide a succinct guide thatremains practical to the education researcher exploringclassroom, departmental, or institution-related interac-tions between people, regardless of the specific questionbeing examined.This is not intended to be a primer. Rather, this paperwill delve into the nuanced aspects of social network anal-ysis, providing guidance along the way that goes beyonda basic explanation of a few centrality measures, and willaddress considerations when collecting data, performinganalyses, and interpreting outcomes. Finally, we focussolely on the context of the physics classroom, using ourresearch of student anxiety in an active-learning settingto illustrate the content. Nevertheless, the applicationsof the SNA topics addressed here, as well as the provided
SNA toolbox code [15], can be used in other physics edu-cation research contexts (and beyond).The paper is organized as follows: After a briefoverview of research on anxiety in the introductoryphysics classroom (Sec. II) and after introducing thephysics anxiety survey (Sec. III A), we proceed to thefirst major section: the “Social network analysis sur-vey” (Sec. III B). This section addresses questions oneshould consider when determining data collection con-text, survey development, administration of surveys, andhandling of multiple collections. In particular, we dis-cuss what constitutes social interactions and how one canmeasure them. We then introduce different types of so-cial networks and present guidance on developing surveysthat yield the network type of interest. We also introducemeasures that can be used to examine weighted networkdata, as well as guidelines for their interpretation withinthe classroom context (e.g., what does it mean for a stu-dent to have high “closeness” centrality, Sec. III C). Fi-nally, we also discuss practical aspects of data collection:the administration of surveys, handling of multiple col-lections, accounting for non-normality of data and han-dling missing data (Sec. III D). The statistical analysistechniques that we use are presented in Sec. III E. Thesecond major section, “Analysis and results” (Sec. IV),shows practical applications of the proposed methodolo-gies in the context of students’ physics anxiety in active-learning introductory physics courses. We conclude witha discussion of our findings, limitations of this work, andrecommendations for future directions in Sec. VI.To make the discussed methodologies more user- friendly, we established a GitHub repository where wemake available the R source-code together with a man-ual and a simple reproducible example that can be eas-ily adapted and used to carry out SNA analyses (opensource, available at GitHub [15]). While presently the
SNA toolbox includes only code used in the analysis fromthis manuscript, it will be continuously maintained andextended further based on the needs of and requests fromthe science education community.
II. ANXIETY IN THE INTRODUCTORYPHYSICS CLASSROOM
To explore the relationship between physics anxietyand in-class student interactions in an active-learningsetting, we adopt a participationist framework. Par-ticipationists primarily view learning as “the develop-ment of ways in which an individual participates in well-established communal activities” [16]. Learning is per-ceived as a construction of mutual understandings withina social context, with emphasis placed on examiningdiscourse and interactions rather than “acquisition” ofknowledge as a commodity or object [17]. As such, weespouse the philosophy that “learning and social interac-tions are not mutually exclusive” [12].Our motivation to focus on physics anxiety is predi-cated on our belief that anxiety shapes how and to whatextent students participate in classroom activities. Ifphysics anxiety hinders participation, our framework sug-gests learning, too, may suffer. Prior work in the realmof social anxiety – not physics anxiety, per se – has foundnegative correlations with participation in activities thatmay be present in active-learning settings. For exam-ple, it has been suggested that social anxiety leads torisk averse behaviors as individuals seek to preserve howothers perceive their image or identity [18]. Such be-haviors can lead to reticence or complete unwillingnessto present before an audience, particularly in settingsframed around the evaluation of content being shared.Active-learning curricula often nurture these kinds of set-tings, where students publicly present results to one an-other. Even when public presentations are not directlyrelated to evaluation, the perception of being evaluatedcan have an impact on behavior [19]. Hills calls out con-structivist teaching styles in particular [20]. In his studyof pre-service math and science teachers, he found thatthose with high social anxiety tended to exhibit risk aver-sion behavior, which manifested in the classroom as lowgroup participation and avoidance of open-ended mathproblems. Even the productivity of group-brainstorminghas been shown to be negatively affected by the level ofgroups members’ social anxiety [21].The correlation between various types of anxiety andphysics learning at the undergraduate level have beendocumented by several researchers. Williams found thatstudents who reported feeling anxious about communi-cating in class, even in non-group, whole-class settings(e.g., when an instructor poses a question to the class)were less likely to score well on multiple-choice examsand less likely to exhibit large gains on the Force Con-cept Inventory (FCI) [22, 23]. Engineering students’math anxiety while learning electricity and magnetismhas been shown to be negatively correlated with courseexam scores, as well as conceptual understanding [24].The idea that, in addition to communication and mathanxiety, physics anxiety should be considered as a uniqueconstruct that affects physics learning is over thirty yearsold and has been associated with studies related to gen-der differences in physics learning [25]. More recently,Sahin [26] explored the physics anxiety of pre-serviceteachers pursuing careers in science, math, and primaryeducation (e.g., physics education, secondary math edu-cation) who were at the time enrolled in an introductoryphysics course. Outcomes of this study showed that thosein the physics education program exhibited less anxietythan those in any of the other programs, but found thatsignificant gender differences existed for physics-focusedmajors, such that female pre-service physics teacherswere more likely to exhibit higher physics anxiety thantheir male counterparts. The study also found that stu-dents with high physics anxiety tended to have earnedeither low (i.e., <
2) or high (i.e., >
3) GPA, which theauthor admits runs contrary to related literature thatessentially admonishes for a linear, indirect relationshipbetween anxiety and performance.The relationship between anxiety, participation, andstudent outcomes motivated our exploration of the po-tential social mechanisms through which it manifests inan active-learning, student-centered classroom. As de-scribed earlier, past research identifies participation inacademic activities as a factor of student anxiety. Thus,we expect students’ physics anxiety to have a negativeeffect on their participation. We also take into accountpast research identifying social support as a mitigatorof anxiety [27]. We thus expect to find a relationshipbetween changes in anxiety and students’ social embed-dedness within the classroom network, such that studentswho seek out relationships with their peers will be morelikely to feel less anxious about physics over time. Wealso hypothesize that the frequency with which studentscarry out repeated interactions with the same individualsexhibits a weaker relationship with anxiety than the num-ber of unique individuals a student interacts with (i.e.,having a greater number of people to provide possiblesupport). Additionally, our analyses take into accountstudents’ self-reported gender and final course grade.
III. METHODOLOGY
In this section, we present the Physics Anxiety RatingScale (PARS) [28] and the social network survey we useto collect data for this study. We also discuss some of theconsiderations we took into account when designing ourexamination of physics anxiety through a social network lens. For completeness, we include the “Social NetworkAnalysis Toolbox” section that presents network mea-sures we rely on when comparing data between differentgroups and sections. While not exhaustive, this list isintended to give flavor for what kind of information canbe extracted and quantified using SNA.
A. Physics anxiety survey
To measure students’ physics anxiety, we use a 16-itemversion of the PARS developed by Sahin [28]. The PARSasks students to rate their agreement with a variety ofstatements on a 5-item Likert scale ranging from stronglydisagree (1) to strongly agree (5). The statements includethe following: “I would feel very embarrassed if the in-structor corrected the answer that I gave to a physicsquestion in front of the class”, “being unable to use unitsof quantities appropriately in physics courses makes mevery anxious”, and “when solving a physics problem, Iworry about not being able to recall relevant formulas orphysics laws”. The survey data is typically collected in pre and post format, using the same instrument at thebeginning and end of the semester, and allows to capturechanges in anxiety. The Cronbach’s alpha reliability co-efficient is 0.92 on the scale using pre data and 0.94 using post data.Since all students coming to the class are expected tohave some level of anxiety [29], which typically variesacross individuals, we are interested in the anxiety shiftrather than the raw anxiety score. As the semester goeson and students experience the curriculum, we expectto see an increase or decrease in their anxiety score, de-pending on their learning experiences. We avoid ascrib-ing value to initial student anxiety (e.g., high anxiety isbad, low anxiety is good) since such practices can conflictwith past research indicating that certain levels of anxi-ety positively correlate with quality performance [30].To provide a measure of the anxiety shift over time, wecalculate the normalized anxiety shift defined as the ratioof the absolute shift to the maximum possible shift [31]: s norm = post − premax. possible. score − pre , (1)where pre and post denote the score of a student on theanxiety survey before and after the course, respectively.This approach allows a comparison of shifts between stu-dents with varying pre scores. The max. possible. score on this survey is 80 and the lowest possible score is 16.Note that, as this measure was developed to assess theexpected average gains on the FCI (i.e., positive shiftsaveraged over the entire class), it is not robust againstdramatic drops in scores of individuals. In particular,for the PARS survey the s norm will be outside of the[ − ,
1] boundaries when the pre score is over 48 and the post score is lower than 2( pre −
40) (see Appendix A formore details). As such, the s norm can be used to iden-tify potential “outliers” – a medium to high pre-coursescore followed by an unexpectedly low post anxiety mayidentify students who did not offer reliable responses onthe post -survey. After careful considerations it might beadvisable to either remove the unusually low post scoresand impute the missing data or to remove such individ-uals from analyses all together. B. Social network analysis survey
Identifying a relevant theoretical framework prior todesigning social network research provides boundariesand guidance for the measurement instrument (e.g., sur-vey design), analysis (e.g., correlational study), and inter-pretation. Here we discuss the design of the SNA surveythat we use to gauge classroom participation. Like anyother research tool, SNA should be applied only when thecontext of a study makes it an appropriate tool. In whatfollows, we discuss when SNA is the right method of anal-ysis, what constitutes social interactions, and how onecan measure them. We also discuss the practical aspectsof data collection, including administration of surveys(e.g., on-line vs. in-person, one-time vs. longitudinalcollection) together with brief analyses of pros and consfor each approach. The main purpose of this section is topresent guidance on developing surveys that yield accu-rate and relevant networks. To illustrate the process weexplain how our study meets the requirements. Duringthe design of the SNA survey and its administration, wecarefully considered our responses to all questions postedbelow. Q : Is SNA an appropriate tool to help answer myquestion?
To use SNA, the research question(s) have to be relatedto interactions of some kind, be it students working ingroups, e-mail exchanges, participation in a forum, orco-authoring a paper, to name a few. In our study, wewant to explore how engagement in a student-centeredphysics classroom contributes to anxiety shifts while alsotaking pre-course anxiety into account. Our focus onstudent-student interactions lends itself to quantificationvia SNA. Q : What interactions am I interested in?
Although the context of a study helps to establish howinteractions should be defined (e.g., conversations, jointpapers, participating in the same meetings, etc.), oneneeds to decide early on what additional characteristicsof interest to incorporate. For instance, is it importantto know who initiated a given interaction (i.e., directed
FIG. 1. Visualization of various types of networks: (a) binaryand undirected, (b) weighted and undirected, (c) binary anddirected, (d) weighted and directed. vs. undirected networks, see Fig. 1(a) and (c))? Is it im-portant to know how frequently a given interaction oc-curred or does it only matter whether it took place (i.e.,weighted vs. binary networks, see Fig. 1(a) and (b))?Whose perspective matters – the initiator’s or the re-ceiver’s? Should all members of a particular group beincluded in the network?For our study, we define “interaction” as a meaning-ful (from a respondent’s perspective) in-class interactionrelated to physics. This may include, among other be-haviors, a discussion of ideas, joint work on a problem,as well as listening to others solve or discuss problems.We also want to know the frequency of interactions be-tween the same two individuals in a given week. Thus,we opt to collect directed network data that captures thefrequency with which the interactions take place withina given collection period (see Fig. 1(d) for a visualiza-tion of this type of network). This grants us flexibilityduring the analyses to calculate centrality measures thatplace more or less emphasis on both directionality andfrequency. Similarly, we invite students to provide infor-mation about their interactions with professors, knowingthat we can later remove those interactions if we decideto focus solely on the peer network. In particular, stu-dents are asked to “...choose from the presented list peo-ple from [their] physics class that [they] had a meaningfulinteraction with in class ... even if [they] were not themain person speaking or contributing” (see Tools in the
SNA toolbox for an example of the SNA survey [15] forthe complete survey). Students are directed to considerall interactions that took place during the week prior tocompleting the survey, including interactions with peersoutside of their small groups. As mentioned earlier, theyare not given written parameters for what counted as a“meaningful” interaction, but, when asked, we encouragethem to think about interactions related to course-relatedactivities and content. To aid recall of their peers’ names,we provide them with a randomized roster of all individu-als enrolled in class, together with names of the teachingstaff. Q : How can I collect network data?
There are multiple ways one can collect social networkdata: videotaping the course, administering a pen-and-paper survey in class, asking students to complete anon-line survey (either in class or at home), using a course-related forum to track students’ interaction, etc. Each ofthese approaches has its own set of pros and cons. Withvideos one has access to the entire course, which providesa very rich data set. It allows for a fine grain analysisof, e.g., the network evolution in real-time. However, theextraction of networks from videos can be challenging.From establishing a reliable coding dictionary that min-imizes coder bias, to determining the most informativetime stamp for “slicing” the data, to coding what couldbe hours of videos, this approach requires a lot of timeand effort [32].Pen-and-paper surveys take significantly less time,most of which is spent on establishing a protocol for digi-tizing the responses and converting them into a network.Once established, such protocol can be utilized on con-secutive collections. Nevertheless, pen-and-paper surveysrequire time to develop and place an extra cognitive loadon individuals completing them. Moreover, such surveyscan be biased and not fully representative of what washappening in class, especially early on when respondentsdo not know the names of all other participants and re-lationships are not yet well formed.The same applies to online surveys, though in this caseconverting responses into network data can be handledwith a simple script. When administered outside of classtime, online surveys tend to suffer from lower responserates. E-mail exchange or forum-based networks offer thesame advantages in terms of converting responses intonetwork data with the use of a script. However, as withvideo data, one has to carefully decide what constitutesan interaction, which is not always straightforward (e.g.,handling “nested” posts on a forum). Such networks canalso suffer from lower response rates, particularly becauseof missing data from students who read posts or e-mailsbut do not respond to them [33]. Another thing to con-sider is whether the participants should receive any in-centives for taking part in the study (e.g., course credit,gift cards).Since a pilot study with both pen-and-paper andcomputer-assisted versions of the survey revealed thatthe online approach tends to be more time consumingand more confusing to students, we decided to collectdata using the pen-and-paper format. To maximize theresponse rates we collect data in the classroom, at theend of a particular class. Our participants do not re-ceive any direct benefit from completing the survey (e.g.,extra points, reduced workload). Moreover, during the administration students are invited to inquire about thepurpose and outcomes of the study by contacting eitherthe professor or the survey administrators. Q : How often should I collect network data?
Another thing to consider is how often one intends tocollect the data and when is the best time for collection.The number of collections should be guided by the re-search question, collection method, as well as previousresearch. How much extra burden one is willing to puton students and, for in-class collections, how much classtime one is willing to spend on administering surveys alsoneeds to be taken into account.In our case, we want to look at students’ social em-beddedness within the in-class network as a predictor foranxiety shift over time, so it is appropriate to collectnetwork data at least at the beginning and end of thesemester. To capture a more granular picture of networkevolution, given student-group rotation and other curric-ular features, we added three additional administrationsthroughout the semester, spaced every 3-4 weeks. Wechose five collections to allow enough time for the net-work to change between collections. During each surveyadministration students are reminded that their partic-ipation is strictly voluntary. Anecdotal data from pastresearch using similar, in-class surveys suggests that morethan five collections may cause survey fatigue. Q : How should I work with longitudinal data?
Collecting data multiple times throughout thesemester gives one flexibility when preparing data foranalysis. Longitudinal data allows for the study ofnetwork evolution. Treating each collection as a sep-arate data set enables one to observe changes in thenetwork as time goes by. For instance, comparison ofpre- and post-course data from lecture-based and active-engagement classrooms reveals that only in the lattercase the in-class network becomes connected, while theformer doesn’t show any development after a semesterof instruction [34]. Analyses of in-class networks fromactive-learning introductory physics courses show thatnetworks gradually evolve throughout the semester, sug-gesting that such environments are in fact conducive toestablishing a relationship network of academic and emo-tional support [7, 14]. However, longitudinal approachesare more sensitive to missingness, as it is quite likely thatdifferent individuals may be physically absent during dif-ferent survey administrations.Aggregating multiple collections into one network rep-resenting the entire semester helps with missingness, asit is reasonable to assume that each student should bein class during at least one collection. Since the surveydistribution schedule was not announced at any point,it seems unlikely that students could intentionally try toavoid classes when data is collected. At the same time,if a given student is absent across multiple survey ad-ministrations, it might signal that the individual is skip-ping more classes and thus is not getting immersed inthe social environment. Treating such an individual asdisconnected from a classroom network might thus bethe appropriate thing to do. However, aggregation lim-its the amount of information contributing to a completeunderstanding of the network’s evolution [8, 12].For weighted, directional data there are a multitude ofways network data be aggregated. This can range fromsimply combining all collections, with weights in the finalnetwork calculated as a sum of weights across all collec-tions, to more nuanced computations involving weightedaveraging between collections. Alternatively one mightsimply assign weights based on either the presence or ab-sence of an interaction on a particular collection. The de-cision of whether to aggregate (and how to proceed withaggregating) should be guided by the research question,previous studies on the population being examined and,if possible, rooted in a theoretical framework.Since we ask students to report meaningful interactionsthat took place during a defined period of time (the weekprior to each data collection) and we do so five times dur-ing the semester, aggregating all data into one networkwill result in the loss of information about which interac-tions happen due to convenience (i.e., sitting at the sametable) and which survive the test of time (i.e., recurrentinteractions regardless of group membership). Thus, inour analyses we treat each collection as a separate net-work. This allows us to capture the effect of modifica-tions to the seating arrangements and the group exam onthe evolution of the network throughout the semester. Q : How can I quantify social interactions?
Some of the remaining considerations include how toconvert interaction data into a network and then how toanalyze the resulting network. As mentioned when dis-cussing the different tools for collecting SNA data (Q3),the protocol for converting data into a social networkwill depend on the particular data collection approach.When digitizing data, one should retain the capability offormatting identified interactions as interaction matricesor lists of the pairs involved in an interaction (i.e., edgelists). Once a matrix or an edge list is created, SNAprovides a very rich toolbox for analysis. From variousnetwork topology measures to a multitude of centralities,there is plenty to choose from. In general, one can exam-ine the interactions in a network from one of two broadperspectives: whole network connectedness (i.e., networktopology) and individual node-level measures (i.e., cen-tralities).To digitize our pen-and-paper surveys into networks,we developed a spreadsheet with built-in self-checks inorder to minimize coding errors. The spreadsheet isavailable as part of the
SNA toolbox [15]. As mentioned earlier, we opted to keep each collection as a separatenetwork. To examine students’ interactions, we calcu-late three centrality measures discussed in Sec. III C. Ourchoice of these particular indices is guided by their abilityto capture the kind of immersion within the network thatwe hypothesize to be relevant for anxiety shifts – over-all embeddedness in the case of closeness and individual-level connectedness in the case of indegree and outdegree.This approach is also supported by previous research thatfound these measures to be informative when studyingperformance [14] and persistence [7, 8], both of whichare related to anxiety.
C. Social Network Analysis toolbox
There are two basic types of static network measures:the network-level measures that describe characteristicsof the network as a whole and the node-level measuresthat focus on characterizing the relational position of aparticular node quantitatively. In what follows, we usethe term “node” in reference to the individuals that makeup a social network (note that social sciences often usethe term “actor” instead) and “edge” (also called “tie”or “link”) when referring to the interaction between twonodes. The following section gives a brief overview of themost commonly used tools for quantifying interactionsfrom an SNA perspective. All metrics discussed beloware implemented in the
SNA toolbox [15].When choosing to combine data across multiple groups(e.g., multiple sections of the same course), it is impor-tant to verify that the networks are similar enough to jus-tify aggregation. Network topology offers understandingof how nodes are connected with one-another on a globallevel. This includes characteristics like network size, den-sity, and distances between nodes. For example, density (∆) offers insight about the overall cohesion of a networkand is expressed as the fraction of existing edges betweennodes to the number of all possible edges:∆ = number of present edgesnumber of all possible edges .
The number of all possible edges between n nodes is ex-pressed as n ( n − / n ( n − Diame-ter ( D ) gives a network’s longest path – where path isdefined as the number of edges in the sequence of edgesconnecting two nodes in a network – and captures thespan of a network. Average path length ( L ), on the otherhand, gives the average shortest path between all pos-sible pairs of nodes. It provides information about howclose (on average) nodes are to one another [35].The global clustering coefficient ( transitivity , Tr) cap-tures the degree to which nodes tend to cluster together.It is based on the notion of open and closed tripletsin a network, where a triplet is defined as three nodesconnected by either two (open triplet) or three (closedtriplet) undirected edges [35]. Transitivity is defined asa fraction of closed triplets of all triplets (opened andclosed) in the network:Tr = number of closed tripletsnumber of all triplets , Since by definition transitivity is calculated for undi-rected and unweighted networks, networks that are morecomplex in nature have to be flattened prior to anal-ysis. This, in return, allows one to vary the strengthof transitivity. For instance, requiring that all edges intriplets are bidirectional will lead to a stronger globalclustering coefficient than the simple presence or absenceof edges. Similarly, requiring that all edges in a tripletare of weight at least w , where w ≥
1, will result instronger transitivity the larger w is. Recently, a gener-alization of the global clustering coefficient that includesweight was proposed [36]. Since we use transitivity onlyto establish similarity between our networks and do notuse it in analysis, we find the basic, binary version to besufficient.Finally, reciprocity captures how frequently interac-tions are mutual. It is calculated as a fraction of allthe interactions that are bidirectional [35]: ρ ↔ = number of bidirectional edgesnumber of all present edges . Once similarity of networks between groups is con-firmed, one can proceed to quantifying the position ofeach node within the network. This is most commonlydone by calculating centrality measures. There are amyriad of such measures, from localized, i.e., focused ona particular node and its direct connections (see, e.g.,Fig. 2(a) and (b)), to global measures that take into ac-count the entire network (see, e.g., Fig. 2(d) and (e)).The choice of a particular measure depends of the con-text of the study. There are various textbooks thatgive a good introductory [37] and more advanced [5, 35]overview of centrality measures, as well as primers thatexplain their use in different contexts (see, e.g., Ref. [9]for primer in education research). Here, we only brieflydescribe measures that we use in our analysis.Building on our previous work [7, 8, 12–14], we cal-culate the following three measures: indegree, outdegreeand closeness. Put simply indegree can be thought ofas a measure of popularity. It is calculated as the num-ber of edges directed towards a given node.
Outdegree –the number of edges that a given node sends to others– can be interpreted as sociability or influence. Finally, closeness captures how well a given node is embeddedwithin the entire network – the “closer” a given node isto everyone else in the network, the more access that per-son might have to resources (e.g., knowledge, educational
FIG. 2. Visualization of various types of centralities. In eachcase, X has higher centrality than Y according to (a) inde-gree, (b) outdegree, (c) eigenvector, (d) closeness, and (e)betweenness. Adapted with permission from Ref. [7]. or emotional support, information about study groups).Here we use the weighted generalization of these mea-sures that accounts for both the edges’ weights and theirnumber [38], with the parameter α tuning the relativeimportance of these two factors. Formally, for degree[ C αD ]( i ) = ( i (cid:48) s binary degree ) (cid:20) i (cid:48) s strengthi (cid:48) s binary degree (cid:21) α , (2)where α ∈ [0 , ∞ ) is the tuning parameter, the node’sbinary degree is the number of incoming edges for inde-gree and outgoing edges for outdegree, and the node’sstrength is a sum of weights of incoming edges for inde-gree and outgoing edges for outdegree. If α = 0, then C αD gives the binary degree and if α = 1, then C αD re-turns the overall sum of all weights (i.e., strength). When α ∈ (0 , α >
1, it is favorable to have a few strong connec-tions (for the same total strength).For closeness, C αC ( i ) = (cid:104) sum of weighted shortestpaths to all other nodes (cid:105) − , (3)where the weighted path linking i and j is defined as d αij =min (cid:0) w − αim + · · · + w − αnj (cid:1) . Like with degree, for α = 0, thebinary version of closeness results (i.e., the weights areignored), while for α = 1 only the weights are important.If α ∈ (0 , α > alpha coefficient. D. Other considerations
1. Accounting for non-normality
Given the interdependence of network data, its distri-bution often fails tests of normality. For example, whenstudent A reports one outgoing interaction with peer B,by definition a researcher records an incoming interactionfor peer B. Because one student’s responses can affect an-other student’s responses, interaction data often violatesthe assumption of independence required by typical sta-tistical analyses. Moreover, centrality measures are notalways normally distributed, which violates requirementsof linear regression models.To account for these violations, we use linear regres-sion permutation tests [39]. Linear regression permuta-tion tests use a type of Monte Carlo method to randomlysample a data set, rearranging the values of its variablesacross all observations. A linear model is tested on thisre-sampled data set, which generates a set of regressionestimates. The regression estimates of the original dataset are then compared to the distribution of estimatesgenerated from the permuted sets in order to determinethe reliability of the outcomes. In addition to not re-quiring data to be normally distributed, this kind of testhelps to minimize the false positive finding (i.e., type Ierror).
2. Handling missing network data
Regardless of whether the data collection takes placein or outside of the classroom, through pen-and-paper oron-line surveys, it is quite unlikely that any given col-lection will solicit a 100 % response rate. Students maynot show up to class on a day when data is collected,they might leave early, or may choose not to completethe questionnaire. In any case, response rates shouldbe considered when choosing an approach for handlingmissing data. To do so, one must first define the networkboundaries.Classroom networks can be defined by one of two typ-ical boundaries: (A) students officially enrolled in theclass or (B) students who choose to share network data.The former treats all enrolled students as members ofa network on each collection, with absentees and non-respondents contributing to the overall “missingness” ofthe network. The latter boundary posits classroom par-ticipation (e.g., attendance on the day of data collec-tion) as a qualifier for inclusion in the network. Bothapproaches have pros and cons. Boundary “A” is mostinclusive, taking into account the behavior of all studentsenrolled in a course, regardless of their attendance or par-ticipation throughout the semester. Research questionsthat aim to understand broad ranges of social behaviorslend themselves to this approach. On the other hand,researchers interested in specific-types of behavior (e.g.,peer-peer interactions) may want to take the second ap- proach and limit the network boundary to those present,given that a student’s absence does not necessarily reflecttheir in-class social behavior. Regardless of the approachmissingness will almost always be present.The challenges that result from missingness in a net-work stem from the inherent interdependence of networkdata. A student’s behavior in a network not only af-fects their position in the network, but also the positionof others in the network regardless of whether or notthe student in question directly interacts with everyonein the network. Typical methods for handling missingdata, such as imputation techniques, do not take intoaccount data interdependency; while they may predict agiven individual’s centrality scores, they fail to accountfor how that would affect the scores of all others in thenetwork. Replacing missing data with substitute valuesincreases the chances of significantly changing the proper-ties of the network. On the other hand, it has been shownthat centrality scores are fairly robust to random miss-ingness. For example, for small networks (40 – 75 nodes)the level of missing data that does not affect the overallstructure is up to 35% for directed degrees and about20% for closeness centrality [40]. The missingness in ournetwork data falls within these thresholds and thereforeno imputation was used. However, if the missingness fallsoutside of those thresholds, it may not be possible to do awhole-network analysis. One can still try to examine ego-networks, i.e., build networks based on all data availablebut look only at individuals who responded to the survey.Such initial analysis can be then complemented by, e.g.,interviews or data from registrars. In either case, cau-tion should be taken when drawing conclusions in lightof what data is available.
E. Statistical analysis
The dependent variable in our study is continuous (thenormalized shift in anxiety). To investigate relationshipsbetween students’ pre-course anxiety, network centrali-ties, gender, final grade and their shift in physics anxiety,linear regression modeling is used. To control for con-founding factors, we perform multiple linear regression.Only significant variables for the simple linear regressionanalysis are incorporated into the full model.In the first stage, we want to determine which centrali-ties carry significant information about the anxiety shift.To do so, we run simple linear regression models witha single centrality as a predictor (i.e., anxiety.shif t ∼ centrality ). To explore the relative effect of the num-ber of edges and their weights, we test four values of thetuning coefficient: α = 0 . α = 0 . α = 1 . α = 1 .
25 (it is better to have less edges, keep-ing strength fixed), see Sec. III C for details.In the second phase, we want to take advantage of thelongitudinal nature of our data. Having identified thestatistically significant centralities from the last surveyadministration, i.e., our fifth collection, we investigatewhich of those measures remain significant on earlier ad-ministrations. To do so, we test simple linear regres-sion models for all earlier collections, i.e., collections onethrough four. We then compare the fits of the modelsto determine the relative importance of the number andweights of edges and identify the most useful tuning pa-rameter α value for our purposes.Finally, after identifying the earliest informative col-lection and α value, we move to testing full linear models(i.e., anxiety.shif t ∼ centrality + gender + f inal.grade + pre.anxiety ). The variance inflation factor for the finalmodel, ranging from 1.0 to 1.1, indicates no collinearitybetween variables.To account for the fairly large number of tested mod-els, we run each test as a permutation test. As previouslydescribed, permutation test randomizes the matching be-tween independent and dependent variables and com-pares the true regression estimates to the distribution ofestimates calculated across a certain number of iterationsof randomization. In our study, we use 5000 iterations.Again, the use of permutation tests helps to address twoconcerns that arise when dealing with network data: (1)missing data and (2) violation of the assumptions of nor-mality and homoscedasticity (i.e., same finite variancefor all random variables in the sequence).For the statistical analyses, we use the R statisticalprogramming language [41]. In particular, we use lm-Perm [42, 43] package for the permutation test for linearmodels, the Amelia [44] package for imputation of anxi-ety data, and the igraph [45] and tnet [46] packages fornetwork analysis. The chi-squared test and Fisher’s exacttest are used to test for statistically significant differencesbetween classroom sections in terms of gender and ethnic-ity. The one-way analysis of variance (ANOVA) is usedto compare the two section in terms of students’ GPAand paired t-test is used to compare the anxiety scoresbetween sections. The Kolmogorov-Smirnov test is usedto compare the original and imputed PARS scores, andShapiro-Wilk test is used to test for normality of the cen-trality scores’ distributions. To adjust the false discoveryrate the Benjamini-Hochberg procedure is implemented[47]. We consider results with p < .
05 as significant. Allprotocols in the project were approved by the Florida In-ternational University Institutional Review Board (IRB-13-0240 exempt, category 2).
IV. ANALYSIS & RESULTS
This section describes practical applications of the pro-posed methodologies in the context of students’ physicsanxiety in introductory physics courses. We set out tounderstand whether students’ social interactions and po-sitioning in the classroom network is predictive of theirshift in anxiety while controlling for their pre-course anx-
TABLE I. Students’ gender and ethnicity distribution. Thenumbers represent the percentage of students is given group.Section A Section BGender: Female 41.5 50.7EthnicityAsian 15.1 2.7Black 11.3 12.3Hispanic 60.4 72.6White 3.8 8.2Other/NA 9.4 4.1 iety, self-reported gender and final course grade. We alsowant to understand when during the semester, if at all,does social integration begin to matter with regard toshifts in anxiety.
A. Demographics
The data for this study was collected at a large re-search university, designated as a Hispanic-Serving In-stitution. In particular, we survey students enrolled inthe Introductory Physics I with Calculus course taughtusing the Modeling Instruction (MI) curriculum. Dueto its inquiry-laden, discourse-based approach, MI pro-vides an ideal context for studying the range of possiblestudent-student interactions in an introductory physicsclassroom [48, 49]. The course combines lab and lec-ture components of Physics I, engaging students withhands-on, group activities in which they develop modelsof physical phenomena through the use of various repre-sentations (e.g., equations, graphs, diagrams or a com-bination thereof) [50]. Students work in small groups ofthree, with two small groups typically sharing a table, inorder to develop representations relevant to the problemat hand. Then students come together in larger groupsof about 25 to 30 to discuss the small group findings.Instructors, teaching assistants, and learning assistantsfacilitate both large and small group discussions. Tradi-tional lecture rarely occurs during the semester. Instead,students participate in a flexible classroom space de-signed for active-learning. Chairs and tables are movableand students are provided with portable white boards.They are permitted to communicate with peers in othergroups and often do so. Small group membership is ran-domly selected and changes several times throughout thesemester.The data for this analysis comes from two MI sec-tions offered in fall 2016 ( N F A = 53, N F B = 73).There were two instructors teaching the course, bothwith several years of experience teaching introductoryphysics using student-centred curricula, including MI.Student demographic data was queried from a universitydatabase and includes self-reported gender (binary: fe-0 (a) (b)FIG. 3. Comparison of the original and imputed data for (a) the anxiety pre score and (b) the anxiety post score. male or male), incoming GPA, and final course grade, seeTable I for details. We find no statistically significant dif-ferences between sections in terms of gender (chi-squaredtest, χ (1) = 0 . p = 0 .
40) and ethnicity (Fisher’s exacttest, p = 0 .
06) distributions. There is also no significantdifference in mean incoming GPA between groups (one-way ANOVA, F (1 , . p = 0 .
16, note that theGPA for one student was not available).
B. Analysis of physics anxiety
Students’ total scores on the PARS were generated byadding up the sum of their scores on the individual itemson the survey. Paired samples t -tests showed no signif-icant difference between the mean pre and post physicsanxiety total scores, regardless of instructor ( t = 0 . p = 0 .
46 for Section A, t = − . p = 0 .
09 for SectionB) , nor when combining instructor data ( t = − . p = 0 . pre survey, 21 for post survey, and addi-tional 7 for both. To account for the missing data, weran a single imputation. Figures 3(a) and 3(b) show thecomparison of distribution for the pre and post scores fororiginal (blue) and imputed (purple) data, respectively.The two sample Kolmogorov-Smirnov test showed no sta-tistically significant differences in the distributions, with p = 1 for both pre and post scores.With the imputed data, the average anxiety score atthe beginning of the semester for instructor A’s sectionwas M preA = 38 . SD preA = 11 .
3) andfor instructor B’s section – M preB = 37 . SD preB = 12 . M postA = 41 . SD postA = 13 . M postB = 36 . SD postB = 13 . t -test, t = 0 . p = 0 .
38) wecombined the data from their courses ( N =126).The range of the imputed PARS scores went from(16 ,
63) at beginning of the semester to (16 ,
79) at theend (for the non-imputed post scores it is (16 , pre to post responses. Qual- itative analysis of histograms reveals slight right skewingwhen comparing the scores from pre to post , indicatingthat while the overall mean did not change significantlyacross the semester, individual students’ anxiety did ex-perience some shifts, see Fig. 4(a). For the followinganalyses, we use individual students’ normalized shift inanxiety in order to take into account their maximum pos-sible shift, see Fig. 4(b) for the shift’s distribution. (a)(b)FIG. 4. (a) The distribution of the imputed pre (yellow) and post (green) anxiety scores showing slight right skewing. (b)The normalized shift at individual level as defined in Eq. (1). (a) First collection (b) Fourth collection (c) Fifth collectionFIG. 5. The in-class network evolution for individual networks in Section A ( N = 53), with the size representing the outdegreefrom fourth collection ( α = 0 .
0) and the color indicating the direction of the shift in anxiety scores (magenta – positive, blue –negative, green – no shift. The networks include only students.
C. Analysis of student networks
As mentioned in Sec. III C, when analyzing networkdata from multiple groups, it is important to verify thatthere is foundation for aggregating the data. The re-sponse rates to the survey were fairly comparable be-tween sections: M A = 80 . SD A = 6 .
8) and M B = 79 . SD B = 11 . χ (1) = 0 . p = 0 . TABLE II. The comparison of network characteristics for first(SNA1), fourth (SNA4), and last (SNA5) collection for fall2016 (two sections, A and B): network size ( n ), density (∆),average path length ( L ), diameter ( D ), transitivity ( T r ) andreciprocity ( ρ ↔ ). Note that instructional staff is removedfrom the network. n ∆ D L T r ρ ↔ SNA1 A 53 0 .
10 7 3 . .
46 0 . .
09 8 3 . .
39 0 . .
11 5 2 . .
25 0 . .
07 7 2 . .
29 0 . .
13 6 2 . .
33 0 . .
09 6 2 . .
34 0 . D. Predicting shifts in anxiety
Depending on the number of data collections that bestfits a study, as well as the number of constructs beingexplored, the number of variables that need to be con-sidered can become enormously large and the number ofstatistical tests to run can reach values that make falsepositive findings more likely. Eliminating irrelevant vari-ables helps to ameliorate some of these concerns. Withthe abundance of various centrality measures, each hav-ing its own advantages and disadvantages, and usuallyquite different interpretations, it might seem appealing totry as many as possible and “see what works”. However,as we stressed earlier, the choice of particular metricsshould be made in light of previous research wheneverpossible.In our case, prior studies indicate that students’ net-works in an active-learning classroom evolve over time,and that in the case of persistence and academic perfor-mance in physics, social networks established by abouthalf way through the semester become more informa-tive [8, 14]. With regard to physics anxiety, however,we found no study that explores it in the context of stu-dents’ classroom network evolution. For this reason wechoose to begin our exploration with student networks atthe end of the semester, i.e., from the fifth SNA surveyadministration. At this point in the semester studentshave had ample opportunity to interact with nearly all oftheir classmates, either in small groups, board meetings,or one-on-one. Moreover, multiple rotations of seatingassignments facilitated and encouraged more extensiveinteracting through, e.g., team work, labs, and other as-signments with different groups of students. Therefore,students had the greatest amount of information withwhich to evaluate the level and quality of their interac-tion with classmates within and outside of their smallgroups.2
TABLE III. Summary of the linear regression for anxiety shift as predicted by weighed outdegree from fourth and fifth collection,with α ∈ { . , . , . , . } : the unstandardized estimate (B), the standard error for the unstandardized estimate (SE B),standardized estimate ( β ), t -test statistic ( t ), and R-squared ( R ). We consider networks without instructional staff. Significant p -values are marked with an asterisk.Centrality Fourth collection Fifth collectionB SE B β t R B SE B β t R C . outD − . ∗∗ . − . ∗∗ − .
28 0 . − . ∗ . − . ∗ − .
51 0 . C . outD − . ∗∗ . − . ∗∗ − .
28 0 . − . ∗∗ . − . ∗∗ − .
84 0 . C . outD − . ∗∗ . − . ∗∗ − .
22 0 . − . ∗∗ . − . ∗∗ − .
16 0 . C . outD − . ∗∗ . − . ∗∗ − .
16 0 . − . ∗∗ . − . ∗∗ − .
30 0 . ** p < .
01, * p < . In order to further reduce the number of variables, weemploy a four phase approach in such a way that eachsubsequent phase of analyses takes into account a nar-rower, but more relevant set of factors.
Phase I: Which centrality measures contribute toanxiety shift?
Given our exploratory approach to investigating therelationship between students’ embeddedness within thein-class network and anxiety, we run simple linear mod-els looking at the predictive value of the centrality indicespresented in Sec. III C on the normalized anxiety shift.The simple models test three measures of centrality as in-dependent variables: indegree, outdegree, and closeness.Because it is unclear from the perspective of physics anxi-ety whether it is more important to weigh repeated inter-actions with the same individuals as opposed to multipleinteractions with different individuals, we calculate eachcentrality measure using four different tuning parame-ters α [38]. As discussed in Sec. III C, α allows to controlfor the relative importance of the number of edges andtheir weights (see Eq.(2) and Eq.(3)). The four valueswe choose, α ∈ { . , . , . , . } , reflect four differentways to weigh the strength of repeated interactions be-tween the same two individuals. In what follow, we usethe subscript convention to indicate which centrality werefer to (i.e., inD for indegree, outD for outdegree and C for closeness) and superscript for the tuning parameterused to weigh interactions when calculating a particulartype of centrality measure (e.g., C . inD denotes indegreewith α = 1 . M slr : anxiety.shif t ∼ centrality . This gives12 different tests, four for each measure. Each test isrun as a permutation test for linear models to verify itsstatistical significance. Our tests on the network datacollected at the end of the semester reveals no signifi-cant relationship between normalized anxiety shifts andindegree, regardless of the tuning parameter value. Out-degree (regardless of α ) and closeness ( α >
0) are sig-nificant predictors of normalized anxiety shift. However, when adjusted for false positives (type I error), only out-degree remains significant (for all α ). The negative es-timates suggest that the greater a students’ outdegree,the more likely that student is to experience a larger de-crease in their anxiety from the beginning to the end ofthe semester (see Table III for the regression estimatesfor outdegree from fifth collection). The standardizedbeta estimates β range from − .
22 to − .
28, with anaverage of − .
26. In other words, on average, for everyone standard deviation increase in a student’s outdegree,their normalized physics anxiety would decrease by 0 . Phase II: When do centrality measures start tomatter?
In order to implement an intervention aimed at miti-gating students’ physics anxiety, it is important to knowwhich students are “at risk” when there is still timeto intervene. Thus, we seek to identify when duringthe semester might be an appropriate time to do so.Since we have access to data collected five times through-out the semester, we proceed to investigate the corre-lation between anxiety shift and outdegree on earliercollections. We run simple linear regressions with out-degree as a predictor of normalized anxiety shifts, i.e, M slr : anxiety.shif t ∼ centrality , for each of the fouruntested data sets, i.e., collections one through four. Wetest each collection for the same values of the tuning pa-rameter α as in Phase I. These tests are also run usingpermutation techniques. We find outdegree to be a sig-nificant predictor of normalized anxiety shift beginningin collection four, regardless of the tuning parameter used(see Table III for the regression estimates for outdegreefrom fourth collection). Outdegree is not significantlycorrelated with the shift in physics anxiety for collectionsone, two, and three.3 Phase III: Which tuning parameter makes the mostsense?
The tests described in Phase II reveal that outdegreecentrality begins to play a role in students’ physics anx-iety shift sometime around the fourth data collection,which took place after the second midterm which alsohappens to be a group exam. In order to determine howto best weigh repeated interactions between the sametwo individuals, we compare the four simple models thatrely on different tuning parameter values using data fromthe fourth collection. All of our models share nearly thesame R-squared value and standardized estimates (seeTable III). The negligible variance across these valuesprovides no justification for choosing one parameter overanother, meaning that giving more weight to repeated in-teractions with the same individuals makes no differencein our models. This suggests that the weighted networkdata is no more informative for anxiety shifts than thesimple, binary network would be. The practical implica-tions of this observation will the discussed in Sec. VI. Forthat reason, we choose to test our final model using out-degree with α = 0 .
0, i.e., the standard version of degreethat does not take frequency of repeated interactions intoaccount.
Phase IV: Determining the final model
Our final linear regression model takes a variety of con-trol variables into account, as per prior literature. Ourcontrol block includes anxiety at the beginning of thesemester, i.e., pre-course scores ( pre.anxiety ), a binarygender variable (female or male, gender ), and final coursegrade ( f inal.grade ): M full : anxiety.shif t ∼ centrality + gender + f inal.grade + pre.anxiety. We find that, regardless of students’ anxiety at the be-ginning of the semester, gender, and final course grade,outdegree with α = 0 . β = − .
19, standard error of the standardized estimate
SEβ = 0 . t -test statistics t = − .
47, significance level p < . β = − . SEβ = 0 . t = − . p < . β = − . SEβ = 0 . t = − . p < . β = − . SE = 0 . t = − . p < . f inal.grade factor removed. As can be seen in Table IV, TABLE IV. Summary of the simplified linear regression modelfor anxiety shift with outdegree centrality from fourth collec-tion ( α = 0 .
0) and with the final.grade factor removed: thestandardized estimate ( β ), the standard error for the stan-dardized estimate (SE β ), and t test statistic ( t ). We considernetworks without instructional staff. Significant p -values aremarked with an asterisk.Factor β SE β tC . outD − . ∗∗∗ . − . − . ∗∗ . − . − . ∗∗∗ . − . *** p < . p < . in the absence of final grades data, the outdegree mea-sure and pre-course anxiety become the most significantpredictors for anxiety shift. For every one standard devi-ation increase in a student’s outdegree, their normalizedphysics anxiety would decrease by 0 .
29 standard devia-tion.
V. DISCUSSION
We start our exploration of the relationship betweenstudents’ classroom interactions and their anxiety bylooking at changes the latter. Students’ average pre and post physics anxiety scores exhibit no statistical differ-ences, yet the data and its distribution indicate that whileoverall shift does not occur, individual shifts do. Somestudents experience increases in anxiety, while others ex-perience decreases. We want to better understand thefactors that might contribute to these changes. Priorresearch in active-learning physics classrooms indicatethat student self-efficacy, a construct related to anxi-ety, correlates with the kinds of classroom interactionsstudents participate in [12]. Moreover, the broader lit-erature on anxiety suggests that student behavior andclassroom participation has reciprocal relationships withanxiety [19–21].We quantify the social integration of students in theclassroom using the tools of SNA. After surveying stu-dents regarding the meaningful academic interactionsthey participated in, the list of interactions derived fromtheir responses are used to calculate three importantmeasures of individuals’ relational position in the net-works: indegree, outdegree, and closeness. Simple lin-ear models between students’ normalized shifts in physicsanxiety and each of these centrality measures reveals asignificant relationship only for the outdegree: the moreinteractions students report having, the more likely theyare to experience a decrease in physics anxiety. Given thecorrelational nature of these models, we would also ex-pect students whose anxiety decreases over time to reporta greater number of meaningful academic interactions.The relationship between physics anxiety and class-room interactions is meaningful, given the overall trendtowards active learning modalities in physics teach-4ing. Research suggests that for some students, activelearning environments may cause discomfort and anxi-ety [13, 51, 52], which can lead to suppressed performanceor loss of interest – factors that affect persistence in amajor [53]. Physics instructors that solicit peer learningmust take into consideration a variety of ways to groupstudents in order to optimize outcomes like learning andimproved attitudes towards physics. Given the relation-ship between these factors and anxiety, our study sug-gests students should be given opportunities to interactwith as great a number of peers as possible.Students’ outdegree can be interpreted in two ways.It can be thought of as the number of interactions thestudent in question actively engages in. This interpre-tation assumes that the student is exercising agency intheir interactions, listing peers they purposefully soughtafter. Overall trends in network data from this and simi-lar physics classrooms suggest this to be the case [54, 55].The other possible interpretation does not necessarily im-ply a form of student agency, but rather considers stu-dent perception instead. Students who perceive havinghad more meaningful interactions, regardless of whetherthey initiated these interactions or not, list these inter-actions on a survey and, as a result, have greater outde-gree centrality than those who do not perceive having asmany meaningful interactions. This interpretation sug-gests a reciprocal relationship between anxiety and thenumber of meaningful interactions students perceive hav-ing. When taking this latter interpretive approach, in-teractions listed may include passive events where thestudent was the subject of someone’s initiative ratherthan the actual initiator. We find this unlikely to bethe case given that indegree, a truly passive measure ofwhich the student has no control, was not a significantpredictor of anxiety shifts. In other words, simply be-ing the subject of others’ interactions is not related toanxiety shifts. More likely, students must initiate the in-teraction in at least some of the cases in order to benefitfrom the relationship between outdegree centrality andnegative shifts in physics anxiety. Regardless of one’s in-terpretation, the act of identifying and listing meaningfulinteractions must be taken by the student.Our analyses also indicate that when exploring studentinteractions in the physics classroom, the advantage pro-vided by taking into account the frequency of repeated in-teractions between the same two individuals is relativelysmall. A comparison of beta estimates and R-squaredvalues reveals only minor differences between the effectsize of outdegree, regardless of whether we used a tun-ing parameter that did not take repeated interactionsinto account (i.e., C . outD ) or one that greatly advantagedstudents with repeated interactions (i.e., C . outD ; see Ta-ble III). No other study examining classroom interactionshas compared the outcomes of not taking repeated inter-actions into account versus doing so. Given the extracognitive effort required for students to recall the repeat-edness of interactions, as well as the additional work in-volved in both collecting and analyzing this type of data, it seems that the frequency of interactions can be ignored(unless prior literature indicates a potential increased ef-fect).On the other hand, students’ self-reported gender, pre-course anxiety and final grade in the course all signifi-cantly contribute to predicting students’ shifts in anxi-ety. As expected, male students are more likely to ex-perience decreases in anxiety, as are students who fin-ished the semester with higher grades [23–25]. Studentswith higher outdegree measured sometime after the sec-ond midterm are also more likely to experience decreasesin physics anxiety.Of all these variables, outdegree lends itself most read-ily to direct intervention design given that it can be easilymeasured and, unlike final grades, plays a role long beforethe semester ends. Instructors can help students feel lessanxious by creating an environment that fosters and in-vites social interactions related to the content. We shouldnote that students in these classrooms, on average, re-ported interacting with more than just their group mem-bers. Average outdegree during the fourth and fifth col-lection is 4.74 ( SD = 4 .
50) and 5.73 ( SD = 4 . VI. SUMMARY
SNA not only provides a novel set of tools that can helpphysics education researchers better understand how so-cial interactions contribute to other factors, it can alsobe used in practical ways to assess social dynamics. Inthis study a simple count of who interacted with whomwould not have drawn out the nuance provided by differ-entiating outdegree from indegree. Moreover we wouldnot have concluded that closeness, the most significantand meaningful centrality measure in terms of predict-ing students’ persistence [8], is not related to changes inphysics anxiety. Our use of SNA makes sense given ourresearch questions, and our outcomes lead to practicalrecommendations for active-learning physics classrooms.In the case of physics anxiety, instructors can use sim-ple SNA surveys throughout the semester to gauge whatkind of interactions their classroom structure is fostering.This data can be used to quickly calculate student cen-trality using programs like R that automate the majorityof the process. Interventions can then be designed to en-courage the kinds of interactions that maximize positivelearning experiences.5Finally, we encourage researchers to think broadlyabout the potential uses of SNA in research. While wefocus here on the classroom environment, SNA can beapplied to studies of informal learning environments, aswell. These kinds of settings do not necessarily takeplace in a physical space either. Mobile phone appli-cations like Whatsapp and Messenger are often used bystudents outside of class to share information and orga-nize meetings. These virtual communication tools lendthemselves to exploration via SNA. Moreover, social net-works do not necessarily have to involve direct interac-tions, but can be defined to capture physical proximitynetworks, attendance-absence networks, or networks de-fined by non-verbal cues, to name a few. We believe thatthe growing prominence of active-learning strategies andthe relationship between social interactions and studentsuccess will further require the use of SNA to help im-prove student persistence and retention. Implementingthe suggestions here gives the ultimate test of their effi-cacy.
Appendix A: The normalized gain
Since its introduction in 1998, the normalized gain hasbeen commonly used as a measure of students averagedimprovement over time in various context. Defined asa measure of the “average effectiveness of a course inpromoting conceptual understanding” [31], it is typicallyused to capture the average trends for the entire class.By adjusting values measured on different scales, it alsoallows comparison between different groups. However,the normalized gain is not robust when a large drop inscores takes place.For simplicity, lets assume that the scores range from0 to 100 %. The normalized gain on an individual levelis defined as: g norm = post − pre − pre , (A1)where pre and post denote the pre- and post-coursescores, respectively. For averaged gain, as introduced inRef. [31], pre and post need to be replaced by the respec-tive averages over the entire class, i.e., (cid:104) pre (cid:105) and (cid:104) post (cid:105) While this equation always yields values smaller or equalto one (simply because post can be at most 100), when post score is lower than pre score (i.e., when a drop inscores rather than gain is observed), it is possible to seevalues g norm < −
1. This happens if post < pre − , that is if, after scoring more than 50 % on the pre-test, anindividual has a post score of no more than 2( pre − pre and post scores are averaged over the entire class, it is stillpossible to see a “normalized gain” that is outside of[ − ,
1] range, invalidating the comparison between sec-tions. However, this lack of robustness against largedrops in scores should not be thought of as an argu-ment against using the normalized gain. On the con-trary, this property of g norm provides researchers with atool for quick detection of atypical performances and pos-sible outliers (e.g., students who did not give genuine re-sponses on the post-course data collection). We do argue,however, that a distribution of individual gains should beconsidered in addition to comparing the normalized gainvalues. As can bee seen in our data, majority of studentsdid experience a shift in their anxiety, either positive ornegative. However, had we railed solely of the measure ofnormalized shift, we would find no differences as the tra-ditional normalized shift for our data is less than 0 . Appendix B: Descriptive statistics for centralities
TABLE V. The summary of the descriptive statistics for theoutdegree centrality from fourth collection ( N = 53). Basedon Shapiro-Wilk test, the null hypothesis about the normaldistribution is rejected for all centralities. The median andinterquartile range (IQR) are used to describe the distributionand dispersion for each measure. Note that instructional staffis removed from the network.Centrality Shapiro-Wilk test Median Mean W pC . outD < .
001 5.0 7.0 C . outD < .
001 7.2 10.2 C . outD < .
001 11.0 15.0 C . outD < .
001 13.9 19.7
R.D. and J.P.Z. contributed equally to this work. [1] P. Bourdieu. The forms of capital. In J. G. Richardson,editor,
Handbook of Theory and Research for the Sociol-ogy of Education , chapter 10, pages 241–258. Greenwood Press, New York, 1986.[2] N. Lin. Building a network theory of social capital.
Con-nections , 22(1):28–51, 1999. TABLE VI. The summary of the descriptive statistics for allcentrality measures from fifth collection ( N = 53). Basedon Shapiro-Wilk test, the null hypothesis about the normaldistribution is rejected for outdegree and closeness, but not forindegree. The median and interquartile range (IQR) are usedto describe the distribution and dispersion for each measure.Note that instructional staff is removed from the network.Centrality Shapiro-Wilk test Median IQR W pC . inD C . inD C . inD C . inD C . outD < .
001 5.0 5.8 C . outD < .
001 8.2 8.8 C . outD < .
001 12.0 13.8 C . outD < .
001 15.2 17.5 C . C < .
001 0.42 0.08 C . C < .
001 0.43 0.09 C . C < .
001 0.43 0.11 C . C < .
001 0.44 0.11[3] S. P. Borgatti, A. Mehra, D. J. Brass, and G. Labi-anca. Network analysis in the social sciences.
Science ,323(5916):892–895, 2009.[4] National Research Council.
Inquiry and the National Sci-ence Education Standards: A Guide for Teaching andLearning . The National Academies Press, Washington,DC, 2000.[5] J. P. Scott and P. J. Carrington (Eds.).
The SAGE Hand-book of Social Network Analysis . Sage Publications Ltd.,2011.[6] H. Cho and J.-S. Lee. Collaborative information seek-ing in intercultural computer-mediated communicationgroups: Testing the influence of social context using so-cial network analysis.
Commun. Res. , 35(4):548–573,2008.[7] Justyna P. Zwolak, Remy Dou, Eric A. Williams, andEric Brewe. Students’ network integration as a predictorof persistence in introductory physics courses.
Phys. Rev.Phys. Educ. Res. , 13(1):010113, 2017.[8] J. P. Zwolak, M. Zwolak, and E. Brewe. Educational com-mitment and social networking: The power of informalnetworks.
Phys. Rev. Phys. Educ. Res. , 14(1):010131,2018.[9] D. Z. Grunspan, B. L. Wiggins, and S. M. Goodreau.Understanding classrooms through social network anal-ysis: A primer for social network analysis in educationresearch.
CBE Life Sci. Educ. , 13:167–178, 2014.[10] J. Bruun and E. Brewe. Talking and learning physics:Predicting future grades from network measures andforce concept inventory pretest scores.
Phys. Rev. STPhys. Educ. Res. , 9(2):020109, 2013.[11] B. Carolan.
Social Network Analysis and Education:Theory, Methods and Applications . SAGE PublicationsLtd, 2014.[12] R. Dou, E. Brewe, J. P. Zwolak, G. Potvin, E. A.Williams, and L. H. Kramer. Beyond performance met-rics: Examining a decrease in students’ physics self-efficacy through a social networks lens.
Phys. Rev. Phys. Educ. Res. , 12(2):020124, 2016.[13] R. Dou, E. Brewe, G. Potvin, J. P. Zwolak, and Z. Haz-ari. Understanding the development of interest and self-efficacy in active-learning undergraduate physics courses.
Int J Sci Educ. , pages 1–19, 2018.[14] E. A. Williams, J. P. Zwolak, R. Dou, and E. Brewe. Stu-dent engagement as a predictor of academic performancein the introductory physics classroom. arXiv:1706.04121 ,2017.[15] J. P. Zwolak. SNA toolbox; 2018 [cited 2018 aug 23]. https://github.com/jpzwolak/SNA-toolbox , 2018.[16] A. Sfard. There is more to discourse than meets theears: Looking at thinking as communicating to learnmore about mathematical learning.
Educ. Stud. Math. ,46(1):13–57, 2001.[17] A. Sfard. On two metaphors for learning and the dangersof choosing just one.
Educ. Res. , 27(2):4–13, 1998.[18] B. R. Schlenker and M. R. Leary. Social anxiety and self-presentation: A conceptualization model.
Psychol. Bull. ,92(3):641–669, 1982.[19] A. Paivio and W. E. Lambert. Measures and correlatesof audience anxiety (“stage fright”).
J. Pers. , 27(1):1–17,1959.[20] T. Hills. Is constructivism risky? social anxiety, class-room participation, competitive game play and construc-tivist preferences in teacher development.
Teacher Devel-opment , 11(3):335–352, 2007.[21] L. M. Camacho and P. B. Paulus. The role of social anx-iousness in group brainstorming.
J. Pers. Soc. Psychol. ,68(6):1071–1080, 1995.[22] D. Hestenes, M. Wells, and G. Swackhamer. Force con-cept inventory.
Phys. Teach. , 30(3):141–158, 1992.[23] K. Williams. Understanding, communication anxiety,and gender in physics: Taking the fear out of physicslearning.
J. Coll. Sci. Teach. , 30(4):232–237, 2000.[24] Johanna Lepp¨avirta.
Engineering Students’ Proficiencyin Electromagnetics: Role of Procedural and Concep-tual Knowledge, and Mathematics Anxiety in Learningof Electromagnetics . PhD thesis, Aalto University, Fin-land, 2011.[25] S. Tobias. Test anxiety: Interference, defective skills, andcognitive capacity.
Educ. Psychol. , 20(3):135–142, 1985.[26] M. S¸ahin. The relationship between pre-service teachersphysics anxiety and demographic variables.
J. Balt. Sci.Educ. , 13(2):201–215, 2014.[27] L. J. Crockett, M. I. Iturbide, R. A. Torres Stone,M. McGinley, M. Raffaelli, and G. Carlo. Accultur-ative stress, social support, and coping: relations topsychological adjustment among mexican american col-lege students.
Cultur. Divers. Ethnic. Minor. Psychol. ,13(4):347–355, 2007.[28] M. S¸ahin. Development and validation of the physicsanxiety rating scale.
Int. J. Env. Sci. Educ. , 10(2):183–200, 2015.[29] P. J. Tilgner. Avoiding science in the elementary school.
Sci. Educ. , 74(4):421–431, 1990.[30] M. W. Eysenck, N. Derakshan, R. Santos, and M. G.Calvo. Anxiety and cognitive performance: Attentionalcontrol theory.
Emotion , 7(2):336–353, 2007.[31] R. R. Hake. Interactive-engagement versus traditionalmethods: A six-thousand-student survey of mechanicstest data for introductory physics courses.
Am. J. Phys. ,66(1):64–74, 1998.[32] K. E. Pomian, J. P. Zwolak, E. C. Sayre, S. V. Franklin, and M. B. Kustusch. Using social network analysis onclassroom video data. In Proceedings of the Physics Edu-cation Research Conference 2017 , pages 316–319, Cincin-nati, OH, 2018.[33] A. Traxler, A. Gavrin, and R. Lindell. Talking by thenumbers: Networks identify productive forum discus-sions. arXiv:1802.08738 , 2018.[34] E. Brewe, L. H. Kramer, and G. E. O’Brien. Chang-ing participation through formation of student learningcommunities.
AIP Conf. Proc. , 1289:85–88, 2010.[35] S. Wasserman and K. Faust.
Social Network Analysis:Methods and Applications . Cambridge Univ. Press, Cam-bridge, England, 1994.[36] T. Opsahl and P. Panzarasa. Clustering in weighted net-works.
Soc. Networks , 31(2):155–163, 2009.[37] C. Prell.
Social Network Analysis: History, Theory andMethodology . SAGE Publications Ltd, 2013.[38] T. Opsahl, F. Agneessens, and J. Skvoretz. Node cen-trality in weighted networks: Generalizing degree andshortest paths.
Soc. Networks , 32(3):245–251, 2010.[39] Marti J. Anderson and John Robinson. Permutation testsfor linear models.
Aust. N. Z. J. Stat. , 43(1):75–88, 2001.[40] J. A. Smith and J. Moody. Structural effects of networksampling coverage I: Nodes missing at random.
Soc. Net-works , 35(4):652–668, 2013.[41] R Core Team.
R: A Language and Environment for Sta-tistical Computing . R Foundation for Statistical Com-puting, Vienna, Austria, 2016.[42] R. E. Wheeler and M. Torchiano. lmperm.[43] R. E. Wheeler.
Permutation tests for linear models in R ,2016.[44] J. Honaker, G. King, and M. Blackwell. Amelia ii: Aprogram for missing data.
J. Stat. Softw. , 45(7):1–47,2011.[45] G. Csardi and T. Nepusz. The igraph software packagefor complex network research.
InterJournal , Complex Systems:1695, 2006.[46] T. Opsahl.
Structure and Evolution of Weighted Net-works . University of London (Queen Mary College), Lon-don, UK, 2009.[47] Y. Benjamini and Y. Hochberg. Controlling the false dis-covery rate: A practical and powerful approach to mul-tiple testing.
J. R. Stat. Soc. B , 57(1):289–300, 1995.[48] D. Hestenes. Toward a modeling theory of physics in-struction.
Am. J. Phys. , 55(5):440–454, 1987.[49] I. A. Halloun and D. Hestenes. Modeling instruction inmechanics.
Am. J. Phys. , 55(5):455–462, 1987.[50] D. McPadden and E. Brewe. Impact of the secondsemester university modeling instruction course on stu-dents’ representation choices.
Phys. Rev. Phys. Educ.Res. , 13:020129, 2017.[51] M. R. Villasenor and E. Etkina. Reformed physics in-struction through the eyes of students.
AIP ConferenceProceedings , 883(1):105–108, 2007.[52] C. Turpen and N. D. Finkelstein. The construction ofdifferent classroom norms during peer instruction: Stu-dents perceive differences.
Phys. Rev. ST Phys. Educ.Res. , 6:020123, 2010.[53] R. W. Lent, S. D. Brown, and G. Hackett. Toward a uni-fying social cognitive theory of career and academic inter-est, choice, and performance.
J. Vocat. Behav. , 45(1):79–122, 1994.[54] J. Bruun and I. G. Bearden. Time development in theearly history of social networks: Link stabilization, groupdynamics, and segregation.
PLOS ONE , 9(11):1–11, 112014.[55] R. Dou.
The interactions of relationships, interest,and self- efficacy in undergraduate physics . PhD thesis,Florida International University, 2017.[56] B. Rienties and D. Tempelaar. Turning groups inside out:A social network perspective.