Beyond performance metrics: Examining a decrease in students' physics self-efficacy through a social network lens
Remy Dou, Eric Brewe, Justyna P. Zwolak, Geoff Potvin, Eric A. Williams, Laird Kramer
BBeyond performance metrics: Examining a decrease in students ’ physics self-efficacythrough a social networks lens Remy Dou, Eric Brewe,
Justyna P. Zwolak,
Geoff Potvin, Eric A. Williams, and Laird H. Kramer Department of Teaching and Learning, Florida International University,11200 S.W. 8th Street, Miami, Florida 33199, USA Department of Physics, Florida International University,11200 S.W. 8th Sreet, Miami, Florida 33199, USA STEM Transformation Institute, Florida International University,11200 S.W. 8th Street, Miami, Florida 33199, USA (Received 6 October 2015; published 9 August 2016)The Modeling Instruction (MI) approach to introductory physics manifests significant increases instudent conceptual understanding and attitudes toward physics. In light of these findings, we investigatedchanges in student self-efficacy while considering the construct ’ s contribution to the career-decisionmaking process. Students in the Fall 2014 and 2015 MI courses at Florida International Universityexhibited a decrease on each of the sources of self-efficacy and overall self-efficacy ( N ¼ ) as measuredby the Sources of Self-Efficacy in Science Courses-Physics (SOSESC-P) survey. This held true regardlessof student gender or ethnic group. Given the highly interactive nature of the MI course and the dropsobserved on the SOSESC-P, we chose to further explore students ’ changes in self-efficacy as a function ofthree centrality measures (i.e., relational positions in the classroom social network): inDegree, outDegree,and PageRank. We collected social network data by periodically asking students to list the names of peerswith whom they had meaningful interactions. While controlling for PRE scores on the SOSESC-P,bootstrapped linear regressions revealed post-self-efficacy scores to be predicted by PageRank centrality.When disaggregated by the sources of self-efficacy, PageRank centrality was shown to be directly related tostudents ’ sense of mastery experiences. InDegree was associated with verbal persuasion experiences, andoutDegree with both verbal persuasion and vicarious learning experiences. We posit that analysis of socialnetworks in active learning classrooms helps to reveal nuances in self-efficacy development. DOI: 10.1103/PhysRevPhysEducRes.12.020124
I. INTRODUCTION
The implementation of active learning environmentsacross science, technology, engineering, and mathematics(STEM) fields has garnered attention from educationresearchers across the country. Their work has revealedwith strong significance the advantage of active learningstrategies over traditional, lecture-based pedagogies [1].Specifically in the arena of physics education, a varietyof active learning approaches have led to the reformationof introductory physics courses in colleges and universities.These include Investigative Science Learning Environ-ments (ISLE), Student-Centered Activities for LargeEnrollment University Physics (SCALE-UP), WorkshopPhysics, Tutorials in Introductory Physics, and ModelingInstruction (MI) among others. To various degrees, theyhave exhibited positive impacts on student learning [2 – – ’ career decision-making process [11 – Published by the American Physical Society under the terms ofthe Creative Commons Attribution 3.0 License. Further distri-bution of this work must maintain attribution to the author(s) andthe published article ’ s title, journal citation, and DOI. PHYSICAL REVIEW PHYSICS EDUCATION RESEARCH = = = ’ classroominteractions merits attention. II. THE NATURE OF SELF-EFFICACY
Of the constructs related to both performance attainmentand career choice, self-efficacy plays a unique, well-tested,and strongly influential role [9,12 – – ’ s self-efficacy beliefs: mastery expe-riences, vicarious learning, verbal persuasion, and physio-logical states [30]. Students ’ self-efficacy on physics relatedtasks is influenced by (i) students ’ past performance onsimilar tasks (i.e., mastery experiences), (ii) observationsof peers to whom they relate succeeding or failing at thosetasks (i.e., vicarious learning), (iii) direct encouragement ordiscouragement from peers, instructors, and others (i.e.,verbal or social persuasion), and (iv) the emotional andphysiological states of each student at the moment oneassesses their self-efficacy or when students think aboutcompleting the task in question (i.e., physiological states).Anxiety, depression, or excitations are examples of physio-logical states that can contribute to students ’ self-efficacy. A. The social nature of self-efficacy development
Although individuals regulate their self-efficacy inter-nally [10], some of the experiences that contribute to self-efficacy development result from external interactionsin social settings. (Here and throughout the rest of the paperwe define social settings as locations where two or moreindividuals work in close proximity on related tasks). Weargue that development of efficacy beliefs, to an extent,relies on social interactions in these types of settings,which are the hallmark of various reformed physicscourses. Our basis for this begins with how theory definesvicarious learning and verbal persuasion — two of the fourestablished sources of self-efficacy. Vicarious learning andverbal persuasion experiences imply social settings.Vicarious learning (VL) requires that an individual inquestion observes another person succeeding or failing at agiven task. For this to occur, two or more persons must findthemselves in the same space, within reasonable distanceto observe one another ’ s performance. While one mayargue that this need not occur in physical proximity (e.g.,watching videos of someone performing the task), thebulk of formal education environments primarily allow forin-person vicarious learning experiences. The presence of peers does more than create VLopportunities, it also nurtures threatening or affirmingcontexts that result in changes to students ’ overall self-efficacy. This holds particular sway in circumstances whereindividuals rate their performance by comparing theirprogress to that of those around them. In the case wherea person observes others surpassing his or her performance,that individual has a higher likelihood of feeling lessconfident about his or her ability to perform the task athand [29]. Educational settings often place students insituations where they find themselves explicitly or implic-itly ranked among their peers according to their academicsuccess. This ranking need not occur publicly or blatantly,but may be perceived by students nevertheless (e.g., ateacher drawing smiley faces on just a subset of gradedexams).Social interactions are also required in circumstanceswhere individuals receive verbal feedback on performance,which may strengthen or undermine their self-efficacy. Ingeneral, classroom structures provide a forum for thesekinds of verbal persuasion (VP) experiences to take place.Students often receive verbal recognition about theirprogress from teachers, peers, and on occasion, adminis-trators. On a similar note, the type of emphasis placed onthese performance evaluations matters [31]. Feedback thataccentuates shortcomings contributes more to the break-down of efficacy beliefs than feedback that focuses onamount of progress [32].Some studies reveal that the socially oriented sourcesof self-efficacy (i.e., VL and VP) play a more significantrole in the development and sustaining of women ’ s efficacybeliefs [21]. For example, Zeldin and Pajares [20] The context of online education may limit such experiencesand render this statement less valid.
REMY DOU et al.
PHYS. REV. PHYS. EDUC. RES.12,
PHYS. REV. PHYS. EDUC. RES.12,
B. The social nature of Modeling Instruction
Of the existing, reformed instructional approachesdirected at introductory university physics curricula (withCalculus) our research focuses on Modeling Instruction(MI), which differs significantly from the more common,lecture-based introductory-course format. MI introductoryphysics (referred to as “ MI ” from here on out) courses havetended to support low student-instructor ratios, short ornonexistent lectures, high numbers of solicited student-student and student-instructor interactions, and classroomsettings designed to promote small group formation andcollaborative learning. Students explore physical phenom-ena and solve classroom assignments in small groups, usevarious representations to summarize their conclusions on awhite board, and come together during a “ Board Meeting ” to share and evaluate group solutions. Board Meetings — acharacteristic feature of MI — reflect the highly socialnature of learning that takes place [6].The originators of MI developed this approach in orderto promote student engagement for the purpose of medi-ating the construction of physics knowledge [33]. Thisgrounding highlights the dialectical process where indi-viduals reconcile their naïve ideas with concepts presentedin the curriculum, which in this case occurs via exper-imentation and argumentation — the latter better describedas social exchanges of ideas. Further development of MI byDesbien [34], as well as Brewe [6], cemented the inherentlysocial nature of knowledge construction espoused by thisphysics teaching method. Grouping students, encouragingthem to develop physics models together, and then havingthem relate group results to a larger classroom settingprovides participants with opportunities to create knowl-edge and shared meanings or interpretations via verbalexchanges. This relationship between the building ofknowledge and discussion is summarized in a commonmotto of the MI process: learning and social interactions arenot mutually exclusive [35]. Additionally, learning occurswithin a physics context. Students in this active learningenvironment employ a variety of physics-relevant tools,including language, to develop representations of physicsconcepts. For a detailed description of MI, please seeBrewe [6]. Studies have shown that MI has led to increased studentunderstanding in physics and improved attitudes towardphysics [36,37]. Results documented in Brewe et al. [36]showed that students in MI courses have a 6.73 timesgreater odds of success than their counterparts in lecturesections. In addition to successfully passing, students in MIcourses have greater pre-post gains on the Force ConceptInventory than students in traditional, lecture-basedcourses. The researchers observed these learning advan-tages for both women and men, though they note that thepresence of a “ gender gap ” remains. Moreover, MI courses,unlike other successful, reformed physics approaches,positively shift student attitudes toward physics even whenexamined across varied instructors (Avg. effect size:Cohen ’ s d ¼ . ) — a feat accomplished by no other studyknown to us [37]. C. Student self-efficacy in college physics
Research studies have reported correlations betweenself-efficacy and final grade in introductory physicscourses, as well as the likelihood of passing the class[36,38,39]. The same can be said about other introductorycourses in STEM fields, including chemistry, biology, andcomputer science [40 – et al. [41] performed a longitudinal study wherefemale students who experienced increases in their self-efficacy during and after high school were more likely toreport stability in their STEM-related vocational choices.This applied when controlling for high school achievementand socioeconomic status. On the other hand, a decline inself-efficacy had the opposite effect for female studentsand no effect on male students. In general, men whosuccessfully attain STEM careers where underrepresenta-tion of women exists report mastery experiences as thebasis for their persistence and ongoing achievement [18].Women in similar contexts, as noted earlier, seem to rely onVL and VP experiences as attributes of their professionalsuccess [20].Previous studies on MI have explicitly explored stu-dents ’ self-efficacy [44,45]. A study by Sawtelle et al. [46]revealed that respondents taking one of several 30-studentcapacity MI courses at a public research university,regardless of gender, did not exhibit a statisticallyBEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
Research studies have reported correlations betweenself-efficacy and final grade in introductory physicscourses, as well as the likelihood of passing the class[36,38,39]. The same can be said about other introductorycourses in STEM fields, including chemistry, biology, andcomputer science [40 – et al. [41] performed a longitudinal study wherefemale students who experienced increases in their self-efficacy during and after high school were more likely toreport stability in their STEM-related vocational choices.This applied when controlling for high school achievementand socioeconomic status. On the other hand, a decline inself-efficacy had the opposite effect for female studentsand no effect on male students. In general, men whosuccessfully attain STEM careers where underrepresenta-tion of women exists report mastery experiences as thebasis for their persistence and ongoing achievement [18].Women in similar contexts, as noted earlier, seem to rely onVL and VP experiences as attributes of their professionalsuccess [20].Previous studies on MI have explicitly explored stu-dents ’ self-efficacy [44,45]. A study by Sawtelle et al. [46]revealed that respondents taking one of several 30-studentcapacity MI courses at a public research university,regardless of gender, did not exhibit a statisticallyBEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12, increase for women on the VP subscale. On the otherhand, the same study revealed that both male and femalestudents in lecture-based introductory physics coursesexhibited a drop in self-efficacy. This drop held true acrossall four sources of self-efficacy. These findings alignsomewhat with findings by Fencl and Scheel [47] whoshowed that calculus-based physics I courses that employ amixture of reformed pedagogical approaches, in particularstudent collaborations, have a stronger positive impact onstudents ’ self-efficacy than traditionally taught courses.This effect is enhanced for physics majors. Another studyby Sawtelle et al. [44] employed logistic regressionanalysis to show that mastery experiences predict the rateat which male students pass or fail introductory physics,while female students ’ success depends more on vicariouslearning.Although the studies done with students participating inMI take a first step toward our understanding of self-efficacy development in these kinds of active learningenvironments, missing from the analyses are careful con-trols for other variables associated with self-efficacy, suchas student ethnicity, as well as a more focused approach tounderstanding the role played by the MI curriculum ’ s mostprominent feature: social interactions. Considering addi-tional limitations, such as potential selection bias intro-duced by the use of online surveys and the amount ofunincorporated missing data, the propositions of the referenced studies in MI warrant further exploration.Moreover, our investigation will allow us to examine theeffect of the curriculum in larger class-size settings. III. PURPOSE
This study aims to more carefully examine both changesin students ’ self-efficacy in a larger MI course as well as testour belief that the prevalent social interactions that occur inthese courses have a notable relationship with self-efficacydevelopment. This approach does not endeavor to compareMI to lecture-based pedagogies, but rather offers a moreintrospective look at the affective outcomes of MI as anactive-learning curriculum. Using self-efficacy theory as aguide, we suggest that individual students come into classwith certain internal expectations about their performancein the MI course. These expectations may differ accordingto each source of self-efficacy. For example, a studentmay have high expectation to receive praise from others(i.e., VP) but lower expectations to learn from peers (i.e.,VL). Classroom experiences will influence students ’ expectations along the four sources of self-efficacy (seeFig. 1). We pay particular attention to VP and VL becausethe social nature of the MI curriculum leads us tohypothesize heightened prevalence for these events. Weexpect these types of experiences influence overall studentself-efficacy at the end of the semester.Given the increases of student conceptual understandingin MI courses, the social nature of self-efficacy develop-ment, and the highly interactive structure of MI courses, we FIG. 1. Our model of self-efficacy development in active learning environments accounts for students ’ initial self-efficacy and itssubsequent development as a result of classroom experiences. In alignment with theory, some of the development arises from learningexperiences not directly related to social interaction (i.e., mastery experiences) [27]. In addition, we postulate that the social nature ofmany active learning environments has the capability of generating opportunities for students to receive verbal feedback or perceiveothers with whom they relate as successful or unsuccessful on physics tasks (i.e., verbal persuasion and vicarious learning experiences).Thus, we posit a link between certain types of classroom interactions and self-efficacy development. REMY DOU et al.
PHYS. REV. PHYS. EDUC. RES.12,
PHYS. REV. PHYS. EDUC. RES.12, ’ in-class social networks as a proximal measure of typesand abundance of potential VL- and VP-related experiencesthat may play a role in mediating self-efficacy shifts.Specifically, we sought to address the following researchquestions:1. Do students in the MI course experience statisticallysignificant changes in physics self-efficacy as mea-sured by PRE and POST scores on a self-efficacy inphysics instrument (i.e., Sources of Self-Efficacy inScience Courses — Physics)?2. Do students in the MI course experience statisticallysignificant changes in physics self-efficacy scoreswhen disaggregated by the four sources of self-efficacy?3. How are social interactions as measured by studentnetwork centrality in the MI classroom associated tochanges in students ’ self-efficacy?4. Do other variables historically associated with stu-dent success in physics, such as gender, major, andethnicity, contribute to the variance in students ’ POST self-efficacy scores when controlling forPRE scores?
IV. A NOTE ON SOCIAL NETWORK ANALYSIS
Although the employment of social network analysis(SNA) in sociology has been taking place since the 1930s[48], its use in education research has experienced agrowing popularity in recent years [35,49 – et al. present a concise introduction targeted at scienceeducation researchers [51].In brief, social network analysts endeavor to quantify therole of particular individuals in a network and the character-istics of a network and its evolution [53]. Our study focuseson measuring the “ centrality ” of actors in our network.Centrality can be calculated from students ’ interactions in avariety of ways. For example, the most basic form ofcentrality is “ degree ” centrality, which simply refers to thenumber of people with whom a person in a networkinteracts [35]. Other measures in the centrality familyinclude inDegree, outDegree, PageRank, Closeness, andBetweenness. These may be calculated using the samestudent interaction information. We collected student network data in order to calculatethree specific measures of directed centrality: inDegree,outDegree, and PageRank. InDegree centrality measuresdirect incoming interactions (i.e., the number of timesstudent Y is listed by peers) and outDegree measures directoutgoing interactions and in some cases can be thought of asa measure of one ’ s sociability (i.e., the number of peersstudent Y lists). PageRank captures direct incoming inter-actions while taking into account the social connectednessof nodes leading to a student. PageRank offers a measure ofweight to being named directly by a student who is oftennamed by others. The PageRank algorithm establishes anode ’ s importance using the number of links to the node,but also each node can then redistribute that importance byits number of outgoing links [35]. It is worth noting thatstudents reported more often by others will have higherinDegrees and tend to have higher PageRanks. That is to saythat inDegree and PageRank may be interpreted as measuresof popularity or recognition from other actors in a socialnetwork since the more often a person is named, the morehis or her PageRank grows. We chose to examine thesethree measures of centrality (i.e., inDegree, outDegree,PageRank) primarily because they limit our analysis ofthe relationship between self-efficacy and social interactionsto students who had direct interactions with one another.They also follow with the uses and recommendations of pastresearch [49 – ’ self-efficacy, we mayget a clearer picture of the kinds of interactions that matterfor student self-efficacy formation in MI courses.Specifically, we secured responses from students on thisquestion: “ Name the individual(s) you had a meaningfulclassroom interaction with today. ” (see Sec. VI for moredetails about the network survey). Responses to this questioncan be used to calculate a plurality of network measures, notjust the ones addressed in this study. Given student responses,InDegree can then be characterized as the number ofincoming connections for a student. The number of partic-ipants a student reports or initiates interactions with is thatperson ’ s outDegree. PageRank takes a more sophisticatedapproach to measuring the “ importance ” of a student or actorin a network. Developed by Brin and Page [54] for the Googlesearch engine algorithm, the measure has been compared tocalculating the probability of a random walker on a directednetwork to arrive at a particular node [55]. This means that notonly does a node ’ s inDegree affect its PageRank, but so doesthe inDegree of its neighbors (see Sec. VII. B.). V. CONTEXTA. Florida International University
Florida International University (FIU), Miami ’ s public,urban research university, boasts a unique population. TheBEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
Florida International University (FIU), Miami ’ s public,urban research university, boasts a unique population. TheBEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12, ’ s students identify themselves as “ Hispanic, ” while13% identify themselves as “ Black, ” another 12% as “ White, ” and 13% as “ Other ” [56]. FIU is classified as aHispanic Serving Institution (HSI), offering criticallyimportant services to the members of its community, whichare primarily Hispanic. Considering recent national callsfor a greater number of STEM majors, many of whichinclude an emphasis on recruiting from underrepresentedgroups [8,16], it is relevant that no other university awardsmore STEM bachelor ’ s degrees to underrepresented minor-ities than FIU [57]. B. Introductory Physics I with Calculus at FIU
Students regardless of major or academic year have theoption of self-selecting into one of the MI sections offeredeach semester or the lecture-based sections of IntroductoryPhysics I with Calculus. The MI course incorporates the labcredit. It is worth noting that student familiarity with the MIapproach varies. For example, students registered in theFall 2015 MI courses responded differently to being askedabout their expectations for the course. Of the 44 surveyrespondents, 9% expected a course only slightly differentthan lecture, 32% expected a much more interactive andhands-on experience, while a similar number expected nodifferences from a traditional lecture-based course.Remaining students either had no expectations or didnot respond. Lecture sections at FIU usually have enroll-ments that range from 120 to nearly 400 students, thoughsome offer a much lower class size limit. Students in alecture section usually register concurrently for a respectivelaboratory course, but are not required to do so. In the Fallof 2014 only one section of MI was offered, limitingstudents ’ scheduling flexibility, but this particular sectionwas the first designed to serve 75 students — over twice thenumber of students previously attempted — in a technology-saturated classroom specifically designed for active learn-ing. Prior iterations of the course limited enrollmentcapacity at 30. Two sections of the large-capacity MIcourse were offered during the Fall 2015 term — one taughtby the same experienced instructor who taught the Fall2014 course and another taught by a postdoc. In order toaccommodate the larger number of students, two graduateteaching assistants and three experienced learning assist-ants (i.e., undergraduate students) helped to facilitateinstruction during courses in both terms. Only data fromclasses taught by the same primary instructor were used inthis study in order to minimize confounding variablesintroduced by having data from different instructors. VI. METHODS
We obtained student data from FIU ’ s database, whichkeeps a record of student responses to demographic questions answered at the time they apply to the university.Some of the majors represented in the courses includedEngineering, Chemistry, Pre-Med, and English. No studentin either MI course (i.e., Fall 2014 and Fall 2015) haddeclared physics as a major at the beginning of thesemester, though we should note that students who declareddual majors were categorized under a larger umbrella(i.e., DUALFIU), which may include physics majors.The classes were composed of four prominent ethnicgroups into which students identified: Asian, White,Hispanic, and Black. The majority of students enrolledin both classes identified themselves as Hispanic (47women and 58 men), while eight identified themselvesas Asian (three women and five men), 13 as White (fivewomen and eight men), and 11 students as Black (twowomen and nine men). Four students identified as other ormore than one race. The race and gender of the remainingsix students in our data set were not available.Self-efficacy surveys were administered in class on thefirst day of each semester (i.e., pre) and once during the lastweek of the semester (i.e., post). We had an overall 92%response rate on the pre based on a total of 147 studentswho registered for Fall 2014 and Fall 2015 MI courses. Ourpost administrations yielded an 80% response rate. A. Self-efficacy survey: Strengths and limitations
We employed the 33-item SOSESC-P survey to gaugethe sources of self-efficacy and to get a measure of overallstudent self-efficacy. We chose this survey for a variety ofreasons, including its specific designation for physicsclassroom settings given that self-efficacy measures requiretask-relevant items in order to align with the construct ’ sdefinition [28]. The SOSESC-P was designed so thatresponses to statements can be disaggregated by each ofthe four sources of self-efficacy. We achieved an overallreliability alpha coefficient of 0.94, and reliability coef-ficients of 0.73 for verbal persuasion (7 items), 0.76 forvicarious learning (7 items), 0.84 for physiological mech-anisms (9 items), and 0.86 for mastery experiences (10items) subscales. These values align with past research ledby the instrument ’ s developers [39]. In that same study thesurvey was shown to correlate well with the Self-Efficacyfor Academic Milestones Strength scale — a positivelyrecognized and validated instrument. Some of the state-ments on the survey included the following: “ I am capableof receiving good grades on assignments in this class ” (mastery experience) and “ I will get positive feedbackabout my ability to recall physics ideas ” (verbal persua-sion). Students used a five-point Likert scale to expressagreement or disagreement with these. Overall scores in ourstudy ranged from 79 to 165. The use of the SOSESC-Palso supported continuity with past studies performed atFIU that employed the same instrument.Though the SOSESC-P was designed for the purposeof measuring overall self-efficacy and the sources ofREMY DOU et al. PHYS. REV. PHYS. EDUC. RES.12,
We employed the 33-item SOSESC-P survey to gaugethe sources of self-efficacy and to get a measure of overallstudent self-efficacy. We chose this survey for a variety ofreasons, including its specific designation for physicsclassroom settings given that self-efficacy measures requiretask-relevant items in order to align with the construct ’ sdefinition [28]. The SOSESC-P was designed so thatresponses to statements can be disaggregated by each ofthe four sources of self-efficacy. We achieved an overallreliability alpha coefficient of 0.94, and reliability coef-ficients of 0.73 for verbal persuasion (7 items), 0.76 forvicarious learning (7 items), 0.84 for physiological mech-anisms (9 items), and 0.86 for mastery experiences (10items) subscales. These values align with past research ledby the instrument ’ s developers [39]. In that same study thesurvey was shown to correlate well with the Self-Efficacyfor Academic Milestones Strength scale — a positivelyrecognized and validated instrument. Some of the state-ments on the survey included the following: “ I am capableof receiving good grades on assignments in this class ” (mastery experience) and “ I will get positive feedbackabout my ability to recall physics ideas ” (verbal persua-sion). Students used a five-point Likert scale to expressagreement or disagreement with these. Overall scores in ourstudy ranged from 79 to 165. The use of the SOSESC-Palso supported continuity with past studies performed atFIU that employed the same instrument.Though the SOSESC-P was designed for the purposeof measuring overall self-efficacy and the sources ofREMY DOU et al. PHYS. REV. PHYS. EDUC. RES.12, ’ context,including gender and ethnicity, may shift the combinationof sources that contribute to students ’ actual self-efficacy.We present this as a limitation of our study and for thatreason we report on analyses of each source of self-efficacyseparately, in addition to students ’ total score on theSOSESC-P, which we interpret as a proxy for studentself-efficacy. We do so on the grounds that we foundsignificant change on all four sources of self-efficacy andcriteria established by past studies [39,44,46,47,59]. B. Social network survey
Since we could not directly measure when a studenthappens to have a meaningful VP or VL experience, weadopted an indirect approach that quantifies the number andtypes of social interactions students have using SNA. Wealso did this to test the model that the quantity and qualityof certain kinds of interactions correlates with changes instudents ’ self-efficacy and sources of self-efficacy. Tomeasure relevant social interactions we administered asocial network survey on the last day of the first weekof class and subsequently once a month until the end of thesemester for a total of 5 administrations. The developmentof this short survey took place under the guidance of thePER group at FIU, building off a previously used survey[49]. Of the open-ended questions appearing on thissurvey, only the first is relevant to this study: “ Name theindividual(s) (first and last name) you had a meaningfulclassroom interaction with today, even if you were not themain person speaking or contributing. (
You may includenames of students outside of the group you usually workwith ). ” We provided a note to participants stating, “ class-room interaction includes but is not limited to people youworked with to solve physics problems and people that youwatched or listened to while solving physics problems. ” Blank space was provided so that participants could list asfew or as many individuals they wished to. We carefullyanalyzed responses in order to identify the students listed.When 100% certainty or agreement could not be estab-lished as to the identity of a written name, a unique codewas created for that specific report. This occurred five timeswhen students with common first names were reported sanslast name. To avoid this issue in the Fall 2015 course, weattached a numbered roster of students to the survey.
VII. RESULTSA. Diagnosing changes in self-efficacy
Prior to performing t tests we imputed student responsesto the SOSESC-P in order to preserve the structure of ourdata, which reduces the rate of type I error by better accounting for nonresponses than would simply removingthose cases from the analysis [60]. Multiple imputation is aMonte Carlo technique that replaces missing values using alikelihood function that assumes missing data is missingat random (MAR) and not because of reporting bias notcaptured by other variables [61]. For that reason weincluded responses to pre and post SOSESC-P surveys,student GPA at the start of the course, gender, and centralitymeasures when estimating values for the missing data.Given that we had no more than a 20% nonresponse rate onthe SOSESC-P we ran five imputations ( m ¼ ) as sug-gested by the literature using the Amelia II package [62] inR [63]. We ran the same analyses on all five data setsand pooled the results according to Rubin [64,65]. Sinceimputed values were generated for missing cases, theresulting N (i.e., N ¼ ) included all unique participantsenrolled in the fall courses during the first week of thesemester.We performed a dependent samples t test to comparethe mean total scores of the pre SOSESC-P responses( M pre ¼ . , SD ¼ . ) to those of the post( M post ¼ . , SD ¼ . ). The outcome revealed astatistically significant drop in physics-related self-efficacyfrom the beginning of the semester to the end of thesemester [ t ð Þ ¼ − . , p < . ] with a small tomedium effect size (Cohen ’ s d ¼ . ). In order to furtherexplore the breakdown of students ’ sources of self-efficacy,we disaggregated responses on the SOSESC-P according tothe following sources of self-efficacy: mastery experiences(ME), VL (i.e., vicarious learning), VP (i.e., verbal per-suasion), and physiological states (PS). Dependent sample t tests on each of these subsections showed a statisticallysignificant drop in students ’ sources of self-efficacy onevery portion of the survey even when setting our thresholdalpha at 0.0125 in order to apply a Bonferroni correction todiminish type I error (see Table I). B. Measuring social interactions
We combined students ’ responses to the social networksurvey across the first four administrations. We did thiswith the goal of preserving uniformity of data collection.We planned for five survey administrations with therequirement that they take place during a typical MI classin which student groups work together on collaborativeactivities. Student interactions were primarily studentgenerated and participants worked on physics related tasks.We achieved this setting across the first four data collec-tions from both semesters in question, which had responserates of over 75%. Final exam scheduling altered theintended environment for the fifth administration both inthe fall of 2014 and in the fall of 2015. Still, we pursuedcollection of data from the last survey, which was givenduring optional final exam review classes where studentswho chose to attend were not encouraged to participate inactive-learning physics related inquiry. This is relevantBEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
We combined students ’ responses to the social networksurvey across the first four administrations. We did thiswith the goal of preserving uniformity of data collection.We planned for five survey administrations with therequirement that they take place during a typical MI classin which student groups work together on collaborativeactivities. Student interactions were primarily studentgenerated and participants worked on physics related tasks.We achieved this setting across the first four data collec-tions from both semesters in question, which had responserates of over 75%. Final exam scheduling altered theintended environment for the fifth administration both inthe fall of 2014 and in the fall of 2015. Still, we pursuedcollection of data from the last survey, which was givenduring optional final exam review classes where studentswho chose to attend were not encouraged to participate inactive-learning physics related inquiry. This is relevantBEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12, ’ interactions, and thestudent network may reflect this. The uncharacteristicenvironment of this setting yielded less than 50% responserates and altered the resulting student network. In order tomaintain fidelity of implementation, data from thesesurveys were not admitted into the final results, thoughanalysis revealed nearly identical outcomes when included.From the responses to the network survey question weconstructed directed edge lists indicating the source of theinteraction (i.e., student responding to the survey) and eachtarget listed on the survey (i.e., student name written inresponse to the question). The edge lists from the first fourcollections were combined and every interaction given avalue of “ ” Repeated interactions with the same targetswere given a weight of þ for each additional time thetargets were listed on other administrations of the samesurvey question (see Fig. 2 for an example of the Fall 2014network structure). After combining data from both semes-ters, students ’ total inDegree ( M ¼ . , SD ¼ . ),outDegree ( M ¼ . , SD ¼ . ), and directed PageRank( M ¼ . × − , SD ¼ . × − ) were calculated in Rusing the igraph package [59]. InDegree was calculated byadding up the number of times a student was listed onquestion one of the four network surveys. OutDegree wascalculated by adding up the number of individuals eachparticular student listed on question one of all four surveys,including instructors. Directed PageRank was calculated in igraph from incoming and outgoing links using the algo-rithm developed by Brin and Page [54] and represented by p ð i Þ ¼ qn þ ð − q Þ X j ∶ j → i p ð j Þ K out ð j Þ i ¼ ; ; … ; n; ð Þ where p is the PageRank of node i , j represents a node in thenetwork linked to i , p ð j Þ and k out ð j Þ are the PageRank and outDegree of node j , respectively, and q is a dampingfactor commonly set at 0.15 as precedent in the literature[55,67].We tested four linear regression models that aimed topredict total post self-efficacy scores while controlling forpre scores. Because network data often fails to meet theassumption of independence, measures of centrality oftenresult in non-normal distributions. Bootstrapped linearregressions do not require assumptions about the distribu-tion; therefore, we used this technique in order to accountfor any dependency in data retrieved from the socialnetwork [68]. Bootstrapping is a Monte Carlo approachthat applies a random resampling of the existing data set tocalculate a set of regression coefficients on that sample. Wedid so over 1000 iterations on each dataset and created adistribution of coefficients by which to compare the valuesin our data [69]. 95% confidence intervals (CI) for ourparameters were calculated using the bias-corrected andaccelerated method developed by Efron [70], which betteraddresses bias and skewness while producing narrowerintervals. These analyses were run on each of our impu-tations with nearly identical results, which were thenpooled. Although, in general, all four models predictedthe dependent variable, the models revealed that PageRankwas the only statistically significant predictor besides thecontrol variable. Regression coefficients for inDegree andoutDegree had confidence intervals that included zero.PageRank explained an additional 3.7% of the variance instudents ’ post self-efficacy scores (see Table II). Because ofpotential collinearity between the centrality measures,we tested these variables using separate models. Thecorrelation between PageRank and inDegree was 0.46( p < . ), between PageRank and outDegree was0.24 ( p < . ), and between inDegree and outDegreewas 0.76 ( p < . ). Again, because centrality measurestypically fail to meet the assumption of normality requiredby traditional statistical tests, the above correlations were FIG. 2. Combined student network in the MI course for Fall2014 drawn using the Force Atlas algorithm on Gephi [66].Sphere size represents PageRank centrality and edge thicknessrepresents weight of tie. Instructors have been removed.TABLE I. Although students in MI courses typically showconceptual and attitudinal gains, these results suggest thatstudents in MI experience a statistically significant drop inphysics self-efficacy. This drop also shows up significantly onall subsections of the SOSESC-P.Changes in sources of self-efficacy scores:Dependent samples comparisons of SOSESC-P shifts(post-pre; N ¼ )Total score ME VL PS VPPre 135.36 40.87 29.74 34.76 30.11 SD .
30 4 .
82 2 . Post 129.11 38.54 28.1 33.28 28.9 SD .
05 6 .
44 3 . Diff. in mean − . a − . a − . a − . a − . a t value − . − . − . − . − . Cohen ’ s d a p < . REMY DOU et al.
PHYS. REV. PHYS. EDUC. RES.12,
PHYS. REV. PHYS. EDUC. RES.12, F statistics with p values of less than 0.001,only PageRank predicted postscores on the ME subsection( B ¼ , β ¼ . , CI ½ ; (cid:2) , SE ¼ ), inDegreepredicted some of the variance in postscores on the VPsubsection ( B ¼ . , β ¼ . , CI ½ . ; . (cid:2) , SE ¼ . ),and outDegree predicted some of the variance in postscoreson the VP ( B ¼ . , β ¼ . , CI ½ . ; . (cid:2) , SE ¼ . )and VL ( B ¼ . , β ¼ . , CI ½ . ; . (cid:2) , SE ¼ . )subsections. No centrality measure predicted students ’ postPS scores in a statistically significant way when controllingfor prescores. Prescores on each section always predictedpostscores. These results lend credence to the argumentthat students ’ relational position in the social networkof a classroom is associated with changes in the sourcesof self-efficacy even when including nonsocial sources ofself-efficacy, like mastery experiences. We address thisfurther in our Discussion section. C. Examining other relevant variables
In order to gauge whether changes in students ’ self-efficacy scores were related to the presence of othervariables associated with student performance, we under-took several additional analyses. Two separate student ’ sindependent samples t tests were run to determine whether or not a difference exists between female and malestudents ’ pre- and postscores on the SOSESC-P. Theanalysis revealed that no statistically significant genderdifference existed at the start of the MI courses or at theend. The same held true when examining the disaggregatedsources of self-efficacy. Furthermore, a multiple linearregression model was examined to determine the abilityof ethnicity and major, along with gender, to predict thevariance in student self-efficacy scores at the end of thecourse when controlling for prescores. The results showedthat the model was statistically significant ( p < . ), butthe only variable that contributed to the model ’ s signifi-cance was prescore. Neither ethnicity nor declared majorcontributed to the variance in students ’ post self-efficacyscores, though to be sure, the low number of representativesfrom certain ethnic groups (e.g., Black) and majors (e.g.,English) limited the power of our model and our ability tomake strong claims about the effect of ethnicity and major.Given that gender differences were not seen on pre- andpost self-efficacy scores, we did not expect this variable tobe significant.Bootstrapped analyses revealed no difference betweenthe mean outDegree nor PageRank of female and malestudents. Nevertheless, male students on average hadslightly higher inDegrees than did female students[ t ð . Þ ¼ − . , p < . , Cohen ’ s d ¼ . ]; seeTable III. VIII. DISCUSSION
Our examination of an active-learning, introductoryphysics course format revealed that regardless of gender,major, and ethnicity, students had on average lower beliefsabout their ability to successfully complete physics relatedtasks at the end of the semester than they did at thebeginning. This negative change was seen across the
TABLE II. Models using network variables predicting post self-efficacy scores. InDegree and PageRank centralities capture a measureof recognition, but PageRank weighs that recognition according to the popularity of peers interacting with the student. Here we showthat only PageRank predicts overall self-efficacy scores. Note standardized regression coefficients (i.e., β ) appear in parentheses.Model-level statistics F statistic F ð ; Þ ¼ . F ð ; Þ ¼ . F ð ; Þ ¼ . F ð ; Þ ¼ . R square 0.276 0.313 0.291 0.29395% CI forR square (0.114, 0.445) (0.159, 0.472) (0.122, 0.453) (0.128, 0.451)Regression coefficientsPredictors Model 1 Model 2 Model 3 Model 4Pre SOSESC-P 0.65 ( β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . PageRank 1380 ( β ¼ . )CI ½ ; (cid:2) ; SE ¼ inDegree 0.37 ( β ¼ . )CI ½ − . ; . (cid:2) ; SE ¼ . outDegree 0.22 ( β ¼ . )CI ½ − . ; . (cid:2) ; SE ¼ . BEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
TABLE II. Models using network variables predicting post self-efficacy scores. InDegree and PageRank centralities capture a measureof recognition, but PageRank weighs that recognition according to the popularity of peers interacting with the student. Here we showthat only PageRank predicts overall self-efficacy scores. Note standardized regression coefficients (i.e., β ) appear in parentheses.Model-level statistics F statistic F ð ; Þ ¼ . F ð ; Þ ¼ . F ð ; Þ ¼ . F ð ; Þ ¼ . R square 0.276 0.313 0.291 0.29395% CI forR square (0.114, 0.445) (0.159, 0.472) (0.122, 0.453) (0.128, 0.451)Regression coefficientsPredictors Model 1 Model 2 Model 3 Model 4Pre SOSESC-P 0.65 ( β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . β ¼ . )CI ½ . ; . (cid:2) ; SE ¼ . PageRank 1380 ( β ¼ . )CI ½ ; (cid:2) ; SE ¼ inDegree 0.37 ( β ¼ . )CI ½ − . ; . (cid:2) ; SE ¼ . outDegree 0.22 ( β ¼ . )CI ½ − . ; . (cid:2) ; SE ¼ . BEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12, — a correction thisprior study did not apply. However, the drop we found isrelatively small compared to the range of the self-efficacyscale and the variance in student responses. The dropmay simply reflect a correction of students ’ overconfi-dence [43].In light of past research on student academic outcomes inMI, what captures our interest is that students experienced adecrease as opposed to an increase in all the sources ofself-efficacy. In fact, we hypothesized increases both onself-efficacy as a whole and on each of the four sources.The decrease found was approximately 73% as large asdecreases seen in past studies with students in lecturecourses [46]. These contrary results point to the need forfurther exploration of this topic, in particular withregard to factors that mediate these shifts. We should alsonote that our students started at higher levels than pre-viously reported studies using the SOSESC-P [46,47].While a variety of variables may have contributed tothis latter attribute, any justification would merely bespeculative.Given the inherently social aspects of self-efficacydevelopment in addition to the emphasis on discourse-based learning in the MI curriculum, we tested whetherstudents ’ social behavior predicted self-efficacy shifts. Weaver that a relationship exists between at least one kind ofinteraction, as captured by student PageRank centrality, andchanges in students ’ overall efficacy beliefs. We found thatthe number of times a student is listed by popular peersmakes a difference (see PageRank in Table II). That is tosay that being named by a student whom others reporthaving a high number of interactions with positively predicts increases in overall self-efficacy. In short, a 1standard deviation increase in student PageRank results in a0.21 standard deviation increase in post self-efficacy aftercontrolling for prescores (see Table II). On the other hand,we did not find that the number of peers a student has ameaningful interaction with (i.e., outDegree) nor thenumber of times a student is recognized by his or herpeers as having contributed to a meaningful interaction(i.e., inDegree) affect changes on the self-efficacy scale asa whole.With regard to the sources of self-efficacy, PageRankalso positively predicted mastery experience scores. Thisdeserves some unpacking, as this source of self-efficacy isnot typically associated with social interactions, but oftenplays a primary role in self-efficacy formation, especiallyfor men [18]. Moreover, the number of both incoming andoutgoing interactions positively predicted verbal persua-sion scores, while only outgoing interactions positivelypredicted vicarious learning scores. None of the inter-actions examined had a statistically significant associationwith students ’ physiological state.These results align with our model of self-efficacydevelopment in active learning environments (seeFig. 1), but also expand on it. They support our beliefthat specific kinds of social academic experiences, asquantified using centrality measures, partially predictstudents ’ postmeasures on the inherently social sourcesof self-efficacy. Yet, the analyses also support expansion ofthe model as centrality was found to have an even strongerrelationship with ME, which we did not consider as asource of self-efficacy related to social networks. In otherwords a student exhibiting a drop because of havingpoor results on a mastery experience (e.g., exam) didnot necessarily strike us as an experience directly relatedto the student ’ s network of peers. Nevertheless, indirectly, itmay be possible that access to a support group in the classmay provide students with capital that leads to improvedperformance as implied by previous studies on teachernetworks and capital theory [71,72].Although our linear models only explain a relativelysmall portion of additional variance, they forge a valuablelink between SNA and the sources of self-efficacy. Asexpected, an increase in the number of times peers interactwith a particular student increases the chances this studenthas positive verbal persuasion experiences. The specificitems on the SOSESC-P suggest that the student isreceiving encouragement about his or her physics ability.This aligns with the fact that others are reporting havingsalient academic interactions with this student. The sameoccurs with regard to a student ’ s outDegree, but this kind ofoutgoing interaction — in the sense that it represents howoften students reach out to peers — is also positively relatedwith vicarious learning experiences. Since vicarious learn-ing experiences theoretically indicate situations where onelearns from watching someone with whom one relates, it is TABLE III. Gender-based comparisons of network centrality.InDegree and PageRank centralities do not differ significantly bygender. On the other hand, female students report more peers (i.e.,outDegree) in response to the network survey question examined.inDegree outDegree PageRankMean differences(female – male) − . a − . T statistic − . − . ’ s d a p < . REMY DOU et al.
PHYS. REV. PHYS. EDUC. RES.12,
PHYS. REV. PHYS. EDUC. RES.12, with whom the interactionoccurs. Interactions coming from popular individuals asdefined by their inDegree positively predict a students ’ sense that they can learn and get good grades in physics.Because students did not know each other ’ s inDegree, wecan infer that students recognize, in some capacity, whothese popular individuals may be and have a perceptionabout their academic popularity. A highly social settingmay catalyze these peer-to-peer judgments.Active-learning environments, like MI, create the kind ofsocial space that allows students the flexibility to interact indifferent ways with different people [45]. Though norelationship was found between gender and self-efficacy,female and male students differ in the kinds of interactionsthey experience. Male students in this class are the subjectsof others ’ meaningful interactions more so than femalestudents. While we did not intend to focus on genderdifferences, we do present these results as evidence thatcertain students experience the social aspects of thistype of environment differently. In our case, major andethnicity did not contribute to these differences, but thatmay have been a result of our relatively low sample size incertain subgroups. The value of having examinedseveral measures of centrality is justified in our abilityto conclude that the types of interactions students experi-ence and with whom they have these interactions matterswith regard to self-efficacy formation. The characteristic ofPageRank as a measure of the kinds of people whomstudents interact with may also help to explain whyPageRank is a slightly better predictor of overall self-efficacy than inDegree or outDegree. Additionally, weknow from past studies that mastery experiences, a sourcewe found associated with PageRank, often plays a greaterrole in self-efficacy formation in physics courses than othersources [18,44].Our surprising results encourage us to think about waysto mitigate effects of the social structure of MI on students ’ efficacy beliefs and vice versa. This might manifest itselfthrough the purposeful stimulation of interactions betweencertain groups of students. Altering how students partici-pate in the social aspects of a classroom in a way that givesall an equitable chance then becomes, in part, an issue ofhow students recognize the value of their peers. We suspectthat the highly social nature of this learning approachexposes students to academic judgment from peers and caninitiate introspective evaluation, specifically while studentssolve problems in groups and when they present solutions to the larger classroom. The increased number of inter-action events may provide students with more opportunitiesto generate perceptions about their peers ’ ability to con-tribute to a physics-related task and, in turn, influencewhom they work with or whom they list when asked torecall meaningful academic interactions. These perceptionscan drive changes in interactions. Although in this examplewe have suggested that these changes may relate toacademic perceptions, they may also relate to students ’ ability to communicate effectively, helpfulness, or evenfriendliness.We faced certain limitations worth noting. No student inthe Fall 2014 and 2015 course had declared physics as asole major. Physics majors may be less susceptible tochanges in self-efficacy via peer recognition because oftheir strong physics identity relative to those pursuing otherSTEM fields [11]. The absence of physics majors in theseMI courses might also point to a possibly unidentifiedsource of self-selection bias. Furthermore, the MI class-rooms in question were among the first at FIU to host thatmany students at once. The novelty of implementing thiscurriculum with more students in a brand new classroommay have led to unrecognized shortcomings. Furtherinvestigation should take place to more clearly understandhow these factors relate to our study.Knowing the powerful role that introductory physicscourses play on career persistence and the underrepresen-tation of certain groups of students [7], we are pressed tosearch for ways to ensure that students complete thesemester feeling more confident in their ability to performphysics tasks rather than less confident — regardless of thegradient. Though we report a somewhat minor 3.79%overall drop in students ’ physics self-efficacy, this is anaverage measure. Individually, students ranged from a 29%decrease from prescore to a 46% increase from prescore.This variance offers a living example of how students in thecourse can exhibit contrary, affective outcomes. Our studyshowed that part of what accounted for these differencesare the kinds of interactions students had. Similar learningenvironments, particularly those that focus on active-learning mediated by student interactions, may exhibitparallel outcomes. Our holistic approach to student learningmotivates us to explore ways to improve MI and interactive-learning approaches in the introductory classroom such thatthe maximal number of students leave not just academicallyprepared, but also affectively equipped to persist in physicscareers. Our study aims to highlight the value of examiningthese facets of student outcomes in these environments,specifically self-efficacy development and course-relatedsocial interactions. It is not enough to simply say thatstudents are learning more. This is especially true in therealm of career decision-making where self-efficacy plays acentral role even for STEM related professions, partiallyexplaining the underrepresentation of certain groups in thesefields [73]. Bandura [29] explains,BEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
PHYS. REV. PHYS. EDUC. RES.12, with whom the interactionoccurs. Interactions coming from popular individuals asdefined by their inDegree positively predict a students ’ sense that they can learn and get good grades in physics.Because students did not know each other ’ s inDegree, wecan infer that students recognize, in some capacity, whothese popular individuals may be and have a perceptionabout their academic popularity. A highly social settingmay catalyze these peer-to-peer judgments.Active-learning environments, like MI, create the kind ofsocial space that allows students the flexibility to interact indifferent ways with different people [45]. Though norelationship was found between gender and self-efficacy,female and male students differ in the kinds of interactionsthey experience. Male students in this class are the subjectsof others ’ meaningful interactions more so than femalestudents. While we did not intend to focus on genderdifferences, we do present these results as evidence thatcertain students experience the social aspects of thistype of environment differently. In our case, major andethnicity did not contribute to these differences, but thatmay have been a result of our relatively low sample size incertain subgroups. The value of having examinedseveral measures of centrality is justified in our abilityto conclude that the types of interactions students experi-ence and with whom they have these interactions matterswith regard to self-efficacy formation. The characteristic ofPageRank as a measure of the kinds of people whomstudents interact with may also help to explain whyPageRank is a slightly better predictor of overall self-efficacy than inDegree or outDegree. Additionally, weknow from past studies that mastery experiences, a sourcewe found associated with PageRank, often plays a greaterrole in self-efficacy formation in physics courses than othersources [18,44].Our surprising results encourage us to think about waysto mitigate effects of the social structure of MI on students ’ efficacy beliefs and vice versa. This might manifest itselfthrough the purposeful stimulation of interactions betweencertain groups of students. Altering how students partici-pate in the social aspects of a classroom in a way that givesall an equitable chance then becomes, in part, an issue ofhow students recognize the value of their peers. We suspectthat the highly social nature of this learning approachexposes students to academic judgment from peers and caninitiate introspective evaluation, specifically while studentssolve problems in groups and when they present solutions to the larger classroom. The increased number of inter-action events may provide students with more opportunitiesto generate perceptions about their peers ’ ability to con-tribute to a physics-related task and, in turn, influencewhom they work with or whom they list when asked torecall meaningful academic interactions. These perceptionscan drive changes in interactions. Although in this examplewe have suggested that these changes may relate toacademic perceptions, they may also relate to students ’ ability to communicate effectively, helpfulness, or evenfriendliness.We faced certain limitations worth noting. No student inthe Fall 2014 and 2015 course had declared physics as asole major. Physics majors may be less susceptible tochanges in self-efficacy via peer recognition because oftheir strong physics identity relative to those pursuing otherSTEM fields [11]. The absence of physics majors in theseMI courses might also point to a possibly unidentifiedsource of self-selection bias. Furthermore, the MI class-rooms in question were among the first at FIU to host thatmany students at once. The novelty of implementing thiscurriculum with more students in a brand new classroommay have led to unrecognized shortcomings. Furtherinvestigation should take place to more clearly understandhow these factors relate to our study.Knowing the powerful role that introductory physicscourses play on career persistence and the underrepresen-tation of certain groups of students [7], we are pressed tosearch for ways to ensure that students complete thesemester feeling more confident in their ability to performphysics tasks rather than less confident — regardless of thegradient. Though we report a somewhat minor 3.79%overall drop in students ’ physics self-efficacy, this is anaverage measure. Individually, students ranged from a 29%decrease from prescore to a 46% increase from prescore.This variance offers a living example of how students in thecourse can exhibit contrary, affective outcomes. Our studyshowed that part of what accounted for these differencesare the kinds of interactions students had. Similar learningenvironments, particularly those that focus on active-learning mediated by student interactions, may exhibitparallel outcomes. Our holistic approach to student learningmotivates us to explore ways to improve MI and interactive-learning approaches in the introductory classroom such thatthe maximal number of students leave not just academicallyprepared, but also affectively equipped to persist in physicscareers. Our study aims to highlight the value of examiningthese facets of student outcomes in these environments,specifically self-efficacy development and course-relatedsocial interactions. It is not enough to simply say thatstudents are learning more. This is especially true in therealm of career decision-making where self-efficacy plays acentral role even for STEM related professions, partiallyexplaining the underrepresentation of certain groups in thesefields [73]. Bandura [29] explains,BEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12, … the stronger people ’ s belief in their efficacy, themore career options they consider possible, the greaterthe interest they show in them, the better they preparethemselves educationally for different occupations, andthe greater their staying power and success in difficultoccupational pursuits. ” Our exploration of this matter reflects our commitmentto not only help our students better understand physics,but also motivate some to join the physics community. This requires that we focus on more than just contentmatter.
ACKNOWLEDGMENTS
This work exists thanks in part to intellectual contribu-tions from the Physics Education Group at FloridaInternational University, as well as support from the FIUDivision of Research and the STEM TransformationInstitute at FIU. This study was made possible byNational Science Foundation Grant No. PHY 1344247. [1] S. Freeman, S. L. Eddy, M. McDonough, M. K. Smith, N.Okoroafor, H. Jordt, and M. P. Wenderoth, Investigatingstudent communities with network analysis of interactionsin a physics learning center, Proc. Natl. Acad. Sci. U.S.A. , 8410 (2014).[2] E. Etkina and A. Van Heuvelen, Investigative ScienceLearning Environment: Using the processes of science andcognitive strategies to learn physics, edited by K. Cummings, S. Franklin, and J. Marx(Rochester, NY, July 25 –
26, 2001).[3] J. Robert, M. Jeffery, J. Rhett, L. Duane, S. David, N.Carolina, D. L. Deardorff, and D. S. Abbott,
Introductionto SCALE-UP: Student-Centered Activities for LargeEnrollment University Physics (Fund for the Improvementof Postsecondary Education, Washington, DC, 2000).[4] P. W. Laws, Millikan lecture 1996: Promoting activelearning based on physics education research in introduc-tory physics courses, Am. J. Phys. , 14 (1997).[5] N. D. Finkelstein and S. J. Pollock, Replicating and under-standing successful innovations: Implementing tutorials inintroductory physics, Phys. Rev. ST Phys. Educ. Res. , 1(2005).[6] E. Brewe, Modeling theory applied: Modeling Instructionin introductory physics, Am. J. Phys. , 1155 (2008).[7] National Research Council, Adapting to a ChangingWorld — Challenges and Opportunities in UndergraduatePhysics Education (National Academies Press,Washington, DC, 2013).[8] President ’ s Council of Advisors on Science and Technol-ogy, Prepare and Inspire: K-12 Education in Science,Technology, Engineering, and Math (STEM) for America ’ ‑ stemed ‑ report.pdf.[9] National Research Council, Discipline-Based EducationResearch: Understanding and Improving Learning inUndergraduate Science and Engineering (NationalAcademies Press, Washington, DC, 2012).[10] D. H. Schunk,
Learning Theories: An EducationalPerspective (Pearson Education, Boston, 2012).[11] Z. Hazari, G. Sonnert, P. M. Sadler, and M.-C. Shanahan,Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice,J. Res. Sci. Teach. , 978 (2010).[12] R. W. Lent, S. D. Brown, and G. Hackett, Toward aunifying social cognitive theory of career and academicinterest, choice, and performance, J. Vocat. Behav. , 79(1994).[13] A. Bandura, C. Barbaranelli, G. V. Caprara, and C.Pastorelli, Self-efficacy beliefs as shapers of children ’ saspirations and career trajectories, Child Development ,187 (2001).[14] X. Wang, Why students choose STEM majors: Motivation,high school learning, and postsecondary context of sup-port, Am. Educ. Res. J. , 1081 (2013).[15] M. Kjærnsli and S. Lie, Students ’ Preference for ScienceCareers: International comparisons based on PISA 2006,Int. J. Sci. Educ. , 121 (2011).[16] National Research Council, Expanding UnderrepresentedMinority Participation: America ’ s Science and Technol-ogy Talent at the Crossroads (National Academies Press,Washington, DC, 2011).[17] National Science Board, Science and EngineeringIndicators 2014 (National Science Foundation, Arlington,VA, 2014).[18] A. L. Zeldin, S. L. Britner, and F. Pajares, A comparativestudy of the self-efficacy beliefs of successful men andwomen in mathematics, science, and technology careers,J. Res. Sci. Teach. , 1036 (2008).[19] K. C. Larkin, R. W. Lent, and S. D. Brown, Self-efficacy inthe prediction of academic performance and perceivedcareer options, J. Counsel. Psychol. , 265 (1986).[20] A. L. Zeldin and F. Pajares, Against the odds: Self-efficacy beliefs of women in mathematical, scientific,and technological careers, Am. Educ. Res. J. , 215(2000).[21] C. Hill, C. Corbett, and A. St. Rose, Why so Few? Womenin Science, Technology, Engineering and Mathematics (AAUW, Washington, DC, 2010).[22] R. Dou and K. D. Gibbs, Engaging all students in thepursuit of STEM careers, Sch. Sci. Rev. , 106 (2013).[23] S. D. Brown, R. W. Lent, and K. C. Larkin, Self-efficacy asa moderator of scholastic aptitude-academic performancerelationships, J. Vocat. Behav. , 64 (1989). REMY DOU et al.
PHYS. REV. PHYS. EDUC. RES.12,
PHYS. REV. PHYS. EDUC. RES.12,
24] R. W. Lent, F. G. Lopez, and K. J. Bieschke, Predictingmathematics-related choice and success behaviors: Test ofan expanded social cognitive model, J. Vocat. Behav. ,223 (1993).[25] R. W. Lent, S. D. Brown, and K. C. Larkin, Comparison ofthree theoretically derived variables in predicting careerand academic behavior: Self-efficacy, interest congruence,and consequence thinking, J. Counsel. Psychol. , 293(1987).[26] F. Pajares and T. C. Urdan, Adolescence and Education (Information Age, Charlotte, NC, 2006).[27] A. Bandura,
Self-Efficacy: The Exercise of Control (W. H. Freeman and Co., New York, 1997).[28] F. Pajares, Adv. Motiv. Achievement , 1 (1997).[29] A. Bandura, Perceived self-efficacy in cognitive develop-ment and functioning, Educ. Psychol. , 117 (1993).[30] A. Bandura, Self-efficacy: Toward a unifying theory ofbehavioral change, Psychol. Rev. , 191 (1977).[31] D. H. Schunk, Ability versus effort attributional feedback:Differential effects on self-efficacy and achievement,J. Educ. Psychol. , 848 (1983).[32] C. S. Dweck, The role of expectations and attributions inthe alleviation of learned helplessness, J. Pers. Soc.Psychol. , 674 (1975).[33] D. Hestenes, Toward a modeling theory of physics in-struction, Am. J. Phys. , 440 (1987).[34] D. Desbien, Ph.D. dissertation, Arizona State University,2002.[35] J. Bruun and E. Brewe, Talking and learning physics:Predicting future grades from network measures and ForceConcept Inventory pretest scores, Phys. Rev. ST Phys.Educ. Res. , 020109 (2013).[36] E. Brewe, V. Sawtelle, L. H. Kramer, G. E. O ’ Brien, I.Rodriguez, and P. Pamelá, Toward equity through partici-pation in Modeling Instruction in introductory universityphysics, Phys. Rev. ST Phys. Educ. Res. , 010106 (2010).[37] E. Brewe, A. Traxler, J. De La Garza, and L. H. Kramer,Extending positive CLASS results across multiple instruc-tors and multiple classes of Modeling Instruction, Phys.Rev. ST Phys. Educ. Res. , 020116 (2013).[38] J. Lynch, Douglas, Coll. Stud. J. , 920 (2010).[39] H. Fencl and K. Scheel, Engaging students: An examina-tion of the effects of teaching strategies on self-efficacy andcourse climate in a nonmajors physics course, J. Coll. Sci.Teach. , 20 (2005).[40] J. M. Smist, in Proceedings of the National Meeting of theAmerican Chemical Society (ACS, Chicago, 1993), pp. 1 – , 373 (2006).[42] P. Askar and D. Davenport, Turk. Online J. Educ. Technol.8 (2009).[43] C. Lindstrøm and M. D. Sharma, Int. J. Innov. Sci. Math.Educ. , 1 (2011).[44] V. Sawtelle, E. Brewe, and L. H. Kramer, Exploring therelationship between self-efficacy and retention in intro-ductory physics, J. Res. Sci. Teach. , 1096 (2012).[45] V. Sawtelle, E. Brewe, R. M. Goertzen, L. H. Kramer, N. S.Rebello, P. V. Engelhardt, and C. Singh, Creating oppor-tunities to influence self-efficacy through modeling in-struction, AIP Conf. Proc. , 339 (2012). [46] V. Sawtelle, E. Brewe, L. H. Kramer, C. Singh, M. Sabella,and S. Rebello, AIP Conf. Proc. , 289 (2010).[47] H. S. Fencl and K. R. Scheel, AIP Conf. Proc. , 173(2004).[48] J. Scott, Trend report: Social network analysis, Sociology , 109 (1988).[49] E. Brewe, L. Kramer, and V. Sawtelle, Investigating studentcommunities with network analysis of interactions in aphysics learning center, Phys. Rev. ST Phys. Educ. Res. ,010101 (2012).[50] J. Forsman, R. Moll, and C. Linder, Extending thetheoretical framing for physics education research: Anillustrative application of complexity science, Phys. Rev.ST Phys. Educ. Res. , 020122 (2014).[51] D. Z. Grunspan, B. L. Wiggins, and S. M. Goodreau,Understanding classrooms through social network analy-sis: A primer for social network analysis in educationresearch, Cell Biol. Educ. , 167 (2014).[52] S. L. Thomas, Ties that bind: A social network approachto understanding student integration and persistence,J. Higher Educ. , 591 (2000).[53] J. Scott and P. J. Carrington, The SAGE Handbook ofSocial Network Analysis , (SAGE, London, 2011).[54] S. Brin and L. Page,
Seventh Int. World Wide Web Conf. (Brisbane, Australia, 1998), pp. 107 – , 2343 (2007).[56] State University System of Florida, Annual AccountabilityReport (Miami, Florida, 2014), http://opir.fiu.edu/BOG/FIU_2014_Annual_Report.pdf.[57] Office of Governmental Relations, STEM Success byNumbers (Florida International University, Miami,Florida, 2014), http://government.fiu.edu/federal/stem/stem ‑ success ‑ numbers/index.html.[58] E. L. Usher and F. Pajares, Rev. Educ. Res. , 751 (2008).[59] V. Sawtelle, E. Brewe, and L. H. Kramer, in Proceedings ofthe Tenth Annual College of Education & GSN ResearchConference , edited by M. S. Plakhotnik, S. M. Nielsen, andD. M. Pane (Florida International University, Miami,2011), pp. 216 – þ years, J. Am.Stat. Assoc. , 473 (1996).[61] J. L. Schafer, Stat. Methods Med. Res. , 3 (1999).[62] J. Honaker, G. King, and M. Blackwell, Amelia II:Aprogram for missing data, J. Stat. Softw. , 1 (2011).[63] R Core, Team, R:A Language Environment For StatisticalComputing ‑ project.org.[64] J. L. Schafer and J. W. Graham, Missing data: Ourview of the state of the art, Psychol. Methods , 147(2002).[65] D. B. Rubin, Multiple Imputation for Nonresponse Surveys (J. Wiley & Sons, New York, 1987).[66] M. Bastian, S. Heymann, and M. Jacomy, in
Proceedingsof the Third International ICWSM Conference (San Jose,CA, 2009), pp. 361 – – BEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
Proceedingsof the Third International ICWSM Conference (San Jose,CA, 2009), pp. 361 – – BEYOND PERFORMANCE METRICS: EXAMINING … PHYS. REV. PHYS. EDUC. RES.12,
68] J. Fox, Bootstrapping regression models, in
AppliedRegression Analysis and Generalized Linear Models ,3rd ed. (SAGE Publications, Inc., Thousand Oaks, CA,2015), pp. 587 – , 129(2001).[70] B. Efron, J. Am. Stat. Assoc. , 171 (1987). [71] M. D. Siciliano, Am. Educ. Res. J. , 227(2016).[72] P. Bourdieu, The forms of capital, in Handbook ofResearch for the Sociology of Education (Greenwood,Westport, 1983), pp. 241 – (2009). REMY DOU et al.
PHYS. REV. PHYS. EDUC. RES.12,