Online Tutoring in Introductory Physics Courses: a Lockdown Experience
OOnline Tutoring in Introductory Physics Courses: a Lockdown Experience
Matteo Luca Ruggiero ∗ Politecnico di Torino, Corso Duca degli Abruzzi 24, Torino, Italy
Lorenzo Galante † DISAT & TLLAB, Politecnico di Torino, Corso Duca degli Abruzzi 24, Torino, ITINFN Torino, Via Pietro Giuria 1, Torino, IT andEnrico Fermi Historical Museum of Physics and Study and Research Centre, Piazza del Viminale, 1, Roma, IT (Dated: November 30, 2020)Social distancing due to the Covid-19 pandemic deeply impacted on education worldwide, sinceschools and universities had to rapidly organise lessons and courses on line. Traditional interactionsbetween teachers and students and, also among students, had to change and was substituted by online connections. In this context, laboratory work and tutoring, which have an important role in thepeer instruction model, needed to be redesigned. Here, we discuss an on line tutoring model adoptedfor the introductory physics courses at Turin Polytechinc University and evaluate its effectivenessby analysing the students performance both during the semester and the summer and autumn examsessions.
I. INTRODUCTION
The impact of the ongoing Covid-19 pandemic on ed-ucation is without precedents since, overnight, it deeplychanged teaching methods and organisations for billionsof students and teachers around the world. Schools anduniversities had (and still have) gigantic challenges toadapt their classes to distance learning as quickly as pos-sible, in order to keep educational continuity.Besides the obvious difficulties - both for teachers andstudents - due to the switch in a very short time to on-line courses, there was an additional complication for thelearning process, provoked by the atypical situation [1–4]. In fact, because of social distancing, interactions withteachers and peers considerably changed, and, de facto ,students were forced to experience independent studymuch more than in the past [5]. Accordingly, the role ofteachers and students became different from those in atraditional learning process: in this case teachers cannotdirectly interact with students and just act as facilitatorsto support them, while students have to independentlydevelop their collaborative efforts [6]. Fortunately, dueto the development of the Internet and related technolo-gies, today’s students have devices such as smartphones,tablets and personal computers available. These technol-goies make it possible and easy to access large volumesof information and to maintain contact with classmatesand educators. Undoubtedly, if this pandemic had spreadonly 30 years ago, its effect on education systems world-wide would have been devastating.In addition, activities such as laboratory work and tu-toring were greatly limited during the pandemic and theyhad to be re-designed in order to be effective in the learn-ing process. These activities have an important role in ∗ Electronic address: [email protected] † Electronic address: [email protected] the peer instruction model, whose effectiveness is wellknown, from early childhood education [7] up to college[8] and university level [9, 10]. In the literature peer col-laboration refers to laboratory work and peer tutoring toa tutoring model in which advanced students (“student-tutors”) or those in later years take on an instructionalrole (see e.g. Boud et al. [11], Collings et al. [12]), mainlyaimed at the revision of the taught content. Peer tutor-ing has several advantages, as reported by Sneddon [13]:for instance, students are freer to communicate with eachother in the absence of teachers; student-tutors have anapproach to the course materials and resources which ismore similar to the student one; the interaction betweenstudents and student-tutors is more direct and personalwhich can lead to a more lively and open learning en-vironment. Usually, in this model there are meetingstwo/three times per week, in which one or more student-tutors works with a small number (ranging from 50-100)of students. There is evidence that in this model boththe tutees and the tutors get benefits [13].Social distancing provoked by the pandemic acts as aserious deterrent for carrying out peer tutoring which istypically intended for weak and less experienced students.For these reasons it is necessary to completely redesignthis model of tutoring, and the same, of course, holdstrue for laboratory work [14, 15].In this paper we report the results of an on line tutoringmodel developed for the introductory physics courses atTurin Polytechinc University during the lockdown due tothe Covid-19 pandemic. The tutoring was based on on-line activities, both synchronous, such as video and chatsessions, and asynchronous, such as questionnaires andexercises. The former were conducted by student-tutorsand organised by staff members. These activities aimedto facilitate the students’ learning process and, also, tomimic the classroom interactions that were not allowed.In this work we aim to evaluate the effectiveness of thetutoring model by analysing the performances of the stu-dents involved, both during its progress and during their a r X i v : . [ phy s i c s . e d - ph ] N ov exams.The paper is organised as follows: in Section II wedescribe the tutoring model and the context, while inSection III we examine the methodologies and give theresults, which are then discussed in Section IV. Conclu-sions are eventually drawn in Section V. II. THE ONLINE TUTORING MODEL
Turin Polytechinc University (TPU) is one of the mostimportant Italian Technical Schools for Engineers andit attracts students not only from Italy, but also frommore than 100 countries around the world; about 45% ofstudents come from outside Piemonte, the region whereTurin is located, and 15% come from abroad. About 5000students start studying at TPU every year, and duringthe first year they are divided into 20 courses: 18 aretaught in Italian and 2 in English. The students in theItalian courses are divided by alphabetical order ratherthan ability. During the first year, students attend the”Physics I” course, which is an introductory calculus-based course including classical mechanics and thermo-dynamics, Newtonian gravitation and basic electrostaticsin which Coulomb’s and Newton’s law are compared. Ineach course, there are between 300 and 500 students: be-sides first year students, there are also students who didnot succeed in passing the exam from previous years. Asa consequence, each year there are approximately 10000students in the Physics I courses.The Physics I course is generally considered as diffi- cult and among those which most frequently contributesto dropping out of TPU. Consequently, TPU organisestutoring activities to help students, in the spirit of theLearning Assistants scheme, developed by the Univer-sity of Colorado [16], where experienced students (usu-ally 2 years older) are employed as tutors and, togetherwith course lecturers, they revise the key topics. Usually,student-tutors work in pairs, and meet small groups ofstudents once or twice per week. The interaction is directand immediate and mainly involves solving exercises.The Physics I course is taught during the secondsemester of the academic year, which usually starts be-tween the end of February and the beginning of March,so in 2020 overlapped with the outbreak of the Covid-19pandemic in Italy. Due to the emergency situation, allcourses were switched to online delivery at TPU and theusual tutoring in conditions of social distancing could nolonger take place; consequently, it was necessary to de-sign a new online tutoring model.The model we developed is based on a Moodle plat-form, integrated in the Internet TPU portal, which isaccessible to all Physics I students; 20 student-tutors(here and henceforth junior tutors ) were recruited andcoordinated by 4 TPU staff members (here and hence-forth senior tutors ), who were lecturers in the PhysicsI courses. The junior tutors were divided into 10 pairsand each pair was assigned to two courses. The activitiesdescribed here refer to the courses taught in Italian onlybecause, due to the very short time available to organisethe tutoring, it was not possible to involve the 2 coursestaught in English.
FIG. 1: Tutoring weekly plan.
The weekly activities began one month after the beginning of the courses and were organised as follows (see alsoFigure 1). (1) Senior tutors prepared a set of multiple choice questions on selected issues from the lectures; thesequestions were about important concepts or aimed to eliminate typical misconceptions. (2) Students were invited toanswer the weekly questionnaire during the weekend. (3) On the basis of the results, senior tutors prepared specificexercises in order to address common errors. (4)The exercises were discussed on the following Monday during an onlinemeeting between senior and junior tutors: the aim of the meeting was to inform junior tutors about the students’results in the previous questionnaire in order to outline the expected errors for the proposed exercises. (5) Eachtutoring group had two weekly sessions (scheduled on different days at different times in order to give all the studentsthe possibility to attend the class); in each session the same exercises were used. The interactions with the studentsduring the tutoring sessions took place in the chatroom of the videoconferencing system and, also, using dedicatedTelegram channels. Each session was divided into two parts. In the first part, students had time to go through theproposed exercises on their own, in the second part the exercises were solved by the junior tutors. All tutoring sessionswere recorded and, then, made available on the Moodle platform, together with the exercises and their solutions. (6)Students were requested to complete the same questionnaire (with the same questions) after each tutoring session inorder to measure the impact of the tutoring activities.The tutoring was organised over 8 weeks (plus 1 forrevision of mechanics) and the topics covered the wholecontent of the Physics I course, in the following sequence: • Week 1: Mathematical prerequisites • Week 2: Kinematics • Week 3: Dynamics • Week 4: Conservation laws and collisions • Week 5: Rigid body • Week 6: Gravitation and electrostatics • Revision of Mechanics • Week 7: Calorimetry and first law of thermody-namics • Week 8: Second law of thermodynamicsThe entire tutoring lasted for 9 weeks, with 18tutoring sessions per group. The whole set of activitieswas recorded both on the Moodle platform and on theTelegram channels: they are currently being analysedand will be discussed in a forthcoming paper [17]. Inthis work, we focus on two research questions that areimportant to evaluate the effectiveness of this tutoringmodel, both during its progress and in the final exams,namely:
Q1: As a result of the weekly activities performed andof the interactions with the junior tutors, is there animprovement in the results of the questionnaires?Q2: As a result of the participation in the tutoringactivities during the semester, is there an improvementin the marks among the students who attended the tu-toring, with respect to the average results of all students?
These research questions will be discussed in the fol-lowing Section.
III. METHODOLOGIES AND RESULTS
The average number of students who attended the tu-toring sessions every week, as registered by the junior tu-tors, was 385: this was the sum of the 9 tutoring groups,which means more or less 43 students per week per group.These data are aggregate, so the number of presences wasnot equally distributed in each group and the same stu-dents did not attend all sessions. Nonetheless these datagive an idea of the synchronous interactions which oc-curred during the tutoring. Interestingly, the number ofasynchronous interactions was far greater than the aver-age number students attending (385): in fact by the endof the summer exam session, we had registered about9000 students on the Moodle platform. Even though thetutoring ended at the beginning of June, students con-tinued to use this platform after this date for the videorecordings of the tutoring sessions, for the solutions ofthe problems proposed and for repeating the weekly ques-tionnaires.The following analysis refers to all data collected untilthe end of September, after the end of the summer andautumn exam sessions. Firstly, we analyse the weeklyquestionnaires, in order to track improvements in the stu-dents’ performance. Subsequently we compare the examresults of the students who attended the tutoring to thoseachieved by all students.
A. Weekly Questionnaires
The weekly questionnaires, one before the tutoring ses-sions, and one after the tutoring sessions, were completedon a voluntary basis. Only one attempt was allowed forboth questionnaires. In Figure 2 we report the number ofstudents who filled out both questionnaires in each week.This number is smaller than the number of students whoanswered each single questionnaire. For instance, duringthe first week 20% of students repeated the test after thetutoring session; in the subsequent weeks the percentagesare 32%, 38%, 36%, 34%, 22%, 29% and 23%.In Figure 3 we compare the percentages of correct an-swers of the pre and post tutoring questionnaires. Tomeasure the students’ performances after the tutoringsession, we calculated the Hake normalised gains (re-
FIG. 2: Number of students, per week, who carried out boththe pre and the post tutoring questionnaires.FIG. 3: Comparison between the percentage of correct an-swers in the pre and post tutoring questionnaires, togetherwith the corresponding gain (g). ported below each column bar in Figure 3) defined as[18] g = P f − P i − P i , where P i , P f are, respectively, the percentages of correctanswers before and after the tutoring sessions. In partic-ular, the gain is • high, if g ≥ . • medium, if 0 . ≥ g ≥ . • low, if g < . B. Final Exams
More specific data about the tutoring were obtainedby an online survey administered to all students on theMoodle platform: we collected 1209 answers. The surveycovered various areas, such as the numbers of sessionsattended, the most interesting topics for students, theperceived effectiveness of the methods and informationsabout the interactions with the junior tutors.
FIG. 4: Description of the sample analysed: number of ses-sions attended.
The number of attended sessions is shown in Figure4. We see that 403 students did not take part in anysession: they did not attend the synchronous activitiesbut they attended the asynchronous ones, such as watch-ing the video recordings, completing the questionnaires,downloading the problems and so on. Hence, in the fol-lowing analysis, the sample size is N = 1209 −
403 = 806students, given by the number of students who attendedat least one tutoring session. We examined their perfor-mances in the exams of June, July and September.In particular, we considered their performance in the first part of the exam (the preliminary test) which consistedof a quiz with 15 multiple choice questions. A minimum of 8 correct answers was needed to access the second part ofthe exam, which was an exercise to be solved on-line. A comparison between the results of our sample and those ofall students is carried out below. The (normalized) results of the statistical analysis are reported in Figure 5, whilethe details are in Table I.As we can see looking at the distributions in Figure 5 and Table I, the average mark in the preliminary multiplechoice test is higher for the sample of students who attended the tutoring sessions: this positive trend is evident inall exam sessions. In order to quantify the statistical significance of the difference between the average marks, wedecided to perform a T-test and to evaluate the p-value. In our case the the null hypothesis is: there is no statisticaldifference among the average marks of the two samples, their difference is due to a statistical fluctuation . Session Sample Quizzes Attempted Quizzes Passed Percentage of Quizzes Passed Average Mark
June
Tutored Students 742 607 81.81 10.02All Students 3522 2557 72.60 9.31
July
Tutored Students 411 354 86.13 10.04All Students 2636 1908 72.38 8.99
September
Tutored Students 195 157 80.51 9.72All Students 1870 1325 70.86 9.02TABLE I: Details of June, July and September exams.FIG. 5: Comparison of the students results in June, July and September Physics 1 exams.
The T-test was carried out on the basis of the statis-tical parameters reported in Table II. In all cases (June,July and September sessions) the resulting p-value issmaller than 0.001, meaning that we have a confidencelevel bigger that 99.9% to reject the null hypothesis. Inother words, the different average marks of the two sam-ples are not due to statistical fluctuations.From the data in table II, we see that the marks of thestudents from tutoring are, on average, 0.8 points higher.It is important to emphasise that, even if this result mightseem to be negligible, it has significantly decreased thenumber of student failures (grade <
8) in the Physics Icourse. By comparing the percentages of students witha grade lower than 8, we can evaluate the improvementin the rate of successful students (see Figure 6). In June the percentage of Tutored students not passing the testwas lowered by 9%, in July by 14%, in September by10%. If we apply the average 11% improvement to thenumber of students attending the Physics I courses eachyear ( ∼ ∼ IV. DISCUSSION
Despite the low completion rate for the questionnaires,a small improvement in the student marks was evident(see Section III A). The average weekly attendance was385 students and in the best case (Week 2), only 142 stu-
Session Sample Variance Standard Deviation Average Mark Sample Size
June
Tutored Students 7.43 2.73 10.02 742All Students 8.56 2.92 9.31 3522
July
Tutored Students 5.33 2.31 10.04 411All Students 7.27 2.69 8.99 2635
September
Tutored Students 6.58 2.57 9.72 195All Students 7.49 2.74 9.02 1870TABLE II: Statistical parameters of the two samples. dents (37 %) filled out both the pre and post question-naires. Moreover, if we exclude the Week 1 questionnairewhich focused on mathematical prerequisites, attendancedecreased significantly between Week 2 and Week 6, andthen increased gradually over the last two weeks.This variation in attendance could be due to a possiblecorrelation between the increasing difficulty of the topicsand the time required for study. Therefore, students mayhave been unable to complete the post questionnaire indue time. Furthermore, the growth in attendance for thelast two weeks could have been provoked by the approachof the exam session.If we consider the results summarised in Figure 3, ac-cording to the Hake scale, the gain is low even if thereis an improvement. As reported by Nissen et al. [19]“Hake adopted this standardizing coefficient because itaccounted for the smaller shift in means that could oc-cur in courses with higher pretest means”. On the otherhand, this coefficient tends to show a low student im-provement when the pretest means are low. Further-more, if we look at Figure 3, we see that in our casethe mean pretest results are around 50 %. However, ifpretest means are around 100%, then improvements arealso high.We obtained better results for the second researchquestion, as reported in Section III B. In Figure 5 wesee that in all exam sessions the distribution of marksfor students who attended the tutoring were higher thanaverage. Moreover, Table I shows that the percentage ofstudents who passed the tests is noticeably greater in thesample of students who attended the tutoring sessions.The T-test analysis suggests that the improvements werenot due to statistical fluctuations.Obviously, it is not possible to attribute these positiveresults entirely to the tutoring intervention. However,we suppose that students who took part in the tutoringactivities were more diligent than the others. Indeed, tu-toring gave them continuous practice and revision of thephysics concepts, in addition to what they learned dur-ing the lectures. It is widely accepted that time-on-taskdeeply influences learning outcomes [20, 21]. However,tutoring can also be considered as important, since itgives all students the possibility to revise difficult top-ics and to reach the required knowledge level for examsuccess.
V. CONCLUSIONS
Due to the lockdown provoked by the Covid-19 pan-demic, schools and universities around the world madehuge efforts to adopt teaching methods based on distancelearning. Social distancing impacted also on laboratorywork and tutoring, which have an important role in thepeer instruction model. In particular, tutoring modelsinvolving advanced students who meet small groups ofstudents, needed to be redesigned, to cope with the newconditions determined by the pandemic. We have dis-cussed the on line tutoring model developed for the intro-ductory physics courses at Turin Polytechinc University,and analysed its impact on the student learning process,both during the tutoring activities and, at the end, dur-ing the exams. In this paper, we have focused on tworesearch questions to evaluate the effectiveness of the tu-tor activities, during the semester, by analysing the re-sults of weekly questionnaires (Q1), and at the end ofthe semester, during the exams in the summer and au-tumn sessions, by comparing the results of the studentswho attended the tutoring with the results of all students(Q2).The results for Q1 showed that the improvement in thestudent performances was low, as measured by the Hakescale. However, it is important to emphasise that onlya small number of students involved in the tutoring ac-tivities completed both the weekly pre and post tutoringquestionnaires. We suggest that, due to the increasingdifficulty in the content of the lectures, students neededmore time to study, so they were unable to completethequestionnaires in time.The data collected during the June, July and Septem-ber exam sessions showed that, in the preliminary tests,the distribution of marks for the students who attendedthe tutoring were better than the overall results; we per-formed a T-test analysis and showed that the improve-ment was not due to statistical fluctuations. We alsosuggest that the time-on-task, due to the activities per-formed during the tutoring, contributed to this improve-ment. Moreover we estimated a 10% increase in the num-ber of successful students in the preliminary tests due tothe tutoring activities.In conclusion, even though we know that furtherimprovements are needed, for instance to stimulate a
FIG. 6: Comparison among the total percentage of rejectedstudents in the preliminary test. In all exams sessions, thesample of students from Tutoring shows a lowered percentage. greater participation, we believe that it was important togive the students the possibility of experiencing the peereducation model through the tutoring activities, duringthe lockdown. Moreover, the improvement in the stu-dents’ performances were encouraging and suggests theeffectiveness of this tutoring model. [1] D. J. O’Brien, arXiv preprint arXiv:2008.07441 (2020).[2] D. Y. Tan and J.-M. Chen, arXiv preprintarXiv:2009.02705 (2020).[3] M. Dew, J. Perry, L. Ford, D. Nodurft, and T. Erukhi-mova, arXiv preprint arXiv:2009.11393 (2020).[4] P. Klein, L. Ivanjek, M. N. Dahlkemper, K. Jeliˇci´c, M.-A. Geyer, S. K¨uchemann, and A. Susac, arXiv preprintarXiv:2010.05622 (2020).[5] M. G. Moore, The Journal of Higher Education , 661(1973).[6] N. Dietrich, K. Kentheswaran, A. Ahmadi, J. Tey-chen´e, Y. Bessi`ere, S. Alfenore, S. Laborie, D. Bastoul, K. Loubi`ere, C. Guigui, et al., Journal of Chemical Edu-cation (2020).[7] W. Damon, Journal of applied developmental psychology , 331 (1984).[8] N. Lasry, E. Mazur, and J. Watkins, American journalof Physics , 1066 (2008).[9] E. Mazur, Peer Instruction: Pearson New Interna-tional Edition: A User’s Manual , Always learning (Pear-son Higher Education & Professional Group, 2013),ISBN 9781292039701, URL https://books.google.it/books?id=OReioAEACAAJ .[10] C. H. Crouch and E. Mazur, American journal of physics , 970 (2001).[11] D. Boud, R. Cohen, and J. Sampson, Peer learning inhigher education: Learning from and with each other (Routledge, 2014).[12] R. Collings, V. Swanson, and R. Watkins, Higher Edu-cation , 927 (2014).[13] P. H. Sneddon, Journal of Learning Development inHigher Education ISSN p. 667X (1759).[14] F. Pols, The Electronic Journal for Research in Science& Mathematics Education , 172 (2020).[15] F. Bradbury and C. Pols, arXiv preprint arXiv:2006.06881 (2020).[16] V. Otero, S. Pollock, and N. Finkelstein, American Jour-nal of Physics , 1218 (2010).[17] L. Galante and M. L. Ruggiero, in preparation (2020).[18] R. R. Hake, American journal of Physics , 64 (1998).[19] J. M. Nissen, R. M. Talbot, A. N. Thompson, andB. Van Dusen, Physical Review Physics Education Re-search , 010115 (2018).[20] J. B. Carroll, Teachers college record (1963).[21] B. S. Bloom, American psychologist29