Determination of the helpfulness of physics exam study methods
AAbstract
Studying for physics exams can be difficult and stressful, especiallyduring a student’s introductory year in physics. For students who donot plan to major in physics, the desire to do well is based less onunderstanding concepts and more on achieving a better grade. Forthis reason, students want to study as efficiently as possible by usingthe most optimal study methods. We have taken surveys over the pastthree years to determine how students study for exams and comparedthat to their exam grades. We found that students who studied usingmethods that they rated as more helpful did better on the exams.By utilizing the study results, we are able to present our current andfuture students with study methods that have been rated as being morehelpful, and give them advice on ways to optimize their study time forexams. a r X i v : . [ phy s i c s . e d - ph ] D ec etermination of the helpfulness of physics examstudy methods Rahul Jilakara and David P. WatersDepartment of Basic Sciences, University of Health Sciences and PharmacySt. Louis, Missouri 63110, USA.December 22, 2020
As we prepare this study, we recognize that the information gathered wasbased on self-report and Likert-type instruments. Self-report means that astudent provides their own ratings instead of using an independent deter-mination, and these have both advantages and disadvantages [1, 2, 3]. Theadvantages of self-reports are that we do not interrupt or influence someoneduring their process, and it allows us to collect a lot of data relatively easily.Unfortunately, students may not be able to accurately recall everything, ormay be biased in their answers [4]. Althubaiti et al. have looked at the in-formation bias in health research using self-reporting, and have found thatrecognizing these biases can help to lessen the effects [5].Study habits are important for college students, and not all studentsknow how to study in an optimal manner [6]. For example, research hasbeen conducted on how best to study for exams in medical school [7], whichlooked at the testing effect, active recall, and spaced repetition, which areall examples of study skills. In our study, we will be looking at study meth-ods that come from preparing for physics examinations during introductoryalgebra-based physics courses. We define study methods as the materialsthat students use to study for exams.It would be ideal if there were a set of universally helpful study methodsthat we could provide to our students. Unfortunately, research has shownthat different students may use this information more or less successfully.For example, high-achieving students have been found to better prepare forexams by using study methods that are more helpful [8, 9, 10, 11] whilelow-achieving students used the order of materials to determine what they2ould need to study [12]. An interesting finding is that some low-achievingstudents would recommend more helpful study strategies - a combination ofstudy skills and study methods - to others, but then would use less effectivestudy strategies for themselves [12]. These students often would not usemore effective strategies because of a lack of time or motivation [13]. Thesefindings show why a student would continue to use a study method thatthey continuously rated as less helpful. It is also possible that a student’sability to recognize whether something was helpful could be biased [14].Previous work on study strategies for exam preparation has found thatproviding information about how to study can be more useful than providingthe exact questions on the exam [14]. The authors found that distributingefforts across many study methods can help in preparing for an exam. Also,they found that knowing the format of the exam can be helpful for students.For this reason, our physics courses provide last year’s exam to students, inorder to both show the format of the exam and give students material toprepare for the exam. Fakcharoenphol et al. find that studying using oldexam practice problems can help, although learning may be shallow [15].Since our exams come right after studying, we do not know if the learn-ing that brought about higher exam scores was salient. Previous researchalso determined that studying on their own using worked-out solutions –old exams and class slides (problems) – and targeted exercises such as thereview problems allowed students to perform better than they did when us-ing traditional study methods, such as rereading the textbook [16]. Finally,completing web-based physics homework problems can lead to higher over-all exam performance [17], so using these homework problems to study mayalso help students improve their exam scores.Although many study methods can improve a student’s exam scores,some study methods have been found to be less helpful. Although stu-dents believed that reading the textbook in order to study for an examimproved their understanding, doing so had no effect on their exam perfor-mance because it gave them a false sense of understanding [18]. This findingis interesting because many studies that look at exam preparation only havetextbook readings as their study method [19]. Pressley et al. determinesstrategies to use textbook reading successfully, which may explain why someof our students label textbook reading as helpful.It has been found that students must adapt their study method to thetype of task that they are expected to complete [20], so a study methodthat results in better exam grades for most students may not be as helpfulin another environment. It may be difficult to make a list of helpful studymethods from our physics courses and implement them in other environ-3ents. One would expect that some study methods would be more helpfulthan others. Some students are going to learn better from certain studymethods while others will not learn as well using the same study methods[19]. This may explain why study methods in our courses work better forsome students than for others. Assuming that our students’ self-reporting isaccurate, it may be that students use the study methods differently or thatthey begin the preparation with different levels of understanding, and thuswill need to use different methods.
The survey was given to undergraduate students, mostly sophomores andjuniors, at the University of Health Science and Pharmacy in St. Louis.The courses in the fall and spring were algebra-based introductory physicscourses. These courses were required for health science majors who weremost often going to enter the pharmacy program or take a health scienceprofessional exam such as the MCAT. The fall and spring courses each hada typical enrollment of 60-80 students.
After each exam, students were asked to fill out a survey. These surveys wereadministered using the course’s LMS (Moodle). The surveys were availableto all students, but participation was voluntary.The survey questions are listed below: • Q1 : Approximately how many hours did you study for the exam? • Q2 : What percentage of time did you use the following materials tostudy for the exam? (10%, 20%, etc. and N/A) • Q3 : How helpful were these material types for the exam? (Scale: 1 =least helpful and 5 = most helpful)Any student who did not use a certain study method answered N/A onQ2 and Q3 for that study method. Some students did not choose N/A forQ3 for a study method that they did not use, so we changed that Q3 valueto an N/A for them.A list of the study methods available along with an explanation of eachwas given: 4 Rereading Textbook - The course uses OpenStax College PhysicsTextbook [21]. Sections of the textbook are given as a smaller PDFbefore each class. The assumption is that students read the textbookbefore class, so this method would involve reading the textbook againbefore the exam. • Class Slides (Conceptual) - During each class, students are askedmultiple choice questions and use a clicker to input the answer. Thesequestions are conceptual in nature, meaning that there usually is notany math needed to answer them. They are similar to the multiplechoice questions on the exam. • Class Slides (Problems) - Each week, after the topics have beenintroduced and discussed, students are given a day to work only onmathematical problems in class. These problems are similar to themathematical problems that students will see on the exam. Worked-out solutions are provided. • Review Questions - Review questions are posted for the students towork on. These involve multiple choice questions that may be eitherconceptual or mathematical in nature. These are all new questionsthat have not been seen before. The answers are provided, but thesolutions are not. For AY 2017-2018, the questions were on our LMS,but the number of questions was limited. For AY 2018-2019, thesequestions were posted externally with more questions added. In AY2019-2020, these questions were posted on the course’s LMS page withmany more questions available. • Old Exam - The previous year’s exam is provided as a study method,both to give insight about the format of the exam as well as to providemore practice opportunities. The answers are provided. • Homework Problems - Homework assignments involve solving math-ematical problems similar to those seen in class and those on the exam. • Reading quizzes - Before each class, students are asked a series ofquestions to see if they have read the textbook and/or watched thevideo. These are simple questions to check for a basic understandingof the pre-class material. 5 .3 Course Exams
The course exams were made up of conceptual (multiple-choice) questionsand mathematical problems. These were paper exams on which studentswould write out their work. For the multiple choice questions, students couldwrite out their work for partial credit. The mathematical problems alsoallowed for partial credit, with the bulk of the points awarded for showinghow to solve the problem. All exam scores included in the data are afterpartial credit was given. Exams usually had about 10 MC questions worth3-4 points each and 3-4 problems with individual parts that made up morethan half of the grade. For our analysis, we include the overall grade for theexam as well as scores for the conceptual section and scores for the problemssection. This separation allowed us to compare study methods that focusedon concepts versus study methods that required mathematical solutions.
The collected data from the student-filled surveys was divided and analyzedaccording to the semester it was collected. For each semester, we determinedthe average amount of time that a student studied for each particular examfrom Q1. Likewise, using the data from Q2, we averaged the percent of timethat each student spent utilizing each of the seven study habits per examover the course of the semester.Next, we took the percent of time spent on each study method from eachsemester and averaged it all together to produce the percent of time spenton each study method for the overall collected data. The percent of timespent per each study method was then multiplied by the average amountof time spent studying (Q1*Q2) to produce the average time spent on eachstudy method for the overall data.
To determine the overall effectiveness and use of each study method, wecompared the different graphs produced. As seen in Figure 1, on average,students spent a majority percentage of time studying using class slides(problems), old exams, and class slides (conceptual). The average amountof time that students spent rereading the textbook was surprisingly small.The graph for average percentage of time studying (Figure 1) was pro-duced with the averaged data. Although we calculated the total time spent6igure 1: Average percentage of time spent studying. The average percent-age of time that students spent on each study method. Out of the 7 studymethods, students spent the least percentage of their overall study time onrereading the textbook and reading quizzes. The most popular study meth-ods were class slides (problems), class slides (conceptual), and old exams.7y multiplying the total time that each student spent studying by the per-centage that each study method was used (Q1*Q2), the graph looked nearlyidentical to Figure 1, so we did not include the graph for the total timespent on each study method. Also, when we discuss comparisons, we findthat the correlation results are similar for both the percentage of time andthe total time spent on each study method.A lingering question throughout this study is whether, if students knowthey will be asked about their study methods (which they realize after thefirst exam in the fall semester), this has an effect on how they prepare forsubsequent exams.
The perceived helpfulness of each study method per exam per semester fromQ3 was averaged. This was done in a similar method to how the averagetotal amount of time that each student spent studying for each exam wascalculated. Then, the helpfulness of each study method for every semesterwas averaged to find the overall average perceived helpfulness for each studymethod. We also took the average of the helpfulness rating for all of thestudy methods over all of the exams throughout all of the semesters. Thisprocess gave us an overall average helpfulness rating.We used grouping of study methods to compare how students could havestudied. Study methods that were more helpful were grouped together, aswere study methods that were less helpful. During data collection, onlythose students who marked that they had used a certain study method inQ2 also recorded the helpfulness of that study method in Q3. Studentswho did not use a study method did not include a helpfulness rating. Thismeans that if a study method had a low helpfulness rating, that was becausethe students who used that method had rated it poorly. We are assuming,although we have no evidence, that those who did not use a study methodchose to ignore this option because they decided it would not have beenhelpful, and thus rarely used study methods probably would have had aneven lower helpfulness rating if all respondents had been asked to rate thehelpfulness of all study methods.We used three different ways to determine whether a study method washelpful or unhelpful. The first was to look at how students individuallyrated the helpfulness of study methods. The second was to look at howstudents rated the helpfulness of study methods each semester on average.8he third way was to create groups of universally helpful and unhelpfulstudy methods.For individual helpfulness (individual), we compare how a student ratedeach study method for that exam to the overall average helpfulness rating.A study method is considered helpful to that student if they rated the studymethod as higher than the overall average, and vice versa. In this case,we cannot create separate groups for study methods that are helpful vs.unhelpful because each student had a different interpretation of which studymethods were helpful for them.For average helpfulness (average), if a study method for a specific exam ina specific semester received an average helpfulness that was greater than theoverall average, then that study method was determined to be helpful andwas placed in the helpfulness group. Similarly, the unhelpful study methodsreceived less than the overall average helpfulness rating. In this case, thehelpful and unhelpful study methods were the same for all students.For helpfulness groups (group), any study method that was more helpfulthan this overall helpfulness rating after averaging all of the semesters to-gether was labeled helpful, and any method that had a lower-than-averagehelpfulness was labeled unhelpful.
To determine the overall average, we took all of the helpfulness ratings forevery study method used by every student for every exam in every semesterand averaged all of these to obtain an overall helpfulness rating. The overallhelpfulness rating (OHR) was 3.44 out of a total of 5. Any study methodthat was rated higher than this value was determined to be helpful for thatstudent (individual) or during that exam (average) or overall (group). Inorder to recognize if a study method was helpful or unhelpful overall, we canlook at Figure 2 to determine the overall helpfulness of each study method,which uses the averages from Q3. The horizontal black line represents theoverall helpfulness rating. We can see that class slides (both types), theold exams, and homework problems were all higher than the OHR. Thesestudy methods are considered helpful to most students and, are placed inthe helpful group in this paper. On the other hand, rereading the textbookand reading quizzes had an overall helpfulness that was lower than the OHR,are considered less helpful to most students, and are placed in the unhelpfulgroup.To get an idea of how the study methods’ helpfulness changes throughouteach semester, we can look at Figure 3, in which the data was made into9igure 2: Study Method Helpfulness. The helpfulness rating for each studymethod averaged over every exam in every semester. The horizontal blackline represents the overall helpfulness rating. Of the 7 study methods, fourare clearly above the overall helpfulness rating of 3.44 and two are clearlybelow. This demarcation allows us to designate these study methods intohelpful and unhelpful groups. Review questions is at the overall helpfulnessrating and is not included into either designation.10igure 3: Study Method Helpfulness for semester. Each series indicateshow a study method changed throughout the 6 semesters. To determinethese values, all of the helpfulness ratings for each student and each examthroughout the semester are averaged together.a line graph to help demonstrate the different helpfulness for each studymethod throughout each semester. Once again, a horizontal black line isprovided as a guide to represent the OHR. We can see that for the mostpart, a study method that starts off as helpful remains that way. Theonly study method that does not follow this rule is review questions. Thiscould be because many more questions were added to the review questionseach year and improvements were made to these questions. Because reviewquestions shifted throughout the semesters, as seen in Figure 3, it cannotbe designated as always helpful or unhelpful.The Average Helpfulness graphs, both the bar graph (Figure 2) as wellas the line graph (Figure 3), indicate the perceived helpfulness of each studymethod. Class slides (conceptual) and class slides (problems) seemed to bedeemed the most helpful when preparing for an exam. Both of these studymethods received the highest rating of all the different study methods; 4 outof 5. Those that used rereading textbook as a study method did not findthat study method to be as helpful to their performance on an exam com-pared to the other study methods. This observation can be strengthened by11igure 3, which indicates a downward trend of ratings of helpfulness fromthe first semester that this survey was administered to its last. Compar-ing the average time that students spent on each study method as well asthe students’ perception of the helpfulness of each study method helped usunderstand the potential impact of each study method.Comparing Figure 1 with Figure 2, we find that students spend moretime using study methods that most of the class believed are helpful. Per-haps unsurprisingly, the most helpful study methods are ones that involvequestions and problems that are similar to exam questions and problems.In the next section, we compare the effect that relying more on the helpfulstudy methods had on exam grades.
To determine how the use of each study method correlated with exam grades,we wrote a program in R that calculated the correlation value and the signif-icance of this correlation. We used the rcorr function in R using the Pearsoncorrelation method. The inputs were the groups of study methods and theexam grades.The three different helpfulness categories were the individual helpful andunhelpful study methods (individual), the average helpfulness (average), andthe grouping of study methods into helpful and unhelpful (group). Thesegroupings allowed us to compare the grades of students who spent more oftheir study time using the 4 helpful study methods and those who reliedmore on the two unhelpful study methods. As a reminder, the groupingswere as follows: • Helpful Study Methods – Class Slides (Conceptual) – Class Slides (Problems) – Old Exam – Homework Problems • Unhelpful Study Methods – Rereading Textbook – Reading Quizzes 12he study method labeled as helpful for one student (individual) or oneexam (average) may not be labeled as helpful every time. Figure 3 offers anoverview of the helpfulness of each study method in a specific semester. Forthe groupings, the same study methods are labeled helpful or unhelpful forall exams.We broke the exam grades into three categories. First, we used theoverall exam averages. Because the exams were split between mathematicalproblems and conceptual questions, we further broke the exam grades outinto problems (2nd category) and conceptual (3rd category).All reported correlations have a significance of p¡0.05. If a value fromthe correlation table had a p-value of greater than or equal to 0.05, we didnot include that value in our results. If a value included a significance ofmore than one star, then 2 stars represents p¡0.01 and 3 stars representsp¡0.001. Since there were 21 exams during the 6 semesters in which datawas collected, we will be discussing the correlations that show up in manyof the exams.
To determine if there was a correlation, we did not look at only one examor only one semester, we looked at every exam from every semester. Forthe individual helpfulness rating, significant correlations showed up in 13 ofthe 21 exams. For the average helpfulness, significant correlations showedup in 7 of the exams. For the groupings, significant correlations showed upin 11 of the exams. We concluded from this that there is strong evidencethat individual helpfulness correlates with exam grades, there is evidencethat there is a correlation between helpfulness groups and exam grades,and there is weaker evidence that there is a correlation between averagehelpfulness and exam grades.An example of correlation data that shows this relationship is shown inFigure 4. These tables represent only 2 of the 21 exams, but they are in-dicative of the strong correlations that we find between using helpful studymethods and exam grades. We see in Exam 1 that there is a significant posi-tive correlation for individual helpfulness (individual) across all overall examgrades and exam grades from problems. These correlations indicate thatwhen a student spent a greater percentage of their time on study methodsthat they labeled as more helpful than the OHR, the student’s grades werehigher. Exam 1 also shows a significant negative correlation between spend-ing more time on individually unhelpful study methods and exam grades,which means students’ exam grades were lower after using less helpful study13igure 4: Correlation table example from the first exam in the fall of 2019and fourth exam in the fall of 2017 showing that there were significant cor-relations between individual helpfulness and average helpfulness ratings andexam grades. Stars represent the level of significance and blank spaces rep-resent correlations that were not significant. Exam grades are broken downfurther into the grades for the mathematical problems and the conceptualquestions.methods. We see in Exam 4 that there are significant correlations for aver-age helpfulness (average) and helpfulness groupings (group) for both overallexam grades and the grades for mathematical problems. We did not includeconceptual grades in these examples because there were fewer correlations.exams by spending more time on study methods that they rated asmore helpful and less time on study methods that they rated as unhelpful(individual) had higher grades on their exams. Students who spent moretime on study methods that they labeled as unhelpful and less time on studymethods that they labeled as helpful had lower exam grades.A similar result is found for students who studied for exams by spendingmore time on helpful study methods (group). An example of these correla-tions is shown in Exam 4 of Figure 4. Exam 4 is only a single example, butit is indicative of the evidence that we find. We see that there is a signifi-cant positive correlation between using study methods that are labeled onaverage as more helpful and the overall exam grades and the grades fromproblems. We did not see as many correlations between using helpful studymethods and the grades on the conceptual parts of the exam, so this wasnot included in the table. 14lthough weaker, there is some evidence that students who spend moretime on study methods labeled as helpful for that exam (average) earnedhigher exam grades. Examples of these correlations are shown in Exams 1and 4 of the table in Figure 4. Because this is weaker, it is possible thatstudy methods may be rated differently for each exam, despite their overallhelpfulness not changing much.By combining all three methods (individual, average, and group), wefind significant correlations between studying using helpful study methodsand exam grades on all but 6 of the 21 exams. These exams are spreadover fall and spring and over each of the exams, so there is not anythingspecific that we can point to explaining why some exams did not show anysignificant correlations.
We find a positive correlation between the time spent studying using studymethods that an individual student labeled as helpful (individual) and higherexam grades. We also find a positive correlation between the time spent us-ing helpful study methods (group) and higher exam grades. When studentsstudy for exams using more helpful study methods, they end up doing betteron their exams. Unfortunately, the definition of what makes a study methodhelpful for each student is difficult to determine. The helpful group suggeststhat we could provide future students with information about the helpfuland unhelpful study methods. We could imagine that providing a sugges-tion of what material will be most helpful in studying for exams might helpstudents to improve their exam scores, especially those who may not be asgood at differentiating which study methods are helpful. On the other hand,the evidence from individual helpfulness suggests that students who studywhat they determine to be helpful have higher grades, and vice versa. Forthis reason, providing the average helpfulness ratings could actually hindera student who finds something helpful that others did not.After finding evidence that students who study using more helpful studymethods do better on their exams, we wonder about the causal relationship.Is this due to the fact that some students are better at choosing helpfulstudy methods? Are they better at recognizing what was most helpful forthem or did the study methods themselves help the students? Answeringthese question is the key to truly understanding what these results meanand what to do with them. For this reason, further research will be needed.One possibility would be to provide information about the helpful group of15tudy method (group) to one group of students and not to a control group.We could track whether having this information had an effect on their choiceof study methods and their exam grades. While we were hoping to comeaway from this project with a definitive list of helpful study methods todisseminate to current and future students, we recognize that more researchis needed.
References [1] G. Schellings, B. van Hout-Wolters, Measuring strategy use with self-report instruments: Theoretical and empirical considerations. Metacog-nition and Learning. , 83–90 (2011).[2] B. Van Hout-Wolters, G. Schellings, Measuring learning strategies. Kindsof measurement methods and their usefulness in educational research andpractice. Pedagogische Studien. , 110–129 (2009).[3] Veenman, M. V. J. (2011). Learning to self-monitor and self-regulate. InR. Mayer and P. Alexander (Eds.),Handbook of research on learning andinstruction (pp. 197–218). New York: Routledge.[4] Veenman, M. V. J. (2005). The assessment of metacognitive skills: Whatcan be learned from multimethod designs? In C. Artelt and B. Moschner(Eds.), Lernstrategien und Metakognition: Implikationen f¨urForschungund Praxis (pp. 75–97). Berlin: Waxmann.[5] A. Althubaiti, Information bias in health research: Definition, pit-falls, and adjustment methods. Journal of Multidisciplinary Healthcare. (2016), pp. 211–217.[6] S. Kumar, Study Habits Of Undergraduate Students. International Jour-nal of Education and Information Studies. , 17–24 (2015).[7] M. Augustin, How to learn effectively in medical school: Test your-self, learn actively, and repeat in intervals. Yale Journal of Biology andMedicine. , 207–212 (2014).[8] A. Kitsantas, Test preparation and performance: A self-regulatory anal-ysis. Journal of Experimental Education. , 101–113 (2002).[9] Pintrich, P. R., and Schunk, D. H. (2002). Motivation in education:Theory, research, and application. Upper Saddle River, NJ: Prentice-Hall.1610] D. L. Sundre, A. Kitsantas, An exploration of the psychology of the ex-aminee: Can examinee self-regulation and test-talking motivation predictconsequential and non-consequential test performance? ContemporaryEducational Psychology. , 6–26 (2004).[11] C. VanZile-Tamsen, J. A. Livingston, The differential impact of moti-vation on the self-regulated strategy use of high- and low-achieving collegestudents. Journal of College Student Development. , 54–60 (1999).[12] J. P. Holschuh, Do as i say, Not as i do: High, Average, and Low-performing students’ Strategy use in biology. Journal of College Readingand Learning. , 94–108 (2000).[13] J. E. Barnett, Self-regulated reading and test preparation among collegestudents. Journal of College Reading and Learning. , 42–53 (2000).[14] S. Van Etten, G. Freebern, M. Pressley, College students’ beliefs aboutexam preparation. Contemporary Educational Psychology. , 192–212(1997).[15] W. Fakcharoenphol, E. Potter, T. Stelzer, What students learn whenstudying physics practice exam problems. Physical Review Special Topics- Physics Education Research. (2011).[16] W. Fakcharoenphol, T. Stelzer, Physics exam preparation: A compari-son of three methods. Physical Review Special Topics - Physics EducationResearch. (2014).[17] R. Dufresne, J. Mestre, D. M. Hart, K. A. Rath, The Effect of Web-Based Homework on Test Performance in Large Enrollment IntroductoryPhysics Courses. Jl. of Computers in Mathematics and Science Teaching. , 229–251 (2002).[18] T. Linderholm, A. Wilde, College students’ beliefs about comprehen-sion when reading for different purposes. Journal of College Reading andLearning. , 7–19 (2010).[19] M. Pressley, L. Yokoi, P. Van Meter, S. Van Etten, G. Freebern, Someof the Reasons Why Preparing for Exams Is so Hard: What Can Be Doneto Make It Easier? Educational Psychology Review. , 1–38 (1997).[20] H. Broekkamp, B. H. A. M. Van Hout-Wolters, Students’ adaptationof study strategies when preparing for classroom tests. Educational Psy-chology Review.19