Active Learning reduces academic risk of students with non-formal reasoning skills. Evidence from an introductory physics massive course in a Chilean public university
aa r X i v : . [ phy s i c s . e d - ph ] S e p Active Learning reduces academic risk of students with non-formal reasoning skills.Evidence from an introductory physics massive course in a Chilean public university
Guillaume Lagubeau, ∗ Silvia Tecpan, † and Carla Hernandez ‡ Universidad de Santiago de Chile, Departamento de Física (Dated: September 4, 2019)We present the findings of a pilot plan of active learning implemented in introductory physicsin a Chilean public university. The model is research based as it considered a literature review foradequate selection and design of activities, consistent with the levels of students’ reasoning skills.The level of scientific reasoning is positively correlated to student success. By contrast to a controlgroup of students following traditional lectures, we observed a significant reduction in failure ratefor students that do not yet posses formal scientific reasoning. This profile of student being themajority, we conclude that implementing active learning is particularly suited to first year of highereducation in the context of a developing country. It fits the particularities of student profile andtypical classroom size, leading to learning improvement and reduction of academic risk as well asbeing financially sound.
I. INTRODUCTION
Low student enrollment and high attrition rates inScience, Technology, Engineering, and Mathematics(STEM) education are part of the major contemporarychallenges in higher education [1]. As a consequence, in-troductory physics courses usually becomes filter coursesfor numerous engineering students [2]. Indeed, this isreflected by our institution’s historical approval rate infirst semester introductory physics course taught for allengineering careers. In an effort to reduce academic risk,as well as to better prepare engineering students to 21stcentury, a methodological change in the teaching method[3] [4] was decided in our institution. Strong evidencesaccumulated over the past 35 years of significant gainsin learning physics [5–7] using active learning motivateda pilot program of implementation of research based [8]active learning in introductory physics course.The pilot program main challenges were making it re-alistically scalable for encompassing a large enrollmentcourse (1600 students) and adapting innovative strate-gies to the context of our traditional and public univer-sity. Our institution has a voluntarily inclusive accesspolicy that favors admissions to higher education of stu-dents with very heterogeneous profiles. Students origi-nates from diverse ways of entry such as university selec-tion test, and high school ranking, among others [9, 10].As deep understanding of physics concepts requires for-mal reasoning [11, 12], it is essential to characterize thelevel of scientific reasoning of our introductory physicscourse students before tailoring a teaching sequence con-sistent with their profiles [13, 14].In this article we report on a professor training modelimplemented at piloting level to transform introductoryphysics courses for engineering programs. This model is ∗ [email protected] † [email protected] ‡ [email protected] inspired by previous similar experiences both in LatinAmerica [15–18] and others countries [19–21]. By com-paring the results of an experimental group following ac-tive learning and a control group, we found a 9.1% re-duction in failure rate (see table I), statistically signifi-cant and coherent with previous reports [5]. In addition,students with transitional reasoning level, being the ma-jority in our context, benefited most of the innovation. TABLE I.
Failure rate in introductory physics. N is theuniverse of students and p ( H O ) is the probability that failurerate decreased by introducing active learning.N traditional active variation p ( H )
304 45.7% 36.1% -9.1% 93.7%II. THEORETICAL FRAMEWORK
The level of scientific reasoning is well reported to be adetermining factor in academic success in the first yearsof university science courses [12–14]. Methodologies thatpromote active learning, that is designing lectures wherethe student is intellectually active [22] have been foundto improve scientific reasoning [23, 24].For designing classroom activities, we used activelearning strategies focused on the need to enhance con-ceptual learning, problem solving skills, collaborativework and hypothesis generation among other skills re-quired for the training of engineers in the 21st century.In particular, we used tutorials [25], interactive lecturedemonstration [26], peer instruction [27], sense-makingtasks [28] and collaborative Solving Problems [29] fittingthe program of the course (introductory mechanics andstatics). In supplementary material, we provide a list ofthe activities used for the active learning sessions.The teaching material was designed by a coordina-tion team, distinct from the group of teachers. Inter-active lectures were given by a professor accompanied bya teaching assistant. The professors had already taughtthe same course in traditional way previously, and un-dergraduate students of our university served as teachingassistants were. Both professors and teaching assistantswere present simultaneously during lecture to support thestudent’s learning process. Implementing active learningstrategies in the classroom requires a preparation workthat we structured in a three-stage cycle (fig.1 top).
FIG. 1. Top: work flow. Bottom: student centered classroom.
The first stage consists in the coordination team re-viewing the literature to prepare the activities that wouldbe used in the two class occurring two weeks after. In thesecond stage, the proposed activities are reviewed and ad-justed, if necessary in a meeting between the coordinationteam, the professors and the teaching assistants. Finally,in stage three, teachers and assistants implement activ-ities in the classroom. This process started two weeksbefore starting the semester and was repeated each week.Classes are held in Scale-Up rooms [30], build for thisproject, of capacity 54 students. Their design promotesstudent-centered learning as the teacher is not the focalpoint. Five projectors provide good visibility of projectedmaterial for all students. The furniture consisted initiallyof normal tables and chairs, arranged in groups of 9 peo-ple to work in micro groups of 3 students, and latterlarge round desks that facilitate collaborative work wereinstalled, as seen in figure 1 bottom. The entire wall ofthe room are covered with boards. In addition, black-boards of 60cm x 80cm are freely available in room forevery group of students. Finally, the room layout allowedan easy circulation of students and professors between desks.Following the flipped classroom model [31, 32] materialis provided to the students gradually: elements of theoryand exercises to complement traditional classes, identi-cal with those of the previous semester, were providedbefore the active session through a virtual institutionalplatform. Indeed this material is available to all the stu-dents, in active modality or not.Such a methodological change may be associated to anincrement in infrastructure and personal costs per stu-dent [33]. It is due to reducing sections sizes and thusincreasing the number of sections in massive courses. Dueto the layout and infrastructure of the university, it is notthe case in our university: the average section of intro-ductory physics is composed by 50 students (31 sectionsfor approximately 1550 students in total). Scaling upthe pilot plan would be realized at constant number ofsections and professors.
III. METHODSA. Experimental design
The pilot program concerned 4 sections from a totalof 31 were experimental (active learning) and 4 sectionsof equivalent historical results as control group. Bothgroups reasoning skills were characterized by taking theLawson Classroom Test of Scientific Reasoning [34] atthe start of the semester.Both control and experimental groups followed thesame weekly program of contents and had access to thesame bibliography or online material. Indeed, online ma-terial and evaluations were not innovated and were sim-ilar to previous semesters. The experimental group fol-lowed two active lectures of 1h30 each and an active ex-ercise session of 1h30. The control group had the sameschedule but using traditional teaching. Three evalua-tions (identical for all students) were carried out withthree problems each, prepared by a teaching committeenot included in the implementation of the pilot plan.
B. Failure rate analysis
For consistency with the meta-analysis of Freeman [5],we compare failure rate of students defined as the numberof student failing the course divided by the total numberof students. We only consider students that attended allthe evaluations during the semester forming a universe of304 students, 146 of which followed traditional learningand 158 active learning. For determining the statisti-cal significance of the difference in failure rate, we testthe following hypothesis ( H ): “introductory physics stu-dents following active lecture are less likely to fail thanstudents following traditional lecture". The null hypoth-esis is then that “introductory physics students followingactive lecture are as likely or more likely to fail thanstudents following traditional lecture". Our statisticalanalysis is the following: failing or passing a course isa binary process. Thus, evaluating the failure rate isequivalent to estimate a so called “cut efficiency". Bayesanalysis allows to theoretically calculate the uncertaintyof efficiency measurement due to size effect [35]. If k isthe number of positive cases and n the population, theprobability distribution of the efficiency is a beta distri-bution of parameters ( α = k + 1 , β = n − k + 1 ). We canthen calculate the density probability of the variation infailure rate (as shown in figure 3) and therefore estimatep( H ). C. Scientific reasoning diagnostic
During the two first weeks of the semester, studentsof the control and experimental groups took the LawsonClassroom Test of Scientific Reasoning [34]. The test iscomposed of 24 questions, organised by pairs. Bao [36]and Mashood [37] used the distribution of correct answeras an indicator of typical reasoning skills in first year ofuniversity, comparing different cultures. For our study,a total of 260 students took the reasoning test and werepresent in all evaluations of the semester (149 from activegroups and 111 from control group).
IV. RESULTSA. Failure rate reduction
Failure rate was 45.2% for students following tradi-tional learning, comparable to the historic rate (45.7%,considering only students that attended all evaluation).By contrast, active learning students failing rate was36.1%, evidencing a 9.1% improvement ( p ( H ) = 0 . )consistent with improvement reported in literature forphysics and STEM [5]. We conclude that active learn-ing methodology is particularly well suited in the Chileancontext for teaching introductory physics contents.Separating results by gender, one can note that whilein both modalities, female failure rate was higher thanmale failure rate, female students benefited more of theinnovation than male students (see table II): female stu-dents following traditional learning were 1.41 time morelikely to fail than those following active learning (1.29 formale students). TABLE II.
Failure rate in introductory physics as afunction of gender . p ( H O ) is the probability that thatfailure rate decreased by introducing active learning. Boldletters are used when H O is statistically likely.N traditional active variation p ( H ) Female 76 67.9% 47.9% -20.0% 94.7%Male 228 39.8% 30.9% -8.9% 90.7% B. Correlation between reasoning skills and activelearning efficiency
For comparison with previous cross cultural studies[36, 37], in fig.2 top, we present the distribution of stu-dents as a function of the number of correct answer to theLawson Classroom Test of Scientific Reasoning. The av-erage reasoning level in our students universe was foundsignificantly lower than those reported in USA, Chinaand India (see table III) thus evidencing our local needto adapt to a profile of student with reasoning skills notyet fully developed. A pair analysis of answers allows tosorts the students in three reasoning level [34]: the socalled "concrete", "transitional" and "formal" reasoningskills. At the concrete level (0-4 pairs) students are ableto classify objects and understand conservation, but notyet able to form hypotheses. At the formal level (9-12pairs), students can think abstractly and are able to con-trol and isolate variables, among other similar tasks. Atthe transitional level (5-8 pairs), students are only capa-ble of partial formal reasoning [14]In Fig.2 bottom we present the categorization of ourintroductory physics students. The majority of studentsare observed to be still in a transitional level of reason-ing. In light of this diagnostic, obtained early in thesemester (week 3), we orientated classroom activities to
FIG. 2. Results of the Lawson Classroom Test of ScientificReasoning for introductory physics students, experimentaland control groups combined (N=260). Top: distribution ofstudents as a function of the number of correct answers likely favor students with transitional or concrete reason-ing skills. Indeed this raises the need to specifically adaptthe first year of our engineering curriculum to progres-sively improve reasoning skills.
TABLE III.
Average level of Lawson Classroom Test ofScientific Reasoning, and standard deviation as com-piled by Mashood [37], and our own measurement (inbold)
USA China India This study . ± .
0% 74 . ± .
8% 69 . ± .
6% 60 . ± . -40% -30% -20% -10% 0% 10% 20% 30% 40%GlobalConcreteTransitionalFormal Physics (Freeman 2014)STEM (Freeman 2014)
FIG. 3.
Difference in failure rate between experimen-tal and control groups . Probability distributions for thewhole student universe as well as distinction of concrete, tran-sitional and formal reasoning skills are shown.TABLE IV.
Failure rate in introductory physics: globalresult and detail by preinstruction reasoning skills. p ( H O ) is the probability that failure rate decreased by introducingactive learning. Bold letters are used when H O is statisticallylikely. N traditional active variation p ( H ) Concrete 73 52.6% 54.3% 1.7% 42.7%
Transitional 152 45.0% 31.5% -13.5% 94.7%
Formal 35 18.8% 21.1% 2.2% 43.2%
In table IV we present failure rates separated byreasoning skills for both the experimental and controlgroups. The reasoning skills level is correlated to theprobability of passing the course: concrete students wereapproximately 1.7 less likely to pass the course than tran-sitional students and 2.5 less likely than formal students.While we observed a null effect on concrete and for-mal students, active methodology strongly reduced fail-ure rate of transitional students (-13.5%) as can be seenin table IV and figure 3. Results are statistically signifi-cant.
V. CONCLUSION
We implemented evidenced based active learning lec-tures in an introductory physics course for engineering,supported by a professor training model. Our study evi-dences specifically a lower academic risk for students thathad not yet developed formal reasoning skills, which arethe majority in our local context.In light of the pilot plan, it is now planed to progres-sively scale up the methodology to all sections of intro-ductory physics. Extrapolation of our findings indicate apotential increment of 150 students approving the courseeach year. In our local context, not exceptional in devel-oping countries, implementing active learning does notincrease the number of sections, making it financiallysound: the initial investment of one classroom renovationbeing equivalent to 8 students fees, would be compen-sated in less than one year by the reduction of academicrisk.
ACKNOWLEDGMENTS
We thank to Vicerrectoría Académica USACH throughConvenio Marco CM USA1856, and Engineering Facultyof USACH.C. Hernández is supported by CONICYT throughProyecto Fondecyt de Iniciación No. 11170580. S. Tec-pan, thank to Proyecto DICYT USACH No. 031931TFThe authors would also like to thank the professorsRodrigo Canto, Juan Francisco Fuentealba, Marcia Me-lendez and the teaching asistant Angel Barra, Ali Godoy,Matias Herrera and Cristobal Hormazabal for their ded-ication to the project. [1] A. Sithole, E. T. Chiyaka, P. McCarthy, D. M. Mupinga,B. K. Bucklein, and J. Kibirige, Student attraction, per-sistence and retention in stem programs: Successes andcontinuing challenges., Higher Education Studies , 46(2017).[2] H. Vasquez, A. A. Fuentes, J. A. Kypuros, andM. Azarbayejani, Early identification of at-risk studentsin a lower-level engineering gatekeeper course, in , 8410 (2014).[6] C. H. Crouch and E. Mazur, Peer instruction: Ten yearsof experience and results, American journal of physics , 970 (2001). [7] M. D. Sharma, I. D. Johnston, H. Johnston, K. Varvell,G. Robertson, A. Hopkins, C. Stewart, I. Cooper, andR. Thornton, Use of interactive lecture demonstrations:A ten year study, Physical Review Special Topics-PhysicsEducation Research , 020119 (2010).[8] G. M. Bubou, I. T. Offor, and A. S. Bappa, Why research-informed teaching in engineering education? a review ofthe evidence, European Journal of Engineering Educa-tion , 323 (2017).[9] M. V. Santelices, X. Catalán, C. Horn, and A. Vene-gas, High school ranking in university admissions at anational level: Theory of action and early results fromchile, Higher Education Policy , 159 (2018).[10] M. V. Santelices, C. Horn, and X. Catalán,Institution-level admissions initiatives inchile: enhancing equity in higher educa-tion?, Studies in Higher Education , 733 (2019),https://doi.org/10.1080/03075079.2017.1398722.[11] A. E. Lawson, Teaching inquiry science in middle andsecondary schools (Sage, 2010).[12] C. Fabby, Examining the relationship of scientific rea-soning with physics problem solving, Journal of STEMEducation (2015).[13] V. P. Coletta, J. A. Phillips, and J. J. Steinert, Whyyou should measure your students’ reasoning ability, ThePhysics Teacher , 235 (2007).[14] V. P. Coletta and J. A. Phillips, Interpreting fci scores:Normalized gain, preinstruction scores, and scientific rea-soning ability, American Journal of Physics , 1172(2005).[15] G. Zavala, H. Alarcón, and J. Benegas, Innovative train-ing of in-service teachers for active learning: A shortteacher development course based on physics educationresearch, Journal of Science Teacher Education , 559(2007).[16] G. Zavala and H. Alarcon, Evaluation of instruction usingthe conceptual survey of electricity and magnetism inmexico, in AIP Conference Proceedings , Vol. 1064 (AIP,2008) pp. 231–234.[17] A. Auyuanet, H. Modzelewski, S. Loureiro, D. Alessan-drini, and M. Míguez, Físicactiva: applying active learn-ing strategies to a large engineering lecture, EuropeanJournal of Engineering Education , 55 (2018).[18] C. Hernández and S. Tecpan, Correct answers with wrongjustifications? analysis of explanations in classical me-chanics with fci test, in Journal of Physics: ConferenceSeries , Vol. 1043 (IOP Publishing, 2018) p. 012056.[19] E. F. Redish, Oersted lecture 2013: How should we thinkabout how our students think? (2014).[20] A. L. Rudolph, B. Lamine, M. Joyce, H. Vignolles, andD. Consiglio, Introduction of interactive learning intofrench university physics classrooms, Physical review spe-cial topics-physics education research , 010103 (2014).[21] L. Deslauriers, E. Schelew, and C. Wieman, Improvedlearning in a large-enrollment physics class, science ,862 (2011). [22] D. E. Meltzer and R. K. Thornton, Resource letter alip–1:active-learning instruction in physics, American journalof physics , 478 (2012).[23] L. Ding, Detecting progression of scientific reasoningamong university science and engineering students, in Physics Education Research Conference (2013) pp. 125–128.[24] C. Fabby and K. Koenig, Relationship of scientific rea-soning to solving different physics problem types, Pro-ceedings from PERC Portland, OR, July 17-18 (2013).[25] L. C. McDermott and P. S. Shaffer,
Tutoriales para físicaintroductoria (Pearson Educación, 1997).[26] D. R. Sokoloff and R. K. Thornton, Interactive lecturedemonstrations, Interactive Lecture Demonstrations, byDavid R. Sokoloff, Ronald K. Thornton, pp. 374. ISBN0-471-48774-0. Wiley-VCH, March 2004. , 374 (2004).[27] E. Mazur, Peer instruction: A user’s manual (1999).[28] C. J. Hieggelke, S. E. Kanim, T. L. O’Kuma, and D. P.Maloney,
TIPERs: Sensemaking Tasks for IntroductoryPhysics (Pearson, 2015).[29] P. Heller, R. Keith, and S. Anderson, Teaching prob-lem solving through cooperative grouping. part 1: Groupversus individual problem solving, American journal ofphysics , 627 (1992).[30] R. J. Beichner, J. M. Saul, D. S. Abbott, J. J. Morse,D. Deardorff, R. J. Allain, S. W. Bonham, M. H. Dancy,and J. S. Risley, The student-centered activities for largeenrollment undergraduate programs (scale-up) project,Research-based reform of university physics , 2 (2007).[31] W. H. Fox-Turnbull, P. Docherty, and P. Zaka, Learn-ing engineering through the flipped classroom approach-students’ perspectives, Design and Technology Educa-tion: an International Journal , 26 (2018).[32] C. Hernández-Silva and S. Tecpan Flores, Aula invertidamediada por el uso de plataformas virtuales: un estudiode caso en la formación de profesores de física, Estudiospedagógicos (Valdivia) , 193 (2017).[33] E. Brewe, R. Dou, and R. Shand, Costs of success: Fi-nancial implications of implementation of active learningin introductory physics courses for students and adminis-trators, Physical Review Physics Education Research ,010109 (2018).[34] L. Bao, Y. Xiao, K. Koenig, and J. Han, Validity evalu-ation of the lawson classroom test of scientific reasoning,Physical Review Physics Education Research , 020106(2018).[35] M. Paterno, Calculating efficiencies and their un-certainties , Tech. Rep. (Fermi National AcceleratorLab.(FNAL), Batavia, IL (United States), 2004).[36] L. Bao, T. Cai, K. Koenig, K. Fang, J. Han, J. Wang,Q. Liu, L. Ding, L. Cui, Y. Luo, et al. , Learning andscientific reasoning, Science , 586 (2009).[37] K. Mashood and V. A. Singh, Preuniversity science ed-ucation in india: Insights and cross cultural comparison,Physical Review Physics Education Research15