Design, Analysis, Tools, and Apprenticeship (DATA) Lab
Kelsey M. Funkhouser, William Martinez, Rachel Henderson, Marcos Caballero
DDesign, Analysis, Tools, and Apprenticeship (DATA) Lab
Kelsey Funkhouser, ∗ William M. Martinez, Rachel Henderson, and Marcos D. Caballero
1, 3, 4, † Department of Physics & Astronomy, Michigan State University, East Lansing, MI, 48824 Vanderbilt Institute of Nanoscale Science and Engineering, Vanderbilt University, Nashville, TN 37235 CREATE for STEM Institute, Michigan State University, East Lansing, MI, 48824 Department of Physics & Center for Computing in Science Education, University of Oslo, N-0316 Oslo, Norway (Dated: April 4, 2019)Recently, there have been several national calls to emphasize physics practices and skills withinlaboratory courses. In this paper, we describe the redesign and implementation of a two-coursesequence of algebra-based physics laboratories at Michigan State University called Design AnalysisTools and Apprenticeship (DATA) Lab. The large-scale course transformation removes physicsspecific content from the overall learning goals of the course, and instead, uses physics concepts tofocus on specific laboratory practices and research skills that students can take into their futurecareers. Students in DATA Lab engage in the exploration of physical systems to increase theirunderstanding of experimental process, data analysis, collaboration, and scientific communication.In order to ensure our students are making progress toward the skills outlined in the course learninggoals, we designed all of the assessments in the courses to evaluate their progress specific to theselaboratory practices. Here, we will describe the structures, scaffolds, goals, and assessments of thecourse.
I. INTRODUCTION
New knowledge in physics is driven by the observa-tion of phenomena, the design of experiments to probethese phenomena, and the communication of and debatearound the resulting measurements in public fora. Lab-oratory courses in physics are thus unique spaces wherestudents can engage in these central aspects of study-ing physical systems. Greater emphasis on these aspectsin laboratory spaces is needed to accurately representthe physics discipline and to engage students in the uni-versal scientific endeavor that is driven by observation,measurement, and communication.Recently, national calls have been made to design lab-oratory instruction such that it emphasizes students’ en-gagement in experimental scientific practices rather thansimply re-enforcing content learning [1, 2]. Such ex-periences would be better aligned with discovery-basedlearning [3], which is more representative of the enter-prise of experimental physics. This focus on sciencepractices is articulated in the American Association ofPhysics Teachers’
Recommendations for the Undergradu-ate Physics Laboratory Curriculum [1]. These recommen-dations call for all laboratories in undergraduate physicsto better represent experimental physics by constructinglaboratory curriculum around science practices such asdesigning experiments, analyzing and visualizing data,and communicating physics. Arguably, middle-divisionand advanced laboratory courses for physics and astron-omy majors – with their more complex experiments andequipment as well as their focus on the professional de-velopment of future physicists – tend to engage studentswith these practices.By contrast, introductory physics laboratory coursestend to have more prescriptive and direct approachesto instruction. In these courses, students often follow awell-documented procedure and do not typically have op- portunities to explore the observed phenomenon and theassociated experimental work. At larger universities inthe United States, these introductory laboratory coursesare taught to thousands of students per semester, whichmakes these more direct approaches to instruction at-tractive as they are quite efficient. At many US schools,engineering students, physical science majors, and biolog-ical science students must pass these laboratory coursesto complete their degree program. The scale of thesecourse offerings provides an additional challenge to in-corporating science practices. There are unique examplesin the literature where students of introductory physicsare engaged with scientific practices such as the Inves-tigative Science Learning Environment (ISLE) [4] andStudio Physics [5]. However, these courses have the ad-vantage of being taught to smaller population of studentsthan most introductory laboratory courses, in the case ofISLE, or having an integrated “lecture” and a modifiedinstructional space, in the case of Studio Physics, andthus can make use of greater instructional resources.In this paper, we describe a stand-alone, introductoryphysics laboratory course sequence for biological sciencemajors at Michigan State University (MSU) that was de-signed specifically to engage students in scientific prac-tices through the work of experimental physics. Studentslearn to design experiments, analyze and visualize theirdata, and communicate their results to their peers andinstructors. Design, Analysis, Tools, and Apprenticeship(DATA) Lab is unique in that it is was explicitly designedwith the AAPT Lab Recommendations in mind. The se-quence is a stand-alone mechanics laboratory (DL1) anda separate E&M and optics laboratory (DL2), which istaught to more than 2000 students per year. Further-more, the process of developing and launching this pairof courses required that we confront and overcome severalwell-documented challenges such as departmental normsfor the course, expectations of content coverage, and the a r X i v : . [ phy s i c s . e d - ph ] A p r lack of instructor time [6].We begin this paper by describing how the learninggoals for the lab sequence were constructed through aconsensus-driven process (Sec. II). In Sec. III, we pro-vide an overview of the course structure – diving deeperinto the details of the course materials later (Sec. IV). Wedescribe the assessments for this course in Sec. V as theyare somewhat non-traditional for a course of this leveland scale. To make our discussion concrete, we highlighta particular example in Sec. VI. Finally, we offer a mea-sure of efficacy using student responses to the ColoradoLearning Attitudes about Science Survey for Experimen-tal Physics [7] (Sec. VII) and some concluding remarks(Sec. VIII) II. LEARNING GOALS
As this laboratory course serves the largest populationof students enrolled in introductory physics at MSU, itwas critical to develop a transformed course that reflectedfaculty voice in the design. While physics faculty are notoften steeped in formal aspects of curriculum develop-ment, sustained efforts to transform physics courses takean approach where faculty are engaged in the process todevelop a consensus design [8–10]. In this process, in-terested faculty are invited to participate in discussionsaround curriculum design, but experts in curriculum andinstruction synthesize those discussions to develop coursestructures, materials, and pedagogy. These efforts arethen reflected out to faculty to iterate on the process.Our design process followed the approach developed bythe University of Colorado’s Science Education Initiative[8–10]. In this process, faculty are engaged in broad dis-cussions about learning goals, the necessary evidence toachieve the expected learning, and the teaching practicesand course activities that provide evidence that studentsare meeting these goals. Below, we discuss the approachto developing learning goals for the course as well aspresent the finalized set of learning goals from which thecourse was designed. We refer readers to Wieman [10] fora comprehensive discussion of setting about transformingcourses at this scale.Prior to engaging in curriculum and pedagogical de-sign, an interview protocol was developed to talk withfaculty about what they wanted students to get out ofthis laboratory course once students had completed thetwo semester sequence. The interview focused discus-sion on what made an introductory laboratory coursein physics important for these students and what roleit should play as a distinct course since, at MSU, stu-dents do not need to enroll in the laboratory course atthe same time as the associated lecture course. A widevariety of faculty members were interviewed includingthose who had previously taught the course, those whohad taught other physics laboratory courses, and thosewho conduct experimental research. In total, 15 inter-views were conducted with faculty. This number repre- sents more than half of the total number of experimentalfaculty who teach at MSU.The discussion of faculty learning goals was wide-ranging and covered a variety of important aspects of lab-oratory work including many of the aspects highlightedin the AAPT Laboratory Guidelines [1]. Interviews werecoded for general themes of faculty goals and the initiallist included: developing skepticism in their own work,in science, and the media; understanding that measure-ments have uncertainty; developing agency over theirown learning; communicating their results to a wider va-riety of audiences; learning how to use multiple sources ofinformation to develop their understanding; demonstrat-ing the ability to use and understand equipment; docu-menting their work effectively; and becoming reflectiveof their own experimental work.With the intent of resolving the faculty’s expressedgoals with the AAPT Lab Guidelines, the goals were syn-thesized under larger headings, which aimed to combineand/or to connect seemingly disconnected goals. In ad-dition, through a series of informational meetings thatroughly 10-12 faculty attended regularly, how these goalswere being combined and connected to interested facultywere reflected upon. Additional critiques and refinementsof these goals were collected through notes taken duringthese meetings. Through several revisions, a set of fourbroad goals that faculty agreed reflected their views onthe purpose of this part of laboratory courses was final-ized. Additionally, these goals were also represented inthe AAPT Lab Guidelines. The finalized goals are listedin Table I along with short description of each; they areenumerated (LG X ) in order to refer to them in later sec-tions.The learning goals formed the basis for the design ofcourse structures including materials and pedagogy. Toconstruct these course structures, constructive alignment[11] was leveraged, which helped ensure that the designedmaterials and enacted pedagogy were aligned with theoverall learning goals for the course. These structuresare described in the next section where we have includeda direct reference to each learning goal that a particularcourse structure is supporting. III. COURSE STRUCTURES
Each laboratory section consists of twenty studentsand two instructors – one graduate teaching assistant(GTA) and one undergraduate learning assistant (ULA)[12]. The students are separated into five groups of four,which they remain in for 4 to 6 weeks – 4 to 6 classmeetings. This time frame works well because it givesthe students time to grow and improve as a group as wellas individuals within a consistent group. In addition,when the groups are switched it requires the students toadapt to a new group of peers. The groups complete 6(DL1) or 5 (DL2) experiments during the semester, mostof them spanning two weeks – two class meetings. Fig. 1
TABLE I. Finalized learning goals for DATA LabLearning Goal DescriptionLG1 - Experimental Process Planning and executing an experiment to effectively explore how differ-ent parameters of a physical system interact with each other. Generallytaking the form of model evaluation or determination.LG2 - Data Analysis Knowing how to turn raw data into an interpretable result (throughplots, calculations, error analysis, comparison to an expectation, etc.)that can be connected to the bigger physics concepts.LG3 - Collaboration Working effectively as a group. Communicating your ideas and under-standing. Coming to a consensus and making decisions as a group.LG4 - Communication Communicating understanding – of the physics, the experimental pro-cess, the results – in a variety of authentic ways – to your peers, in a labnotebook, in a presentation or proposal.FIG. 1. Week-by-week schedule of DATA Lab I & II. provides an overview of the two-semester sequence andwill be unpacked further below. We indicate the labo-ratories that students complete with light green squares(introductory experiments) and dark green squares (twoweek labs). The students keep a written lab notebook,which they turn in to be graded at the end of each ex-periment. In this laboratory course, each group con-ducts a different experiment. This is possible because,in general, students tend to follow a similar path withrespect to the learning goals and there is no set end-point for any individual experiment. As long as studentscontinue to work through the experimental process andcomplete analysis of their data, they are working towardsthe learning goals and can be evaluated using the alignedassessments (Sec. V). This approach also emphasizes thatthere is not one way to complete an experiment; this hasadded benefits for students’ ownership and agency of thework as they must decide how to proceed through theexperiment. In addition, having no set endpoint and twoweeks to complete most experiments takes away the timepressure to reach a specific point in a given time. All ofthese aspects allow students to more fully engage withthe work they are doing and, in turn, make progresstoward the learning goals. Having each group conduct a different experiment addressed a significant point ofdiscussion among the faculty; specifically, not coveringthe same breadth of content was a major concern. Al-though, through this design, students do not completeall of the experiments, they are introduced to all of theconcepts through the peer evaluation of the communica-tion projects (red squares in Fig. 1, addressed in detailbelow).
A. Laboratory Activities
The laboratory activities were designed around thelearning goals. As such, the experiments follow a sim-ilar path from the beginning of the experimental processthrough analysis, with communication and collaborationas central components throughout. The course structuresin relation to each of the learning goals are highlightedbelow. The core component (i.e. lab activities) of thecourse sequence is outlined in Fig. 2.
LG1 - Experimental Process:
The students begineach experiment by broadly exploring the relevant pa-rameters and their relationships. Typically, students in-vestigate how changing one parameter affects another by
FIG. 2. A snapshot of an experiment from pre-class homeworkthrough the communication project. making predictions and connecting their observations tophysics ideas (qualitative exploration in Fig. 2). Fromthese initial investigations, students work toward de-signing an experiment by determining what to measure,change, and keep the same. This often requires groundingdecisions on some known model or an observed relation-ship (quantitative exploration, experimental design, andinvestigation in Fig. 2).
LG2 - Data Analysis:
After additional formal investi-gations in which data has been collected, students sum-marize the raw data into an interpretable result. Thistypically includes some form of data analysis; for exam-ple, constructing a plot to evaluate a model or determin-ing a quantitative relationship between the different vari-ables in the data. In this work, the students are expectedto make claims that are supported by their results. Thisoften involves the students finding the slope and/or inter-cept in a plot and interpreting those results with respectto their expectations (discussion and analysis in Fig. 2).
LG3 - Collaboration:
Throughout the experimentalwork and analysis, students discuss and make decisionswith their peers in their lab group. Students are encour-aged to develop a consensus approach to their work –deciding collectively where to take their experiment andanalysis. Furthermore, students are expected to makethese decisions by grounding their discussions in theirexperiment, data, and analysis.
LG4 - Communication:
Overall, the entire process re-quires that students communicate with their group andinstructors. Additionally, students communicate theirexperimental approach and the results of their work in-cluding their analysis in their lab notebook. Later, stu-dents provide a more formal presentation of their workin the form of the communication projects.It be should emphasized that this process is not con-tent dependent; each laboratory activity conducted bya student group follows this process. This generalization enables the core components of the course to be repeated(see Fig. 1) to help address external constraints, such aslimited equipment and time to work on experiments.
B. Communication Projects
DATA Lab is also defined by the focus on authen-tic scientific communication through the communicationprojects (CPs). The CPs are a formal way for the stu-dents to present their work and they are one of the as-sessments of the course in which the work done by thestudents is completed individually. CPs replace the labpractical from the traditional version of the course wherestudents would conduct a smaller portion of a laboratoryby themselves. CPs occur in the middle and at the end ofthe semester (red squares in Fig.1). In DL1, the CP is awritten proposal that summarizes the work the studentsconducted in one of their previous experiments and pro-poses an additional investigation. In DL2, the studentscreate and present a research poster on one of (or a por-tion of one of) their experiments. In both courses, theprojects are shared with and reviewed by their primaryinstructor and their peers in the class.Through the CPs, students continue to engage with thefaculty consensus learning goals (Sec. II) as described be-low:
LG1 - Experimental Process:
Students are expectedto reflect on and summarize the process through whichthey went to complete the experiment. In so doing, theymust communicate their rationale and reasoning for fol-lowing that process.
LG2 - Data Analysis:
The students must show thatthey can turn their raw data into an interpretable result.Again, this is often and, ideally, done in the form of aplot of their data with the emphasize of a model, includ-ing a fit, is needed. Students also present and explainwhat the results mean in the context of the experimentand a physical model.
LG3 - Collaboration:
While the experiment was com-pleted with the student’s group where they may haveconsulted with their group mates, the CPs themselvesare not inherently collaborative. However, in DL1, thereviews that students perform on each other’s projectsare done collaboratively in their groups.
LG4 - Communication:
The CPs are the formal com-munication of a student’s experimental work. In bothcourses, a student’s CP is reviewed by their peers andfeedback is provided describing successes and shortcom-ings along with suggestions for improvements.
C. Final Projects
The course structure was designed with the intent toprovide students with a variety of ways to engage in theexperimental physics practices. The final projects arean additional form of communication including an anal-ysis and interpretation of experimental results throughcritiquing other scientific results (DL1–Critique Project)and describing a new experimental design (DL2–DesignProject).
Critique Project : For the final project in DL1, stu-dents critique two sides of a popular science topic. In theprior week, students are arranged into new groups andbefore the class meeting, they must choose, as a group,from a list of possible topics such as climate change andalternative energy. In class, students collectively writeup a summary and critique both sides of the scientificargument.
Design Project : For the final project in DL2, studentschoose an experiment that was conducted previously anddesign a new experiment for a future semester of DATALab. Similarly to DL1, the students are sorted into newgroups and they must decide, as a group, which exper-iment they will be working on before the class meeting.Due to the structure of the course, specifically every-one doing different experiments throughout the semester,this choice may be an experiment that individual mem-bers of the group did not complete; negotiating this de-cision is part of the process of the Design Project. Inclass, students construct two documents: (1) a documentthat explains the design of the new experiment and (2)a document that would aide a future DATA Lab instruc-tor to teach the experiment. Through this final project,DL2 students can design a project covering material thatthey may not have had the chance to explore during thecourse.For both final projects, students turn in one assign-ment per group and they receive a single grade (as agroup) for the assignment. Students also assess theirown in-class participation, providing themselves a par-ticipation score (on a 4.0 scale) for the day. This score issubmitted to their instructor along with their rationalefor assigning themselves the grade.These projects offer the final opportunity for DATALab students to engage with the faculty-consensus learn-ing goals:
LG1 - Experimental Process:
In DL1, students eval-uate and summarize both sides of the chosen argument byreviewing the relevant data and experiments. Althoughstudents are not conducting an experiment, they are stillasked to be critical of the experimental process in eachside of the argument. In DL2, students must create aclear procedure for their proposed experiment. Here,they must consider the available equipment as well ashow the data would be collected and why.
LG2 - Data Analysis:
In DL1, the students must eval-uate the evidence provided in each article. They mustdecide if there are obvious flaws in the way the analysiswas conducted and if the analysis is compelling; that is,if the overall claims made in article align with the dataand analysis. In DL2, students must consider the kindof analysis that would fit with their experiment and thedata that they would collect. In addition, students arealso expected to reflect on their analysis in light of the models that are available to explain the data they wouldcollect.
LG3 - Collaboration:
In both courses, students con-tinue to work as a group and are graded accordingly. Inaddition, the students have been put into new groups,which they must adjust to.
LG4 - Communication:
In both courses, students con-tinue to communicate with their group as part of the col-laboration. In DL1 specifically, the final project providesan opportunity to communicate their own evaluation andcritique of a scientific arguments. Students in DL2 areexpected to communicate to different audiences, includ-ing future DATA Lab students and instructors, abouttheir newly planned experiment.
IV. OVERVIEW OF KEY SUPPORTS
As the students’ work in this course is sufficiently open-ended, specific supports to ensure they feel capable ofconducting the lab activities have been designed. Sincethe CPs are the main assessments in the DATA Labcourse sequence and are a large portion of their over-all grade for the course, the goals of the key supports areintended to provide students with the tools to help themsucceed in the projects. Each of the supports designed forDATA Lab will be discussed in detail below (Secs. IV A& IV B). Assessments will be discussed in Sec. V]Broadly, the key supports for the students are outlinedin Fig. 2. Before each class day, students complete a pre-class homework assignment (vertical green lines). Stu-dents also have three communication project homework(CPHW) assignments during the semester (vertical pinkline) to help them complete their CPs. These supports,in addition to feedback on students’ in-class participa-tion and lab notebooks, apply for any of the regular twoweek experiments (green squares Fig. 1). In the followingsection, these will be described in detail along with theadditional supports that were designed for the courses.
A. Typical Experiment
Each two-week experiment follows a similar path, high-lighted in Fig. 2 and described, in part, in Sec. III. In thissection, details of the general course components neces-sary to maintain the flexibility of the path students takethrough each experiment will be described.
Pre-Class Homework:
At the beginning of an ex-periment, students are expected to complete the pre-classhomework assignment which includes reading throughthe lab handout and investigating the suggested research.This assignment is usually 2-4 questions designed to havestudents prepare for the upcoming experiment. For ex-ample, before the first day of a new lab, students areasked what they learned during their pre-class researchand if they have any questions or concerns about the labhandout. Between the first and second class meeting ofthe two-week experiment, students are expected to re-flect on what they have already done and prepare forwhat they plan to do next. Typically, the 2-4 questionsinclude reflections from the prior week, such as any issuestheir group ran into on the first day, and what they in-tend on doing during the second day of the experiment.Answers to the pre-class homework serve as additionalinformation that the instructors can draw on during theclass; knowing what questions and confusions that theirstudents might have can help instructors be more respon-sive during class. Overall, the goal of the pre-class home-work is for the students to come into class prepared toconduct their experiment and this assignment is used tohold them accountable for that preparation.
In-class Participation : With the overall intent ofimproving students’ specific laboratory skills and prac-tices that are outlined in the course learning goals(Sec. II), students receive in-class participation gradesand feedback after every lab section (green squares inFigs 1 & 2) on their engagement with respect to thesepractices. As the lab handouts do not provide studentswith specific steps that they must take to complete theexperiment, students are expected to make most of thedecisions together as a group. Generally, students havecontrol over how their investigation proceeds; however,this control varies between experiments (i.e. studentschoose how to set up the equipment, what to measure,how to take measurements, etc.). The in-class partici-pation grades and feedback are where students are as-sessed most frequently and where they have the quickestturnaround to implement the changes. See Sec. V A forthe details of how in-class participation is assessed.
Lab Notebooks : For each experiment that the stu-dents engage in, they are expected to document theirwork in a lab notebook. In comparison to formal labreports, lab notebooks are considered a more authenticapproach to documenting experimental work. Further-more, lab notebooks provide students with space to de-cide what is important and how to present it. The labnotebooks are the primary source that the students useto create their CPs. Like in-class participation, studentsreceive lab notebook feedback much more regularly thanCP feedback, so they have greater opportunity to reflectand make improvements. The specific details of the as-sessment of lab notebooks will be explained in Sec. V A.
CP Homeworks:
Three times during the semesterthe students complete CPHW assignments in addition tothat week’s pre-class homework. Each CPHW focuses ona relevant portion of the CPs (e.g., making a figure anda caption). Through the CPHWs, the aim is for studentsto develop experience with more of the CP components.In addition, students receive feedback on these differentaspects (see Sec. V A) , which they can act upon beforethey have to complete their final CPs.
Communication Projects:
Throughout eachsemester, the students complete two CPs, the first ofwhich is a smaller portion of their overall course grade.With the goal of providing the students with a second opportunity to conduct a CP after receiving initial feed-back, this course design feature intends to create lesspressure on students during their first CP assignment.Students are expected to reflect on the process, theirgrade, and the feedback before they have to completeanother CP. The CP assessment details will be discussfurther in Sec. V B.
B. Additional Supports
Along with the support structures for the core com-ponents of the course sequence, additional supports havebeen designed to ease students into the more authen-tic features of DATA Lab such as designing experimentsand documenting progress in lab notebooks. DL1 be-gins with three weeks of workshops (purple squares inFig. 1), followed by the introductory experiment (lightgreen squares in Fig. 1) that all of the students complete.DL2 begins with an introductory experiment as well,under the assumption that the students already wentthrough DL1. The workshops and introductory exper-iments are designed to assist the students in navigatingthe different requirements and expectations of the overallcourse sequence, and of a typical experiment within eachcourse. The additional support structures are describedin detail below.
DL1 Workshops:
The first workshop focuses on mea-surement and uncertainty with a push for the students todiscuss and share their ideas (LG1,3). The students per-form several different measurements – length of a metalblock, diameter of a bouncy ball, length of a string, massof a weight, and the angle of a board. Each group dis-cusses the potential uncertainty associated with one ofthe measurements. Then, students perform one addi-tional measurement and assign uncertainty to it. Thesecond workshop also focuses on uncertainty but in re-lation to data analysis and evaluating models (LG2,4)using the concept of a spring constant. Students collectthe necessary measurements, while addressing the asso-ciated uncertainty and plot the measurements to analyzehow the plot relates to the model of a spring. The fi-nal workshop focuses on proper documentation. The labhandouts do not contain their own procedure, so eachstudent is expected to document the steps they take andtheir reasoning (LG4) in their lab notebook. In prepa-ration for the third workshop, as a pre-class homework,students submit a procedure for making a peanut butterand jelly sandwich, which they discuss and evaluate inclass. Students are then tasked with developing a pro-cedure to determine the relationship between differentparameters (length of a spring and mass added, angle ofmetal strip and the magnets placed on it, or time for aball to roll down a chute and how many blocks are underthe chute. At the end of each workshop the students turnin their notebooks, just as they would at the end of anyexperiment.
Introductory Experiments:
In DL1, the introduc-tory experiment occurs after the three workshops. Allstudents conduct a free-fall experiment where they mustdetermine the acceleration due to gravity and the termi-nal velocity for a falling object. In DL2, the introductoryexperiment is the first activity in the course. This is be-cause students will have already completed DL1 priorto taking DL2; rather than being slowly introduced towhat DATA Lab focuses on, students can be remindedin a single experiment. The introductory experiment forDL2 involves Ohm’s Law; students must determine theresistance of a given resistor.As these are the first DATA Lab experiments for ei-ther course, the instructors take a more hands-on andguiding approach than they will later in the semester.In DL1, these instructional changes represent a dramaticshift from the guidance students had during the work-shops where instructors are often quite involved. In DL2,the one week lab is intended to be simple enough that stu-dents can be reminded of the expectations with respectto the overall learning goals of the course.
CP Prep Day:
As discussed in the prior section, theCPs comprise a large portion of the students’ total gradein the course. In addition to the supports that were al-ready mentioned – in class grades, notebooks, CPHW,and a lower stakes CP1 – in the spring semester, theMSU academic calendar offers time for a communicationproject prep day (pink squares in Fig. 1). This gives thestudents an extra day where they have time to work ontheir CPs in class. They can take additional measure-ments, seek help from their group or instructor, or workon the project itself. This prep day allows for a gentlertransition into the CPs with a bit more guidance. It alsoreduces the amount of work that the students have to dooutside of class.
V. IN COURSE ASSESSMENTS
The DATA Lab activities described above were de-signed around the overall learning goals outlined inSec. II. As such, the course assessments were also alignedwith these overall course goals. There are two types ofassessments used in DATA Lab – formative (to help thestudents improve upon their work) and summative (toevaluate the students’ output); these are separated forclarity. In this section, the various assessment tools arediscussed with respect to the overall learning goals of thecourse.
A. Formative Assessments
In DATA Lab the formative assessments are comprisedof students’ work on their in-class activities, lab note-books, and CPHWs. Other than the pre-class homework,which is graded on completion, there is a rubric for eachactivity for which students receive a score. Each is struc-tured to ensure that any improvements students make carry over to their CPs.
In-class Participation : In-class participation feed-back is broken into group, which covers the general thingseveryone in the group or the group as a whole needs towork on, and individual, which is specific to the studentand not seen by other group members. The general struc-ture of the feedback follows an evaluation rubric used inother introductory courses and focuses on something theydid well, something they need to work on, and advice onhow to improve [13]. It is expected that students willwork on the aspects mentioned in their prior week’s feed-back during the next week’s class. Students are gradedbased on their response to that feedback. Any improve-ments they make with respect to the learning goals inclass will also likely impact how well they complete theirCPs.Students’ in-class participation is assessed with respectto two components, group function and experimental de-sign. Specifically, group function covers their work incommunication, collaboration, and discussion (LG3,4).For communication they are expected to contribute toand engage in group discussions. To do well in collabo-ration, students should come to class prepared and ac-tively participate in the groups activities. Discussionmeans working as a group to understand the results oftheir experiment. Experimental design evaluates the pro-cess that students take through the experiment and theirengagement in experimental physics practices (LG1,2).They are expected to engage with and show competencein use of equipment, employ good experimental practices(i.e., work systematically, make predictions, record ob-servations, and set goals) and take into account whereuncertainty plays into the experimental process (i.e., re-duce, record, and discuss it).Specifically for the DL1 Workshops, instructors gradestudents differently than they would for a typical exper-iment. The emphasis for the workshops is on the groupfunction aspect of the rubrics, communication and par-ticipation. This is because the students are being easedinto the expectations that the instructors have aroundexperimental work.
Lab Notebooks : Feedback and grades for lab note-books are only provided after the experiment is com-pleted (the two week block in Figs 1 & 2). Studentsreceive individual feedback on their notebook, althoughmembers of a group may receive feedback on some of thesame things simply because they conducted the experi-ment together. Like for in-class participation, it is ex-pected that the students will work on the aspects men-tioned in their feedback for the next lab notebook andthe instructor can remind them of these things in classduring the experiment.Lab notebooks are also graded over two components,experimental design and discussion. Experimental designfocuses on the experimental process and how studentscommunicate it (LG1,4). Here, instructors typically lookfor clearly recorded steps and results, and intentional pro-gression through the experiment. Discussion covers un-certainty in the measurements and the models, as wellas the results, with respect to any plots and conclusions(LG2,4). These evaluation rubrics for the lab notebookswere designed to be aligned with the those for the CPs,so that when students work toward improving their note-books they are also making improvements that will ben-efit their CPs. For example, if a student is getting betterat analyzing data and communicating their results withintheir notebooks, instructors should expect the same im-provement to transfer to their CPs.For the DL1 Workshops, the lab notebooks are gradedon the same components but the grades and feedback arespecifically focused on the parts of the rubric that thestudents should have addressed in each of the previousworkshops. For example, as documentation is empha-sized in the last workshop, the students are not heavilypenalized on poorly documented procedures in the firsttwo workshops.
CPHW : The goal of this CPHW is to have studentsthink about creating a more complete CP that con-nects their in-class work to the bigger picture. Stu-dents are evaluated on the quality and relevance of theirsources, including the background and real-life connec-tions (LG2,4). Each CPHW has a different rubric be-cause each one addresses a different aspect of the CPs.
Figure and caption : The students create a figure witha robust caption based on the data from one of the labsthey completed. Both the figure and the caption are eval-uated on communication and uncertainty (LG2,4). Forthe plot, the students are expected to visualize the dataclearly with error bars and it should provide insight intothe various parameters within the experiment. For thecaption, students need to discuss what is being plotted,make comparisons to the model including deviations, anddraw conclusions that include uncertainty.
Abstract : For a given experiment, students write a re-search abstract that covers the main sections of theirproject including introduction, methods, results, andconclusion. These are assessed on experimental process(motivation and clarity of the experiment) (LG1,4), anddiscussion (results and conclusions) (LG2,4).
Critique (DL1 only) : Students are given an example pro-posal that they must read, critique, and grade. This as-signment plays two roles. First, students must examinea proposal, which should help to produce their own. Sec-ond, students must critique the proposal, which shouldhelp them provide better critiques to their peers. Stu-dents’ performance is evaluated based on their identifi-cation of the different components of a proposal, and thequality of the feedback they provide (LG4).
Background (DL2 only) : Students are tasked with findingthree out-of-class sources related to one of their optics ex-periments, which they must summarize and connect backto the experiment.
B. Summative Assessment
The CPs form the sole summative assessment of stu-dent learning in DATA Lab. As described above, each ofthe formative assessments are designed to align with thegoals of the CPs.
CPs : As mentioned above, although students conductthe experiments together, the CPs are completed individ-ually. In DL1, students’ CP is a proposal that emphasizestheir prior work and discusses a proposed piece of futurework. As a result, the CP rubric is divided into twosections, prior and future work. Within those sections,there is a focus on experimental design and discussion.This rubric was iterated on after piloting the course fortwo semesters as it was found that students would of-ten neglect either their future work or prior work whenthey were not directly addressed in the rubric; the rubricswere reorganized in order to account for this. Experimen-tal design, which covers methods and uncertainty, focuseson the experimental methods and the uncertainty in mea-surements, models, and results when students discusstheir prior work (LG1,2,4). In future work, experimentaldesign refers to the proposed experimental methodologyand the reasoning behind their choices (LG1,4). For thestudent’s discussion of prior work, the rubric emphasizeshow the they communicate their results (LG2,4). Whenstudents discuss their future work, the rubric emphasizesthe novelty of the proposed experiment and the argu-ments made on the value of the project (LG1,4).In DL2, students’ CP is a poster that they presentto their classmates for peer review. The rubric includesan additional component on the presentation itself, butthe rubric still emphasizes the experimental design anddiscussion. Experimental design covers communicationof the experimental process including students’ reason-ing and motivation. Discussion focuses on the discussionof uncertainty (i.e., in the measurements and models)and the discussion of results (i.e., in the plot and con-clusions). The additional component focusing on presen-tation is divided into specifics about the poster (i.e., itsstructure, figures, layout) and the student’s presentationof the project (i.e., clear flow of discussion, ability to an-swer questions).
VI. EXAMPLE EXPERIMENT
Overall, the course structures, supports, and assess-ments of DATA Lab have been discussed. In this section,the key supports will be grounded in examples from aspecific experiment. The details of a specific two-weekexperiment will be described to better contextualize thefeatures of the course. Additional experiments are listedin Tables II and III in the Appendix. The chosen experi-ment is from DL2 and is called “Snell’s Law: Rainbows”.In this experiment, students explore the index of refrac-tion for different media and different wavelengths of light.Before attending the first day of the laboratory activ-
Research Concepts
To do this lab, it will help to do some research onthe concepts underlying the bending of light atinterfaces including: • Snell’s Law (get more details than presentedhere) • Refraction and how it differs from reflection • Index of refraction of materials • Fiber optics • Using this simulation might be helpful:http://goo.gl/HEflDI • How to obtain estimates for fits in your data(e.g., the LINEST function in Excel -http://goo.gl/wiZH3p)FIG. 3. Pre-class research prompts for the Snell’s Law lab. ity, students are expected to conduct the pre-class home-work assignment, including the recommended research inFig. 3. In addition, the homework questions for the firstday of a new experiment address the pre-class research,as follows:1. Describe something you found interesting in yourpre-class research.2. From reading your procedure, where do you thinkyou may encounter challenges in this lab? Whatcan you do to prepare for these?3. Considering your assigned lab, is there anythingspecific about the lab handout that is unclear orconfusing?The first day of the lab begins with exploring refrac-tion in a water tank. Students are asked to qualitativelyexplore the index of refraction of the water using a simplesetup (Fig. 4). The exploration is fully student led; theyinvestigate the laser and tank, discussing what they seewith their group as they go and recording their observa-tions in their notebooks. Students observe that the pathof the light changes once the laser crosses the air-waterboundary. Students are then lead to a quantitative ex-ploration by determining the index of refraction of thewater; instructors expect the students to have an idea ofhow to do this after their pre-class research. If studentsare not sure how to start, they are encouraged to searchfor Snell’s Law online where they can quickly find a rele-vant example. The instructors check in with the studentstoward the end of this work. Typically, instructors willask about the questions outlined in the lab handout.The next part of the experiment is where students workto gain precision in their measurements and evaluate themodel of the system. This part is most similar to a tra-ditional laboratory course. The difference is that thestudents are told the goal but not how to proceed (seeFig. 4). There are a number of decisions they must makeas a group as they progress. Students record and explain their decisions in their lab notebooks; they might alsodiscuss them with their instructor.Typically by the end of the first day students know howto set up their experiment and have documented that intheir lab notebooks. They are unlikely to have takenmore than one measurement (the design and investiga-tion phase in Fig. 2). They will return the following weekto complete their experiment. The homework questionsbetween the first week and the week that they return em-phasize students’ reflections on the previous week. Stu-dents also are asked think about the experiment outsideof class. The typical homework questions prior to Week2 are the following:1. Because you will be working on the same lab thisweek, it is useful to be reflective on your currentprogress and plans. Describe where your groupended up in your current lab, and what you planto do next.2. Now that you are halfway through your current laband are more familiar with the experiment, whathave you done to prepare for this upcoming class?3. Describe something that you found interesting inyour current lab and what you would do to inves-tigate it further.The second week starts with setting up the experimentagain and beginning the process of taking multiple mea-surements. At this point, students often break up intodifferent roles: someone manipulating the equipment, oneor two people taking measurements, and someone record-ing the data and/or doing calculations. These roles arewhat students appear to fall into naturally, and are notassigned to them. Although, if one student is alwaysworking in excel or always taking the measurements, in-structors will address it in their feedback where they en-courage the students to switch roles.The next step depends on the amount of time that stu-dents have left in the class. If there is not much time,students focus on the data from one wavelength of light.If they have more time, they can make the same mea-surements with lasers of different wavelengths. In bothcases, students can determine the index of refraction ofthe acrylic block. With multiple wavelengths, studentsare able to see that the index of refraction depends onwavelength. This leads to a conversation with the in-structor about how this relates to rainbows and a critiqueof the model of refraction – Snell’s Law.Most of the analysis that students conduct in this ex-ample experiment is the same regardless of how manylasers they collected data (discussion and analysis inFig. 2). While considering the different variables in theirexperiment, students are expected to make a plot wherethe slope tells them something about the physical sys-tem. In this case, the design is intended for the studentsto plot the sine of the angle of incidence on the x-axis andthe sine of the angle of refraction on the y-axis, which0
Part 1 - Observing Light in Water
At your table, you have a tank of water and a green laser. Turn on the green laser and point it at the water’ssurface. • What do you notice about the beam of light in the water? • What about the path the light takes from the source to the bottom of the tank?Let’s get a little quantitative with this set up. Can you measure the index of refraction of the water? Youhave a whiteboard marker, a ruler, and a protractor to help you. Don’t worry about making manymeasurements, just see if you can get a rough estimate by taking a single measurement. • What does your setup and procedure look like for this experiment? • What part(s) of your setup/procedure is(are) the main source of uncertainty for this measurement? • Can you gain a sense of the uncertainty in this measurement? • How close is your predicted value to the “true value” of the index of refraction of mater?On the optical rail you have a half circle shape of acrylic that is positioned on a rotating stage, with angularmeasurements. You also have a piece of paper with a grid attached to a black panel (i.e., a “beam stop”).Using this setup, you will test Snell’s Law for the green laser. Your group will need to decide how to set upyour experiments and what measurements you will make. You should sketch the setup in your lab notebookand it would be good to be able to explain how your measurements relate to Snell’s Law (i.e., how will thelaser beam travel and be bent by the acrylic block?). In conducting this experiment, consider, • What measurements do you need to make? • What is the path of the laser beam and how does it correspond to measurements that you are making? • What is a good experimental procedure for testing Snell’s Law? • What kind of plot is a useful one to convey how the model (Snell’s Law) and your measurementsmatch up? • Where is the greatest source of uncertainty in your experimental setup? What does that mean aboutthe uncertainty in your measurements?FIG. 4. Snell’s Law: Rainbows Lab Handout.
Top : Exploring refraction, first day of Snell’s Law.
Bottom : Beginning modelevaluation, main Snell’s Law activity. makes the slope the index of refraction of the acrylicblock. The optics experiments occur in the second halfof the semester after the students have become familiarwith constructing linear plots from nonlinear functions.For this lab, students usually do not have much difficultydetermining what they should plot. After they obtainthe slope and the error in the slope, students will typi-cally compare it to the known index of refraction of theacrylic block. They must research this online as it is notprovided anywhere for them in the lab handout.The second day of the experiment ends with a discus-sion of their plot. Students construct a conclusion intheir notebooks that summarizes the results, what theyfound, what they expected, reasons for any differences,and an explanation of what it all means in the largerphysics context.After the experiment, the students may have theirthird and final CPHW, background/literature review. Inthe case of Snell’s Law, students would be asked to findthree additional sources where these concepts are used insome other form of research, often in the field of medicinebut also in physics or other sciences. Students then sum-marize what they did in class and connect their experi- mental work to the sources that they found.The student can choose to do their second CP on thisexperiment. An example of a poster can be seen in Fig. 5.In the figure, three key features are highlighted. First, inthe blue box, is the graph where students plotted all threewavelengths of light. In the green box, is the slope foreach color, which is the index of refraction of the acrylicfor each laser. Finally, in the red boxes, are their re-sults and conclusion. In the top box, students explainedwhy their indices are different, that is, because of the as-sumption that Snell’s Law is wavelength independent. Inthe bottom box, they make the connection to rainbows.The student would present this poster during the in-classposter session, to their peers and their instructor.
VII. REDESIGN EFFICACY
To measure the efficacy of the DATA Lab course trans-formation, the Colorado Learning Attitudes about Sci-ence Survey for Experimental Physics (E-CLASS) [7] wasimplemented in the traditional laboratory course as wellas the transformed courses. The E-CLASS is a research-1
FIG. 5. Sample of a student’s Communication Project for DL2.
Blue : graph with sine of the angle of incidence plotted againstsine of the angle of refraction for each wavelength of light.
Green : The slope for each wavelength, which is the index of refractionof the block.
Red : Results and conclusions where they discuss the differences in the indices of refraction and how that is relatedto rainbows. based assessment tool used to measure students’ epis-temology and expectations about experimental physics[14–16]. The well-validated survey consists of 30 items(5-point Likert scale) where students are asked to ratetheir level of agreement with each statement. The scor-ing method of this assessment was adapted from previ-ous studies [17]. First, the 5-point Likert scale is com-pressed into a 3-point scale; “(dis)agree” and “strongly(dis)agree” are combined into one category. Then, stu-dent responses are compared to the expert-like response;a response that is aligned with the expert-like view is as-signed a +1 and a response that is opposite to the expert-like view is assigned a −
1. All neutral responses are as-signed a 0. For our comparison between the traditionaland transformed courses, we will report the percentageof students with expert-like responses.In DL1 and DL2, the E-CLASS was administered asan online extra credit assignment both pre- and post-instruction. Throughout the course transformation, DL1and DL2 collected a total of 1,377 and 925 students, re-spectively, with matched (both pretest and post-test) E-CLASS scores. Figure 6 shows the fraction of studentswith expert-like responses in the traditional course andthe transformed course for (a) DL1 and (b) DL2. Stu-dents in the traditional courses had a decrease of 3% and and 1%, respectively, in their expert-like attitudes andbeliefs toward experimental physics from pre- to post-instruction. However, in the transformed DATA Labcourses, the students’ expert-like views of experimen-tal physics increased by 4% in DL1 and by 6% in DL2.To explore the impact of the course transformation af-ter controlling for students’ incoming epistemology andexpectations about experimental physics, ANCOVA wasused to evaluate the student’s attitudes and beliefs post-instruction between the traditional courses and the trans-formed courses. For both DL1 and DL2, results showedthat there was a significant difference in ECLASS post-test percentages between the traditional courses and thetransformed courses ( ps < . FIG. 6. Fraction of students with expert-like responses for(a) DL1 and (b) DL2.
VIII. CONCLUSION
In this paper, the large scale transformation of theMSU algebra-based physics labs for life science studentswas described. The design was divorced from the spe-cific physics content because the learning goals developedfrom a faculty consensus design did not include specificcontent. This design means that the individual lab ac-tivities do not matter per se , but instead the structureof the course and how students work through the lab arewhat is important. Theoretically, one could adapt thisdesign to a chemistry or biology lab by making adjust-ments to the kinds of lab activities, and relevant changesto the learning goals. That being said, there are still keystructures to ensure the functioning of the course whichwill be covered in detail in a subsequent paper (e.g. aleadership team of four instructors, two GTAs and twoULAs, tasked with maintaining consistent grading andinstruction across the sections).The transformation was centered to emphasize experi-mental physics practices. The overall efforts were focusedon the two course series because the majority of the stu- dents that are taking courses in the physics departmentat MSU are enrolled in the introductory algebra based se-ries, specifically 2,000 students per year. In addition, themajority of the student instructors in the MSU physicsand astronomy department, nearly 80 graduate teachingassistants and undergraduate learning assistants, teachin these labs. Because of its scale, special attention wasgiven to the voice of the physics faculty in the devel-opment of the learning goals for DATA Lab [10]. Theentire course was designed around the faculty-consensuslearning goals, which are all based around physics labo-ratory practices (Sec. II). From course structures to as-sessments, everything was intentionally aligned with theoverall learning goals. Each component of the coursebuilds upon another through the two semester sequence.Each individual lab activity builds upon skills that willbe valuable for each subsequent activity, from lab hand-outs to pre-class homework assignments. Such an effortwas put into designing this course sequence in large partbecause of the number of MSU undergraduate studentsthey are serving. The value in physics labs for these non-majors lies in the scientific practices on which the re-design was centered. Those skills and practices are whatthey will take with them into their future careers.
ACKNOWLEDGMENTS
This work was generously supported by the HowardHughes Medical Institute, Michigan State University’sCollege of Natural Science, as well as the Department ofPhysics and Astronomy. The authors would like to thankthe faculty who participated in the discussion of learninggoals. Additionally, we would like to thank S. Beceiro-Novo, A. Nair, M. Olsen, K. Tollefson, S. Tessmer, J.Micallef, V. Sawtelle, P. Irving, K. Mahn, J. Huston whohave supported the development and operation of DATALab. We also thank the members of the Physics Educa-tion Research Lab who have given feedback on this workand this manuscript. ∗ [email protected] † [email protected][1] J. Kozminski, H. Lewandowski, N. Beverly, S. Lindaas,D. Deardorff, A. Reagan, R. Dietz, R. Tagg, J. Williams,R. Hobbs, et al., AAPT Recommendations for the Un-dergraduate Physics Laboratory Curriculum (2014).[2] N. Holmes, J. Olsen, J. L. Thomas, and C. E. Wieman,
Value added or misattributed? a multi-institution studyon the educational benefit of labs for reinforcing physicscontent , Physical Review Physics Education Research ,010129 (2017).[3] S. Olson and D. G. Riordan, Engage to Excel: ProducingOne Million Additional College Graduates with Degreesin Science, Technology, Engineering, and Mathematics.Report to the President. , Executive Office of the Presi- dent (2012).[4] E. Etkina and A. Heuvelen,
Investigative Science Learn-ing Environment: Using the processes of science and cog-nitive strategies to learn physics (2001).[5] J. M. Wilson,
The CUPLE physics studio , The PhysicsTeacher , 518 (1994).[6] M. Dancy and C. Henderson, in National Academies ofScience Promising Practices Workshop (2008).[7] B. M. Zwickl, T. Hirokawa, N. Finkelstein, andH. Lewandowski,
Epistemology and expectations surveyabout experimental physics: Development and initial re-sults , Physical Review Special Topics-Physics EducationResearch , 010120 (2014).[8] S. V. Chasteen, K. K. Perkins, P. D. Beale, S. J. Pollock,and C. E. Wieman, A thoughtful approach to instruction: Course transformation for the rest of us , Journal of Col-lege Science Teaching , 24 (2011).[9] S. V. Chasteen, S. J. Pollock, R. E. Pepper, and K. K.Perkins, Transforming the junior level: Outcomes frominstruction and research in E&M , Physical Review Spe-cial Topics-Physics Education Research , 020107 (2012).[10] C. Wieman, Improving how universities teach science:lessons from the science education initiative (HarvardUniversity Press, 2017).[11] J. Biggs,
Enhancing teaching through constructive align-ment , Higher education , 347 (1996).[12] V. Otero, S. Pollock, and N. Finkelstein, A physics de-partments role in preparing physics teachers: The Col-orado learning assistant model , American Journal ofPhysics , 1218 (2010).[13] P. W. Irving, M. J. Obsniuk, and M. D. Caballero, P : apractice focused learning environment , European Journalof Physics , 055701 (2017).[14] B. R. Wilcox and H. Lewandowski, Students epistemolo-gies about experimental physics: Validating the colorado learning attitudes about science survey for experimentalphysics , Physical Review Physics Education Research ,010123 (2016).[15] D. Hu, B. M. Zwickl, B. R. Wilcox, and H. Lewandowski, Qualitative investigation of students views about experi-mental physics , Physical Review Physics Education Re-search , 020134 (2017).[16] B. R. Wilcox and H. Lewandowski, A summary ofresearch-based assessment of students’ beliefs about thenature of experimental physics , American Journal ofPhysics , 212 (2018).[17] W. K. Adams, K. K. Perkins, N. S. Podolefsky, M. Dub-son, N. D. Finkelstein, and C. E. Wieman, New in-strument for measuring student beliefs about physics andlearning physics: The colorado learning attitudes aboutscience survey , Physical review special topics-physics ed-ucation research , 010101 (2006). Appendix: Experimental Descriptions TABLE II. DL1 ExperimentsExperiment Main IdeaWorkshop 1 Taking a variety of measurementsand determining the uncertaintyWorkshop 2 Taking simple measurements,learning how to plot parametersin useful ways, and incorporatinguncertainty into analysis andresultsWorkshop 3 Using the example of making apeanut butter and jelly sandwichto evaluate procedures and discussproper documentationFree Fall Exploring free fall and terminal ve-locity with coffee filters(In)elastic Collisions Exploring elastic and inelastic col-lisions on an air trackStanding Waves Exploring waves on a string, nodes,and resonant frequenciesImpulse Exploring impulse with bouncingballs and relating that to concus-sionsEnergy Transfer Exploring energy loss of a bouncingballPendulum Exploring a simple pendulum5