How computation can facilitate sensemaking about physics: A case study
Odd Petter Sand, Tor Ole B. Odden, Christine Lindstrøm, Marcos D. Caballero
HHow computation can facilitate sensemaking about physics: A case study
Odd Petter Sand, Tor Ole B. Odden, Christine Lindstrøm, and Marcos D. Caballero
1, 2 Centre for Computing in Science Education (CCSE), University of Oslo, N-0316 Oslo, Norway Department of Physics and Astronomy and Create for STEM Institute, Michigan State University, East Lansing, MI 48823
We present a case study featuring a first-year bio-science university student using computation to solve aradioactive decay problem and interpret the results. In a semi-structured cognitive interview, we use this caseto examine the process of sensemaking in a computational science context. We observe the student enteringthe sensemaking process by inspecting and comparing computational outputs. She then makes several attemptsto resolve the perceived inconsistency, foregrounding knowledge from different domains. The key to makingsense of the model for this student proves to be thinking about how to implement a better model computationally.This demonstrates that integrating computation in physics activities may provide students with opportunities toengage in sensemaking and critical thinking. We finally discuss some implications for instruction.
I. INTRODUCTION
It is a well-known problem that students can progressthrough introductory physics courses, sometimes with goodgrades, and still lack understanding of the underlying prin-ciples, relations, and concepts. A common scenario is thatstudents employ “plug and chug” strategies to manipulatemathematical formulae without engaging with the underly-ing physical principles. With this in mind, getting students toengage in sensemaking is crucial for achieving learning goalsin critical thinking and understanding the physics itself [1].Computation is important for students of physics to learnbecause it reflects current practices in the field, teaches impor-tant skills for research and other careers, and allows studentsto solve a greater number of more realistic problems [2]. Con-sequently, research-based efforts to sensibly integrate compu-tation into the physics curriculum are well underway [3–5].Therefore, we want to study to what extent computation pro-vides a potential for students engaging in sensemaking, andunder which conditions that potential may be fully realised.We present evidence for sensemaking in the case of Sophia,a bio-science student who is interviewed while solving aphysics problem on radioactive decay. Sophia uses both com-putational and non-computational arguments to make senseof the model. The process of modifying her program andcomparing the outputs turns out to facilitate Sophia’s sense-making. We justify this claim by presenting evidence for howcomputation was helpful in the sensemaking process. Finally,we discuss implications for teaching and future research.
II. ANALYTICAL FRAMEWORK
The analytical framework for this study is founded on thefollowing definition of sensemaking from [6], pp. 5-6: “A dy-namic process of building or revising an explanation in orderto [. . . ] resolve a gap or inconsistency in one’s understand-ing.” While there have been numerous other attempts to de-fine what sensemaking is, we chose this one because it unifiesseveral aspects of sensemaking that others have highlighted:sensemaking as an epistemological frame, a cognitive pro- cess, and a discourse practice, all of which are relevant to thisproject.The process of sensemaking involves (a) realising thatthere is a gap or contradiction in one’s knowledge, (b) iter-atively proposing ideas and attempting to connect them to ex-isting knowledge or other ideas, and (c) evaluating that theseideas are consistent and do not lead to additional contradic-tions [6]. In this paper, we will use this definition to studyhow computational activities may provide opportunities forsensemaking in interdisciplinary science problems.
III. METHODS
The case comes from a pilot study conducted with first-year bio-science students at a large research-intensive uni-versity in Norway. These students learned computation in-tegrated with biology in the previous semester and were fol-lowing a physics course in the semester when this study tookplace. The physics course had not yet covered radioactivedecay by the time we interviewed the students. We targetedstudents with a wide range of self-reported programming ex-pertise who were also comfortable thinking aloud.Subsequently, we performed a series of semi-structuredcognitive interviews in Norwegian where students worked onthe task alone. The interviews borrowed heavily from think-aloud protocols, but students could ask for help with syntaxshould they need it, provided they were able to articulate whatthey wanted the code we gave them to do.Follow-up questions on students’ reasoning were asked bythe interviewer on various occasions, interspersed throughoutthe think-aloud segments. This tends to change the students’thought processes, often improving the results. Protocols ob-tained in this way tend to be more valid than the ones werestudents recall their reasoning after the fact, however [7].We gave the interviewees a toy model starting off with1000 radioactive nuclei and told them that 10% of the re-maining nuclei would decay every month. The students firstcalculated the remaining number of nuclei for the first twomonths (where the answers were still integers) by hand. Thenext step for the students was to reproduce these answers by a r X i v : . [ phy s i c s . e d - ph ] N ov riting a Python program in Jupyter Notebook. This is thefamiliar programming environment they used throughout theprevious semester. Finally, they were asked to extend the cal-culations to 60 and 100 months and (if time allowed) plot theresults.This task was specifically designed to allow students to dis-cover a perceived trade-off between accuracy and realism thatwould require sensemaking to resolve. After a while, youneed several decimal points to mathematically describe 10%of what remains, yet when counting nuclei, in general oneexpects the numbers to be integers. While the toy model weprovided may be approximately correct for a large numberof nuclei, at lower amounts one would have to interpret theoutput as an average across many identically prepared exper-iments for the numbers to make sense.All the students interviewed (N=5) at some point consid-ered rounding the answers to the closest integer to avoidworking with fractions of nuclei, although some did this onlyin response to follow-up questions from the interviewer. Ev-ery student also expressed some concern about the mathemat-ical accuracy of their results when rounding the numbers inthis way. Two of the interviewees made some progress towardresolving this contradiction by interpreting the un-roundednumbers as an average, one of which was Sophia.The typical length of an interview was about one hour. Allinterviews were recorded on audio and video, both of the stu-dent and the computer screen. Subsequently, the transcriptswere translated from Norwegian into English. We analysedthe transcripts using the definition in [6] and looked for thefollowing: The student (a) realising she cannot fully explainthe physical phenomenon she is modelling or aspects of themodel itself, (b) proposing explanations and trying to connectthem to scientific or everyday knowledge and (c) evaluatingthese explanations to ensure consistency.We then looked at what the student was doing with compu-tation inside and outside of these sensemaking episodes, andasked the following questions: What happens in this compu-tational context when the student engages in sensemaking? Isthe computational aspect of the task a help or hindrance tothis process?The case we present illustrates how sensemaking may hap-pen in a computational context. While not the most typicalcase for this group of students, Sophia’s interview was cho-sen for analysis because her sensemaking was rather explicitin the transcript. Additionally, she ended up using languagethat was clearly computational to make a profound argumentabout how to model the physical phenomenon and interpretthe results. IV. COMPUTATIONAL SENSEMAKING CASE “Sophia” (pseudonym) is a Norwegian student in her mid-20s, a few years older than most students taking first-yearuniversity courses. She describes her experience with pro-gramming as one of a fair degree of mastery in most cases. Compared to the average student in the programming coursefor bio-science students, she comes across as more confidentand relaxed than most when working with computer code.During the interview, she rarely asked for confirmation thatshe was on the right track, and she did not hesitate long be-fore trying something out.We begin our analysis at the point where Sophia has set upher program to calculate the number of remaining nuclei forthe first three months: 1000, 900.0 and 810.0, respectively.
Sophia [14:35]
There. Now it’s right. [But] nowI might want to round these [indicates 900.0 and810.0] to get. . . well, just whole numbers.
She implements this rounding to the closest integer whendisplaying the output from the program, but not in the actualcalculations, and checks that it works.
Interviewer [15:05]
Could you tell me a littlemore about why you would want to round them?
Sophia
Because these are atoms, and you sort ofcan’t have half. . . or I don’t know. . . it seemsa little unnecessary to include, like, 810.0 atoms,in a way.
We interpret “you sort of can’t have half. . . ” as that youcannot have a fraction of a nucleus and still call it a nucleusof that particular element, which is a point Sophia returns tolater on.At this point, we have reached the start of the sensemakingprocess. It is divided into three separate segments that cor-respond to the three ideas Sophia proposes to make physicalsense of the numbers given to her by her program.
A. Sensemaking segment I
Sophia moves on to the next part of the task, modifying herprogram to repeat calculations all the way up to 60 months.She inspects the output and indicates the last ten months inthe sequence, with 3, 3, 3, 2, 2, 2, 2, 2, 1, and 1 nucleus,respectively.
Sophia [16:30]
This looks a little strange. . . Be-cause here there are no decimals. So. . . here I’dinclude the decimals because, like. . . you can’ttake 10 percent of. . . or, I get that you get, like,the same number several months in a row. [indi-cates the earlier sequence 6, 6, 5, 5]
Because 10percent of 6 is still above 5, like. I’m going toinclude the decimals.
While cutting the decimals for large numbers seems fine toher, Sophia realises that for smaller numbers there is some-thing she needs to find an explanation for. Why does thenumber of nuclei remain constant for several time steps andthen changes more than 10% rather abruptly? In terms of ourensemaking framework, the sensemaking process thus startsin reaction to the computational output when she realises thatsomething is “a little strange” .Using computation also allows her to include the decimalsand test this change, which she immediately does. Yet, interms of our framework, the idea that Sophia proposes hereis first and foremost mathematical. She talks about numbersin a sequence, decimals and percentages, but this discussionstands on its own removed from the physics and computa-tional contexts it occurred in.
B. Sensemaking segment II
After resolving some bugs (one syntax error and a few log-ical errors), Sophia sees the un-rounded numbers for all 60months. After verifying that they seem to be the correct num-bers mathematically, she is told that she is free to move on tothe next task. Despite this suggestion, she decides she wouldrather continue making sense of the model.
Sophia [20:18]
Umm, yes. Right now, I’m think-ing – I just have to say it, because right now Iam a little unsure about. . . because there arenow so many decimals and. . . [indicates the finalmonths with 2.21. . . , 1.99. . . and 1.79. . . nu-clei] because one atom can’t. . . you can’t take10 percent of one atom, like. So, this becomessort of random whether, in a way. . . whether itsplits or, like, if it loses one atom to radioactiv-ity or not. So, I’m really not entirely happy withthese numbers. But I can move on to the nextone, I guess.
We interpret this as Sophia revisiting her earlier statement:Can you have a fraction of a nucleus? The outstanding featureof this segment is the critique of her previous choice, whichaccording to our framework is indicative of sensemaking go-ing on.Initially, Sophia seems hesitant to exit the sensemakingprocess prematurely, and she may be experiencing some fric-tion between the sensemaking and how she frames the inter-view situation. The initial “
I just have to say it ” at 20:18seems to indicate that at that point she was about to engage inan activity she considered not wholly appropriate for the wayshe was framing the activity at the time [8].One should also note that in contrast to the previous sense-making attempt, this one foregrounds the ideas from physics(atoms, radioactivity) with a nod to the mathematics embed-ded in them (percentages, randomness).
C. Sensemaking segment III
At this point the interviewer intervenes and invites Sophiato discuss a little more why she is not happy with the num-bers, in effect sustaining the sensemaking frame. Initially this invitation is met with minor resistance, possibly because itwas suggested she move on in segment II. Sophia states thatshe does not want to spend so much time and energy think-ing about an open-ended task which is not clear about what itwants from her, so she is “choosing the easy way out” . Afterbeing asked what she would do if she were a scientist and thiswas an important result to her, Sophia resumes the sensemak-ing process:
Sophia [23:20]
So, already after the third monthhere, then I would have taken, like, [indicatesmonth 4 with 656.1 nuclei] here it reads point 1 –then I might have put in a for loop with choice?
Ithink it is [random.choice() ] you use. Whetheror not, like, that one. . . like, whether the deci-mal, whether that is a whole atom that goes awayor not. So, in a way it becomes a sort of choice. . .thing. Such that when you run it as a model forthe first time, then maybe. . . yes. Then maybeall. . . eh, the radioactive atoms are spent after,like, 56 months. . . and then the next time theyare spent after 60 months. And the time after thatmaybe after 70 months. Eh, and then I would. . .yes, then I would have made a program or maybea def- function and then run that many times andlook at, percentage-wise, then, how probable is itthat, eh, all the atoms. . . yeah, are gone after 50months or after 70 months. So, I’d rather makethat kind of model, because. . . eh, you kind ofcan’t make this [indicates the output] completelyaccurate... But at the same time, when I thinkabout it, it is. . . the probability of when that isgoing to happen is a little present in these num-bers, too. At this point Sophia is using computation as a tool forsensemaking, something that was not explicitly evident inher earlier attempts. The mathematics and physics are stillpresent in the background. Referring back to our frameworkonce again, Sophia did mention randomness in segment II, yetthis is the first time she proposes to interpret each un-roundednumber as an average. But critiquing that idea leads to thequestion: an average of what? Of different simulations. “Iwould have made a program [. . . ] and then run that manytimes and look at, percentage-wise, then, how probable is itthat [. . . ] all the atoms [. . . ] are gone after 50 months or after70 months.”
We claim that this point, firmly embedded in thecomputational nature of the task, is key for Sophia’s bridgingthe gap in her understanding she has been wrestling with.As opposed to the simplified difference equation she wasworking with originally, the approach suggested here incor-porates randomness: two sets of 1000 nuclei would not nec-essarily decay in identical ways. This realisation does not https://docs.python.org/3/library/random.html ean she has a complete idea of how to implement it compu-tationally, but sensemaking is about how you get there.In summary, we have identified three sensemaking seg-ments, in which Sophia foregrounds knowledge from the fol-lowing domains: • Segment I: Mathematics • Segment II: Physics • Segment III: ComputationThese segments together clearly demonstrate the sense-making process: Sophia (a) realises that rounding the num-bers hides information. It seems inaccurate that the numberof nuclei appears unchanged for several time steps and thenabruptly changes significantly more than 10%. But not round-ing the numbers leads to working with fractions of a nucleus,which conflicts with her intuition about how the world works,as established prior to segment I. In each segment Sophia (b)iterates by proposing ideas and (c) critiquing these to makesure they are consistent in themselves and with other ideas.The sensemaking process ends with the resolution ofchanging the interpretation of the numbers in the toy model.Instead of the actual number of nuclei in one experiment theyrepresent an average across an ensemble of computationalsimulations. At this point, we interpret Sophia’s statementsto mean that she regards both integers and decimal numbersas valid outputs from her program. She has also attained arough idea of how to implement the simulations in question.
V. DISCUSSION AND CONCLUSIONS
In this paper, we have shown that computation helpedSophia in two ways. First, she was able to modify her pro-gram back and forth between rounding and no rounding withrelative ease. In the first two sensemaking segments, inspect-ing and comparing the outputs of these approaches providesan entry point into the sensemaking process: “This looks alittle strange. . . ”
Second, we argue that the key to Sophia’s interpretation ofher output as an average is to think computationally about theproblem, which is what happens in segment III. When dis-cussing how to implement a more realistic model computa-tionally, she realises that her current results can be interpreted as an average of several such simulations: “The probability ofwhen that is going to happen is a little present in these num-bers, too.”
Without claiming that this case is common or representa-tive for this group of students, we argue that this case studyprovides an existence proof that computation can provide fer-tile ground for student engaging in sensemaking. Specifically,working computationally allowed Sophia to (a) realise a gapin her understanding, (b) implement ideas and (c) test and cri-tique the results for consistency. We observed that in this con-text, the idea that drew most heavily on computational knowl-edge proved the most fruitful in the sensemaking process.To determine under which circumstances this potential forsensemaking can be fulfilled, further research is needed. Inthe other four interviews, we did note other examples of stu-dents beginning to engage in sensemaking in response to theoutput of their programs. What is special about Sophia’s casewas the way her computational resources helped her makesense of the apparent contradiction between the physics (re-alism) and mathematics (accuracy) in the model. She also ini-tially ignored the interviewer’s suggestion to move on at thestart of segment II. It remains to investigate how this wouldplay out in a classroom setting, where there is no interviewerto help sustain the sensemaking process like at the start ofsegment III. Video observations is one possible way to probethis.Future studies could also identify the thresholds for en-tering and successfully resolving a sensemaking process, re-spectively, using computation. This would have profound im-plications for how instructors integrate computation in sci-ence classes, for instance when designing tasks that go be-yond procedural use of computer programming as a tool. Ifcritical thinking is important to us, we should attempt to re-alise the full sensemaking potential in computational activi-ties. It is then necessary to ensure that our students have suf-ficiently strong computational foundations to engage in thesesensemaking tasks.
ACKNOWLEDGMENTS
This study was funded by the Norwegian Agency for Qual-ity Assurance in Education (NOKUT), which supports theCentre for Computing in Science Education. [1] D. Maloney,
Phys. Teach. , vol. 53, no. 7, pp. 409–411, Sep.2015.[2] https://aapt.org/Resources/upload/AAPT_UCTF_CompPhysReport_final_B.pdf. Retrieved 06-Jul-2018.[3] M. D. Caballero, M. A. Kohlmyer, and M. F. Schatz,
Phys. Rev.Spec. Top. - Phys. Educ. Res. , vol. 8, no. 2, p. 020106, Aug.2012.[4] R. Chabay and B. Sherwood,
Am. J. Phys. , vol. 76, no. 4, pp.307–313, Mar. 2008. [5] A. Buffler, S. Pillay, F. Lubben, and R. Fearick,
Am. J. Phys. ,vol. 76, no. 4, pp. 431–437, Mar. 2008.[6] T. O. B. Odden and R. S. Russ,
Sci. Educ. , Jun. 2018.[7] M. W. van Someren, Y. F. Barnard, and J. A. C. Sandberg, Aca-demic Press, Inc, 1994.[8] R. S. Russ, V. R. Lee, and B. L. Sherin,