Applying a resources framework to analysis of the Force and Motion Conceptual Evaluation
AApplying a resources framework to analysis of the Force and Motion ConceptualEvaluation
Trevor I. Smith and Michael C. Wittmann
Department of Physics and Astronomy, University of Maine, Orono, ME 04469
We suggest one redefinition of common clusters of questions used to analyze student responses onthe Force and Motion Conceptual Evaluation (FMCE). Our goal is to move beyond the expert/noviceanalysis of student learning based on pre-/post-testing and the correctness of responses (either onthe overall test or on clusters of questions defined solely by content). We use a resources framework,taking special note of the contextual and representational dependence of questions with seeminglysimilar physics content. We analyze clusters in ways that allow the most common incorrect answersto give as much, or more, information as the correctness of responses in that cluster. Furthermore,we show that false positives can be found, especially on questions dealing with Newton’s Third Law.
PACS numbers: 01.40.Fk, 01.40.gf
I. INTRODUCTION
Students are not yet physicists. They have not hadthe extensive training that we, as physicists, rely on.As a result, it is often inappropriate to categorize andgroup student responses to physics questions solely onthe basis of agreeing with correct Newtonian principles,as is commonly done with standardized tests such asthe Force Concept Inventory (FCI) and the Force andMotion Conceptual Evaluation (FMCE).[1, 2, 3] Unfor-tunately student assessment using the FMCE, includingwork previously done by one author (M.C.W.), regularlydoes just that.[4] The test questions are grouped intoclusters according to a physicist’s view of equivalent con-tent areas, and students’ responses are evaluated basedon their agreement with a physicist’s viewpoint withoutregard for why students might choose incorrect answers.This may be a valid form of assessment for determininghow well students think like physicists, but it is oftenan insufficient method for determining how students rea-son about scenarios in a physics context. In order toeffect greater conceptual development in our students,we must understand not only where we wish them to endup, but also where they are beginning in terms of theirunderstanding of the world around them. Only by havingthis entire picture may we devise a manner by which tohelp our students truly gain a physicist’s understandingof their surroundings.This paper describes ways in which the FMCE canbe organized and used to get more detailed informationabout students. Presently, many researchers and edu-cators using the FMCE follow a particular procedurethat includes three steps: 1) administering the FMCEpre- and post-instruction, 2) dividing the questions intocontent-based clusters, and 3) evaluating the correctnessof each student’s responses within each cluster as well asover the entire test. Several years ago a template wasdeveloped by one author (M.C.W.) to analyze students’responses to the FMCE within five clusters (Velocity, Ac-celeration, Force (Newton’s 1 st and 2 nd laws), Newton’s3 rd law[5], and Energy).[4] The template automatically scores each response as correct or incorrect, groups ques-tions into the aforementioned clusters, and calculates aclass’s normalized gain for each cluster as well as over theentire test. This template has become widely used dueto its availability and its succinct analysis of students’responses. Recent research using the FMCE and model-ing using a resources framework[6, 7, 8] has convinced usthat analysis based on the use of this template lacks thedepth and rigor we have come to expect from studies onstudents’ understanding of Newtonian mechanics. Wepropose several modifications to the described analysisincluding a redefinition of clusters and a deeper analysisof students’ incorrect responses.The clusters mentioned previously divide the FMCEvery nicely into groups of questions that each exam-ine a different content aspect of physics. But manystudies, including those by Beichner[9] and Kohl andFinkelstein[10, 11], have shown that the manner in whichmaterial is presented may significantly affect students’abilities to demonstrate their understanding. For exam-ple, Beichner has shown that students’ understanding (orlack of understanding) of graphs can have a profoundimpact on their responses to physics questions involvinggraphs.[9] Furthermore, results by Dykstra and othersshow that elements of the physical situation greatly affectreasoning. Dykstra reports on students’ troubles reason-ing about motion in which an object has a turning point;that is, when an object under the influence of a constantforce moves in a particular direction while slowing downand then reverses direction and speeds up.[12]Our goals in this paper are not new. Several re-searchers (notably Thornton[13, 14] and Dykstra[12])have used results from the FMCE to give fine-grainedanalyses of students’ responses to the FMCE. Unfortu-nately, their methods are not widely used among physicseducators and education researchers. Also, we wish toanchor our analysis in a resources framework, making ex-plicit connections to the representational and contextualdependence of responses. In addition, Bao has proposednew clusters of questions to investigate reasoning. Werespond to Bao’s work in more detail in section II. a r X i v : . [ phy s i c s . e d - ph ] M a r In section II we both respond to existing clusteringmethods (including our previous one) and propose newclusters. In section III we discuss various incorrect men-tal models that correspond with the content areas de-scribed by each of our clusters. In section IV we examinehow these mental models are aligned with particular re-sponses to questions in the FMCE. Using a definition ofclusters consistent with a resources framework allows usto go into greater detail about students’ incorrect mentalmodels and identify responses that correspond to thesemodels. In this section, we include a discussion of falsepositives, as measured by looking at responses on sev-eral questions within a representationally and contextu-ally consistent set of questions.
II. REVISED QUESTION CLUSTERS FOR THEFMCE
The five analysis clusters, shown in table I, were chosenby Wittmann as a quick and dirty analysis of classroomperformance on the FMCE. These clusters were definedbased on the content of each question on the FMCE—the Velocity cluster asks students about the velocity ofan object undergoing a series of described motions, theForce (Newton I & II) cluster asks students about theforce (s) exerted on an object during a described motion,and so on. Several questions are not included in anyof the clusters; Thornton and Sokoloff omit these fromregular analysis of the FMCE because they are intendedfor diagnostic purposes (such as reading ability) or donot give a definitive indication as to whether or not astudent is properly using Newtonian reasoning.[3] Moredetails of these omissions can be found in refs. [14, 15].
TABLE I: Clusters of questions on the FMCE as previouslydefined by Wittmann.Cluster QuestionsVelocity 40–43Acceleration 22–29Force (Newton I & II) 1–4, 7–14, 16–21Newton III 30–32, 34, 36, 38Energy 44–47
There are obvious flaws to the old clustering of ques-tions, the largest being that different representations andcontexts are asked about in many clusters. If studentsare inconsistent in their thinking about the physics (asassumed in a resources framework), then results in eachcluster should be noisy and inconsistent, as well.
A. Defining new clusters
To understand student reasoning better, we should usea finer-grain resolution in the questions that we analyze and group. We use the questions of the Force (Newton I& II) cluster, shown in figures 1–3, to illustrate.In answering the questions in figures 1–3, studentsmust determine the force on an object (sled, toy car,coin) undergoing a described motion. In terms of repre-sentations, figures 1 and 3 use pictorial representations,while figure 2 uses graphical representations. In terms ofcontext, the questions in figure 1 and 2 are about mo-tion in a single direction, while the questions in figure 3involve an object that reverses the direction.
FIG. 1: Questions 1–7 of the FMCE[3]: contained withinoriginal Force (Newton I & II) cluster and revised Force Sledcluster.
To measure dependence of student reasoning on therepresentational and contextual cues in figures 1–3, wehave created new clusters which replace the old Force(Newton I & II) cluster: the Force Sled cluster (contain-ing the questions in figure 1), the Force Graphs cluster(containing the questions in figure 2), and the Revers-ing Direction cluster (containing the questions in figure3 as well as others). The Reversing Directions cluster hasbeen expanded to include questions about acceleration aswell as force; questions 27–29 on the FMCE inquire aboutthe acceleration of a coin tossed in the air as it moves upand back down, isomorphic to questions 11–13 but in thecontext of acceleration.Table II shows the full definitions of the revised clus-ters, ordered by question number on the test rather thandifficulty of the physics material. These clusters are con-sistent with those used by Thornton.[13] Note that thedefinitions of the original Newton III, Velocity, and En-ergy clusters have been directly transferred to the revisedclusters. To conform with the specificity of the ForceGraphs and Acceleration Graphs clusters, the Velocity
FIG. 2: Questions 14–21 of the FMCE[3]: contained withinoriginal Force (Newton I & II) cluster and revised ForceGraphs cluster. cluster has been renamed the Velocity Graphs cluster.Most of the original Acceleration cluster has been trans-ferred to the Acceleration Graphs cluster (the other accel-eration questions are in the Reversed Direction cluster).These revised clusters increase the information that canbe taken from an analysis of the FMCE by highlightingand isolating content areas, types of representations, andspecific situations with which students may struggle. Wegive examples in section IV.
TABLE II: Revised clusters on the FMCE.Cluster QuestionsForce Sled 1–4, 7Reversing Direction 8–13, 27–29Force Graphs 14, 16–21Acceleration Graphs 22–26Newton III 30–32, 34, 36, 38Velocity Graphs 40–43Energy 44–47
B. Comparing to other clusters
Other ways of clustering questions on the FMCE exist.In his Ph.D. dissertation, Bao claims that mixed model
FIG. 3: Questions 8–13 of the FMCE[3]: contained withinoriginal Force (Newton I & II) cluster and revised ReversingDirection cluster. states are more easily detected using samples of questionsthat span several physical contexts.[16] As such he hasdefined the cluster of questions in table III to comparestudents’ use of two particular models (force proportionalto acceleration, and force proportional to velocity). Hiscluster contains questions from both the Force Sled (ques-tion 2) and Reversing Direction (questions 11 & 12) clus-ters as well as question 5, which Thornton and Sokoloffsuggest should only be used as a measure of reading abil-ity rather than Newtonian thinking.[15]Using a resources framework, we can explain the pres-ence of mixed model states as being the result of con-textual cues or representational cues. Thus, by creat-ing a cluster that mixes cues, he has primed his data toshow mixed model states while not being able to explainthe source of model mixing, be it contextual, represen-tational, or due to deeper issues with the physics. Ofcourse we expect our students to understand concepts inall contexts, but mixing cues makes our analysis of stu-dent reasoning much more difficult and fails to give theresolution that we, as researchers and instructors, desire.A further weakness of the mixed context cluster is theuse of question 5, which describes two kinds of motion(accelerating followed by constant velocity), and shouldbe checked against question 2 on the FMCE for consis-tency. Typically, students score very well on this question Table III also shows the response to each question that corre-sponds with each of these models. (unless they have reading problems in understanding thequestion). Thus, adding this question necessarily skewsthe data toward correct model use.We believe that it is far more beneficial for instructorsto first group questions that deal with a single physicalcontext (e.g. constant force applied to move an objecthorizontally) before combining questions across diversecontexts. Our approach is more consistent with the as-sumptions of a resources framework, and gives new in-sight (as described below in the section on false positives).We do still observe students’ use of mixed models, butwithin a single physical context. Such a mixed modelindicates a kind of inconsistency in thinking about thephysics that a mixed-context cluster cannot.
TABLE III: Question cluster defined by Bao to use ModelAnalysis with the FMCE.[16]QuestionNumber F ∝ ∆ v ∆ t F ∝ v Other2 D B others5 D B others11 A G others12 A D others
III. FACETS, RESOURCES AND MENTALMODELS OF STUDENT REASONING ON THEFMCE
As we have stated from the outset, we use a resourcesframework to cluster responses on the FMCE. We arenot using a conceptions approach[14, 17, 18] because weare seeking a higher resolution to understand what stu-dents have mastered in their physics learning (and howbest to help those who have not yet mastered the ma-terial). The resources framework can be thought of as aschema theory that emphasizes knowledge-in-pieces, suchas phenomenological primitives (p-prims)[19], facets[7],and resources[8, 20]. The differences and connectionsbetween a concept-based and resource-based analysis ofstudent reasoning and teaching are discussed in more de-tail by Scherr.[21]Student thinking is rarely described in terms of an in-dividual, small idea such as “closer means stronger”, forwhich appropriate applications are sitting by a fire tobe warmer or moving from speakers at a concert to saveone’s hearing, and false applications include attributingthe change in seasons to a difference in the distance fromthe earth to the sun. Instead, succinct descriptions ofstudent thinking often require us to recognize the use ofmultiple resources in connection. We represent this as atype of nodal mental space network, a resource graph.[8]The connections, or links, between these resources varygreatly in both strength and duration. Assuming thatstudents are using a set of resources related to mechan-ics and kinematics, not all activated in every question, we may examine how various resources could combine to cre-ate robust (and often incorrect) mental models (or con-cepts) that students use when reasoning about physics.We give several examples below.In this section, we describe the resources that studentsoften use when answering questions on the FMCE. Insome cases, we draw resource graphs. In section IV, weconnect these descriptions of resources to the questionson the FMCE.
A. Dynamics
1. Newton’s 1st and 2nd laws
The notion that the force exerted on an objectis proportional to its velocity has been reported bymany researchers and is very prevalent among physicsstudents.[14, 22, 23] The F ∝ v model has been describedas being similar to the Impetus view of physics[23] butcan be described in more detail by several of Hammer’sresources including “activating agency” and (in partic-ular) a “maintaining agency”[6] that is “dying away.” The “activating agency” resource is the notion that ev-ery event must have a cause, i.e. every object that is inmotion must have had something to get it started. The“maintaining agency” resource embodies the idea thatobjects in motion must have something (some “agent”)to keep them in motion. While neither of these resourcesis incorrect in and of itself, the F ∝ v model is evidentwhen the “agent” required for each of them is seen as theforce applied to an object. The “maintaining agency”when used in this context contradicts Newton’s first law,but students’ intuitive ideas are not unreasonable. Theyare, instead, consistent with years of experience in ourfriction-filled world where a continuous application offorce is almost always needed to maintain an object’smotion.
2. Newton’s 3rd law
Studies have shown that students use a variety ofstrategies when reasoning about the forces exerted be-tween two interacting bodies.[24, 25, 26] Bao, Hogg, andZollman identified four “contextual features” that stu-dents use when responding to questions regarding New-ton III: velocity, mass, pushing and acceleration.[24] Forexample, an object with a larger initial speed will ex-ert more force than an object with a smaller initial These and many other resources are lightly derived from diSessa’sp-prims.[19] One could argue that this is only a problem if the students usethe net force for each of these resources; however, the fact thatthe FMCE presents questions that involve a single applied forceon a frictionless surface makes this a moot point. speed (during a collision), and more massive objects ex-ert more force than less massive ones. The velocityand mass features work well together to illuminate stu-dents’ implicit confusion between momentum and veloc-ity. Based on their ideas about kinematics, students oftenhave a desire to represent force as F = mv ;[23] further-more, students may use the terms momentum and forceinterchangeably.[24] The pushing feature is contained inthe notion that, when one object pushes another, the ob-ject that is pushing must exert more force than the objectthat is being pushed. This idea is typically accompaniedby the reasoning that if both objects exerted the sameforce on each other, neither would move.We have previously reported on students’ use of threefacets when considering Newton’s third law: the massdependence facet, the action dependence facet, and thevelocity dependence facet.[25] These mental models cor-respond with Minstrell’s facets of knowledge,[7] in par-ticular facets 62 (The moving object or a faster movingobject exerts a greater force), 63 (The more active or en-ergetic object exerts more force), and 64 (The bigger orheavier object exerts more force). The mass dependencefacet has a direct correlation with the mass contextualfeature described by Bao, et al.[24] The action depen-dence facet combines the velocity and pushing contex-tual features described above to create a mental modelthat is applicable to both pushing and collision situa-tions. The velocity dependence facet describes students’use of force as an intrinsic property of an object, similarto momentum, agreeing with the velocity contextual fea-ture described by Bao, et al.[24, 25] Maloney uses manysimilar ideas to describe students’ use of a “dominanceprinciple” to reason about the interaction between twobodies.[26] Maloney’s “dominance principle” is also veryclosely related to diSessa’s “Ohm’s p-prim” as well asHammer’s “more is more” resource.[6, 19]The “more is more” resource might manifest itselfwithin students’ thinking of Newton’s third laws as aseries of connections: the more mass or speed an objecthas, the more damage it can do; the more damage anobject can do, the more force it must exert on any otherobject. Another connection that can be made along theselines is the idea that the more an object reacts after a col-lision, the more force must have been applied to it.[27] Inthis case, the “more is more” resource is used indirectlyto describe the object that is exerting the force on theobject in question.We draw a resource graph of student reasoning aboutNewton’s third law in order to summarize these com-ments. In figure 4 we show how four resources cancombine in groups of three to create the two observedmass and action dependencies. Note that the combina-tion of “More means more” and “Unbalanced (competi- Definitions of these facets can be found in refs.[7, 25]. Possibly tacit indicators of momentum or kinetic energy. tion)” can be interpreted as diSessa’s Ohm’s p-prim[19],in which more resistance (say, a mass in the way) requiresmore force for equal effect.
FIG. 4: The mass dependence and action dependence re-sources may be derived from universal primitives and obser-vations of a scenario.
B. Energy
The “more is more” resource discussed above may alsobe applied when discussing students’ views of the transferof energy. For this particular discussion we will use thescenario of a sled starting from rest at the top of anicy hill and sliding all the way down (as is seen in theFMCE). In this case, the “more is more” resource may beused quite nicely (and correctly) in stating that the moreheight a hill has, the more kinetic energy (and therebyspeed) the sled will have once it has reached the bottom.Students may connect “more is more” to other elementsof the problem, instead, including more steepness or morelength. Students might, for example, take the approachthat the steeper a hill is, the faster (or more energetic) thesled will be when it gets to the bottom. Students mostlikely take this idea from their own experiences slidingdown hills; the steeper hills are always more fun and getthem to high speeds sooner. They attach the “more ismore” resource to the acceleration of the sled, i.e. therate of change of the velocity varies with the slope of thehill but not the total change in speed.
IV. MENTAL MODELS EVIDENT IN EACHCLUSTER
We return to the revised clusters defined in sectionII, applying the resources presented in section III. Foreach cluster of questions, we examine the possible re-sponses to individual questions and determine which cor-respond with the use of correct Newtonian reasoningand which indicate the use of one of the mental mod-els discussed in section III. We will also compare theseresponses with the most common student responses re-ported by Thornton,[13] showing that the FMCE can beinterpreted in ways consistent with a resources frame-work. Furthermore, we will show that the use of a re-sources framework lets us conclude that some student re-sponses are actually false positives (i.e. correct responsesgiven for incorrect reasons).
A. Force Sled Cluster
The Force Sled cluster (see figure 1) asks questions inplain language (i.e. not graphically), has no reversing di-rection questions, and deals with a single applied forcethat is therefore also the net force on the sled. Offeredresponses on this cluster include the correct idea that thenet (and applied) force is proportional to its acceleration(or rate of change of velocity), as well as the notion thatthe net force on the sled is proportional to its velocity.Table IV shows the responses that correspond with eachof these models as well as the most common student re-sponses. The most common student responses found byThornton[13] are the same as those indicating a student’suse of the F ∝ v model. This similarity provides strongevidence that many students believe that the net forceon an object is proportional to its velocity, rather thanits acceleration. We interpret these results in terms ofresource activation, though this interpretation is, in thiscase, not necessary. TABLE IV: The “Force Sled” cluster on the FMCE.Question Most CommonNumber F ∝ ∆ v ∆ t F ∝ v Student Response[13]1 b a a2 d b b3 f c, g c4 f g g7 b e e
B. Reversing Direction Cluster
The Reversing Direction cluster (see figure 3) asksquestions in which an object has been tossed in the air(or rolled up a hill). Students must think about the netforce and the acceleration throughout its up-and-down,free-fall motion. Within the Reversing Direction clus-ter, the questions are broken into three sub-clusters (8–10, 11–13, and 27–29), each of which involves a singleobject undergoing an up-and-down motion. In each of Note that, while the ramp on which the toy car rolls in questions8–10 prevents it from truly being in free-fall, the up and downmotion of the car travelling under its own volition is analogousto the coin toss in questions 11–13.[13] these sub-clusters the students are asked about the forceexerted on the object (questions 8–13) or the accelerationof the object (questions 27–29) as it goes up (questions 8,11, and 27), at the highest point in its journey (questions9, 12, and 28), and as it comes back down (questions10, 13, and 29). According to Thornton and Sokoloff,student responses are only considered correct when allthree questions within a given sub-cluster are answeredcorrectly.[3] We expand on this point below.
1. A generalized force-in-direction-of-motion model
As with the Force Sled cluster, the Reversing Direc-tion cluster provides possible responses that corresponddirectly with the F ∝ v model (or a ∝ v model for ques-tions 27–29). For the questions in the Reversing Direc-tion cluster, however, it is beneficial to consider a general-ization of the F ∝ v model: the force/acceleration-in-the-direction-of-motion model. This more general model ig-nores the magnitude of the force or acceleration through-out each part of the motion and only describes the di-rection. Consider questions 11–13 (coin toss force ques-tions). A student using the F ∝ v model would indicatethat the force on the coin is upward and decreasing as thecoin goes up, zero at the top of its motion, and downwardand increasing as the coin comes back down. But what ifa second student thinks that the force is upward and con-stant while the coin travels up, but agrees with the firststudent on the other two questions? This student cannotbe considered as using the F ∝ v model, but may beclassified within the direction-of-motion model. In fact,our F ∝ v student may also be categorized as using thedirection-of-motion model. In this way the direction-of-motion model allows a broader classification of studentswho have similar, but not necessarily identical ideas.Our decision to used the generalized direction-of-motion model is data-driven. Research conducted by oneauthor (T.I.S.) suggests that considering the direction-of-motion model for the Reversing Direction cluster allowsmany more students to be classified into a common modelthan the F ∝ v model.[28] Incorporating the direction-of-motion model into the results from the Force Sled orForce Graphs clusters, however, did not add any signifi-cant information. We suspect this is due to the fact thatonly the Reversing Direction cluster includes scenarios inwhich an object moves in more than one direction dur-ing a single described motion. Moreover, the ReversingDirection scenarios do not provide information as to howthe speed of the object changes throughout its motion.Table V shows the responses that corresond with eachof the models described above as well as the most com-mon student responses. The most common student re-sponses correspond directly with the responses indicat-ing the use of one of our described models. We notethat Thornton only provided answers that indicated thedirection of the force, not its magnitude.[13] As such,there is no way to tell from his data the likelihood thata student used the F ∝ v model. Also, Thornton’s workonly looked at the questions pertaining to the force onthe object.[13] His results, however, were replicated byone author (T.I.S.) for questions 27–29 asking about theobject’s acceleration.[28]
2. False positives in vertical toss situations
We return to the point of requiring that students an-swer all three questions correctly (responses a–a–a withineach question triplet). Consider the pattern of “a–d–a”responses on a given question triplet.[28] This studentmight believe that a constant downward force is exertedon the object while it is moving both upward and down-ward, but that no force is exerted while the object is“stopped” at the apex of its motion. This line of reason-ing may come from difficulties distiguishing between in-stantaneous velocity and change in velocity (as it relatesto acceleration).[29] The student may use the reasoningthat since the ball has zero velocity, it is not moving;therefore, the acceleration is zero and the force exerted onthe object must also be zero by Newton’s second law. Forall of these reasons it is widely accepted that responsesto the questions in the Reversing Direction cluster mustbe examined in conjunction with one another rather thanindividually, otherwise a student with serious problemsunderstanding direction reversal will get 2/3 correct onthis question triplet.Furthermore, consider two students who give very simi-lar incorrect responses: Student 1 answers “g–d–b” (con-sistent with F ∝ v ) while Student 2 answers “g–d–a.”Student 2’s “a” response to the third question indicatesa constant downward force or acceleration. The moregeneral direction-of-motion model accounts for both setsof responses, though. Again, it would be inappropriateto give Student 2 a 1/3 correct score, even though answer“a” is correct. The correct response also distinguishes the F ∝ v model from the direction-of-motion model. We donot believe, though, that the direction-of-motion modelis 1/3 more correct than the F ∝ v model! For this rea-son, we agree with Thornton and Sokoloff that a studentshould not be considered correct any part of a sub-clusterunless that student answers correctly on all three ques-tions. We thereby avoid measuring false positives in theReversing Directions cluster. C. Force Graphs Cluster
The Force Graphs cluster (see figure 2) asks questionsabout motions sometimes identical in physics content tothose found in the Force Sled cluster, but differing in The F ∝ v model would require an increasing downward forcefor each of these questions. presentation. Students are provided with a description ofthe motion of a toy car and asked to select a graph thatdepicts the force exerted on the car. All of the correctresponses indicate that the applied force is either zero ornonzero and constant. Table VI shows how responses tothe questions in the Force Graphs cluster correspond withthe various mental models as well as the most commonstudent responses.As with the Force Sled cluster, the most common stu-dent responses for the Force Graphs cluster correspondalmost exactly with the responses indicating a student’suse of the F ∝ v model. We separate the clusters toobserve if students master the content of one cluster be-fore the other. Research suggests[9, 10, 11] that studentsdo not display as much knowledge when working withgraphs as when using descriptive language, and data fromthe FMCE support the separation of questions into twoclusters.[14] The answers to the physics depend on thecontext and format of the question. Philosophically, thissupports the use of a resources framework, which canaccount for differences in reasoning based on contextualand representational differences in resource activation.Note that in table VI we designate multiple responsesfor a single model on question 21. Students are askedabout the force on a car once it has been released (afterbeing pushed). On one hand, response “a” seems to fitperfectly with the F ∝ v model (see figure 2): the carmoves at a constant velocity so a constant force must beapplied. On the other hand, what if the students don’tignore friction (though they are explicitly told to do) oruse an “impetus/force dies away” model? In each of thesecases, the car would slow down at a (perhaps) steadyrate, indicating a positive yet decreasing force (response“h”). Both of these responses can be considered a F ∝ v model in the sense of the need for a “maintaining agency,”with response “h” including the use of the “dying away”resource.Also consider question 17, where research by one au-thor (T.I.S.) has shown that some students who primarilyuse the F ∝ v model will choose response “a” instead of“b” (the F ∝ v response). Response “a” is not entirelydifferent from “b” as it is congruent with the F ∝ v model in magnitude but not direction. This may corre-spond to a confusion between left and right as negativeand positive, indicating a difficulty with coordinate sys-tems rather than with forces. As such we have decidedto categorize response “a” to question 17 as indicativeof the F ∝ v model if the student displays use of thatmodel in other responses to this cluster. Strong empiri-cal evidence and the fact that question 17 is one of onlytwo questions in the Force Graphs cluster to describe amotion with constant velocity heavily influenced our de-cision to consider response “a” as corresponding to the F ∝ v model. TABLE V: The “Reversing Direction” cluster on the FMCE.QuestionNumber Constant DownwardForce or Acceleration Force or Acceleration inthe Direction of Motion
F, a ∝ v Most Common StudentResponse[13]8–9–10 a–a–a (e, f, or g)–d–(a, b, or c) g–d–b (e, f, or g)–d–(a, b, or c)11–12–13 a–a–a (e, f, or g)–d–(a, b, or c) g–d–b (e, f, or g)–d–(a, b, or c)27–28–29 a–a–a (e, f, or g)–d–(a, b, or c) g–d–b g–d–b aa The most common responses for questions 27–29 can be foundin Ref. [28].
TABLE VI: The “Force Graphs” cluster on the FMCE.QuestionNumber F ∝ ∆ v ∆ t F ∝ v Most CommonStudent Response[13]14 e a a16 a c c17 e a, b b18 b h h19 b d d20 g f f21 e a, h h, f, a
D. Acceleration Graphs Cluster
The Acceleration Graphs cluster is similar to the ForceGraphs cluster. Students are asked about the accelera-tion of a toy car undergoing various types of motion.Again, students must choose the graph they believe bestrepresents the acceleration of the car for each scenario.It should be noted that the parenthetical reminders of“(constant acceleration)” that are found in the questionsof the Force Graphs and Force Sled clusters are omit-ted from these questions. We examine the AccelerationGraphs cluster from the perspective of students’ difficul-ties distinguishing between the concepts of accelerationand velocity reported by Trowbridge and McDermott[29]to form a type of acceleration-proportional-to-velocity( a ∝ v ) model. The most common student responsesshown in table VII correspond closely with the a ∝ v model. TABLE VII: The “Acceleration Graphs” cluster on theFMCE.QuestionNumber a ∝ ∆ v ∆ t a ∝ v Most CommonStudent Response[13]22 a e e23 b g f24 c b b25 b f f26 c a a
We note a discrepancy between the a ∝ v modeland the most common response for question 23. This question asks students to choose the appropriate graphof acceleration vs. time for a car that “moves towardthe right(positive direction), slowing down at a steadyrate.”[3] Figure 5 shows responses “f” and “g” that cor-respond to the most common student response and the a ∝ v response, respectively. Visually, responses “f”and “g” are incredibly similar, with identical magnitudeslopes. They are also presented above one another (seefigure 5). Even though response “f” would only accu-rately fit the a ∝ v model for a car moving to the leftand speeding up (as in question 25), it is not surprisingthat students would choose “f” for question 23. Again, aswith question 17, the problem may indicate issues withcoordinate systems more than the relationship betweenacceleration and velocity. FIG. 5: Responses “f” and “g” for the Acceleration Graphscluster.
E. Newton III Cluster
The Newton’s third law cluster is the only cluster thatcommonly elicits two different incorrect student models:the mass dependence model and the action dependencemodel described above. Table VIII shows how the re-sponses to the questions in the Newton III cluster sepa-rate into these models.The most common student responses shown in tableVIII incorporate aspects of both the mass dependenceand the action dependence models. We see that the It should be noted that Thornton’s study (Ref. [13]) did not in-clude the questions from the Newton III cluster. The most com-mon student responses are those reported by one author (T.I.S.)
TABLE VIII: The “Newton III” cluster on the FMCE. *–Categorization for this response depends on the student’schoice for question 36. Most CommonQuestion Forces Mass Action StudentNumber Equal Dependence Dependence Response[28]30 e a XX a31 e a b e, f32 e a b b34 e XX b b36 a b c c38 a b* b*, c b most common set of responses shows more agreementwith the action dependence model(questions 32, 34, 36,and 38) than the mass dependence model(question 30).Response “b” as the most common response for question38 might seem a bit ambiguous (as it can be classified aseither mass or action dependence), but one can use re-sponse “c” for question 36 to then categorize both as usesof the action dependence model. As with much of ouranalysis, this requires the assumption that students arereasoning relatively consistently from question to ques-tion.
1. False positives in collision situations
The assumption of consistent reasoning has seriousconsequences when one considers question 31. Many stu-dents who answer “e” on question 31 (the correct answer)answer incorrectly on questions 30, 32, and 34. (seefigure 6).We can infer with a fair degree of certainty why moststudents respond the way they do to questions 30 and32. In question 30, both vehicles are moving at the samespeed before the collision (making action dependence amoot point), but the truck is much heavier than the carcausing students to lean heavily toward mass dependencereasoning. In question 32, the truck is still much heav-ier than the car, but it isn’t initially moving. As such,the greater “activeness” of the car wins out and studentsuse action dependence. But what happens between thesesituations? If the same two objects can interact in twodifferent ways to get opposite results, there must be a sit-uation in which the effects of mass dependence and actiondependence will compromise or cancel out. In question31, the smaller, lighter car is initially moving much fasterthan the bigger, heavier truck, but the truck is moving.In this case our “most common student” must decide in Ref. [28]. question 33 is not included in analysis of the FMCE.[14] FIG. 6: Questions involving collisions within the Newton IIIcluster of the FMCE. Please note that question 33 is not in-cluded within analysis of the FMCE.[14] how to deal with mass dependence ideas from question 30and action dependence ideas from question 32. Response“e” is one logical conclusion. The two effects cancel eachother out to result in the car and the truck exerting forceson each other that are equal in magnitude but oppositein direction. A more discerning student, however, mayfeel that the effects will cancel each other to some degreebut not necessarily completely, leading to response “f,”that more information is needed. Figure 7 shows how themass dependence and action dependence resources may“balance” to produce the correct conclusion.To avoid the measurement of false positives, we suggestclustering responses 30–32 into a “triplet” sub-cluster asdone on the question triplet sub-clusters in the ReversingDirections cluster. Otherwise, one incorrectly rewardsstudents for a situation where two different wrongs do,in fact, make a right.
2. False positives in pushing situations
A situation exists where two identical wrongs make aright, as well. As shown in table VIII, the most com-mon responses for question 36 and 38 are “c” and “b,”0
FIG. 7: The mass dependence and action dependence re-sources may be used together to cancel out some of theireffects based on the situation (as in question 31). respectively (the questions are given in figure 8). Manystudents, however, choose responses “c” and “a” for thesetwo questions. Response “a” for question 38 indicatesa correct answer of equal and opposite forces exerted bythe two vehicles on each other. Response “c” for question36, on the other hand, indicates the student’s use of theaction dependence line of reasoning.We again assume some consistency of student rea-soning within a cluster of questions (that are contextu-ally and representationally similar). In question 36 thesmaller car is pushing the heavier truck, and the twoare speeding up. Use of the action dependence resourcesuggests that the car is exerting a greater force thanthe truck (response “c”). This result agrees with thepushing contextual feature reported by Bao, Hogg, andZollman.[24] In question 38, however, the two vehiclesare at a constant cruising speed, and the truck begins toapply its brakes, causing both vehicles to slow down. Re-sponse “b” for question 38 (the truck exerts more force)might be indicative of action dependence reasoning.When the truck begins applying its brakes, it maybecome the more active object in the student’s mind,causing the vehicles to slow down. Once again, we havethe possibility of two effects of incorrect resoning act-ing against one another. The car is the active agent,pushing the truck forward. The truck is a second activeagent, pushing back against the car. A student mightbelieve that the two effects will perfectly balance eachother and might arrive at the correct response (“a”) inwhich the two vehicles exert equal amounts of force onone another. This result is quite similar to that found forresponse “e” on question 31 with the exception that ques-tion 38 presents the opportunity for students to use twoconflicting versions of action dependence and eliminatesthe need for mass dependence. These responses often occur with the most common responseslisted in table VIII for questions 30–34.[28]
FIG. 8: Questions involving one vehicle pushing anotherwithin the Newton III cluster of the FMCE. Please note thatquestions 35 and 37 are not included within analysis of theFMCE.[14]
F. Velocity Graphs Cluster
The Velocity Graphs cluster is very similar to the pre-viously described “graphing” clusters. Students are pre-sented with various descriptions of a car’s motion, andthey must choose the correct velocity vs. time graphi-cal representation of the motion. As with the Acceler-ation Graphs questions, the incorrect model we exam-ine is derived from Trowbridge and McDermott’s studiesof students’ understanding of kinematics and their diffi-culty distinguishing position from velocity.[30] This ve-locity/position confusion model is also closely related toBeichner’s proposition that students view graphs as a pic-ture of the situation no matter what the axes indicate.[9]Table IX shows how the responses in this cluster corre-spond with the various student models.Once again, more than one response to a single ques-tion may indicate the use of our incorrect model. Onquestion 42, the toy car is said to be “moving toward theleft (toward the origin) at a steady (constant) velocity.”Responses “c” and “h” (shown in figure 9) both indicatea graph that gets steadily closer to the horizontal axisas time progresses. For response “c” a student could bepicturing the car starting at the right and moving toward“0,” and students choosing “h” could be triggered by theword “left” to choose a graph that depicts negative ve-locity.As an aside, we note that questions 17, 23, and 421
TABLE IX: The “Velocity Graphs” cluster on the FMCE.Correct Velocity/ Most CommonQuestion Model for Position StudentNumber Velocity Confusion Response[13]40 a d d41 f g g42 b c, h c43 d XX XX aa Thornton reports the most common student response for ques-tion 43 as “not significant.”[13]
FIG. 9: Responses “c” and “h” for the Velocity Graphs clus-ter. form a cluster which lets one see if students have prob-lems understanding coordinate systems, allowing for afiner grained analysis of students’ understanding of forceand motion. The FMCE only measures this topic implic-itly, though, and the cluster is therefore relatively badlydefined.
G. Energy Cluster
The Energy cluster on the FMCE contains questionsthat ask students to reason about the speed and kineticenergy of a sled after sliding down a hill. The incor-rect model for the questions in the Energy cluster, asdescribed in section III, corresponds with the idea thatsteeper hills will cause a greater change in speed and ki-netic energy as the sled slides down. Table X shows howthe possible responses to questions in this cluster are di-vided among the correct and this incorrect model. Themost commonly given incorrect answers correlate withthe responses indicating a student’s use of the slope de-pendent model.
V. SUMMARY
We have described a method for clustering questionson the FMCE that lets us use a resources frameworkto account for the most common student responses asreported by either Thornton[13] or in the work of oneauthor[28]. Our clustering allows us to categorize cor-
TABLE X: The “Energy” cluster on the FMCE.Energy/ Energy/ MostQuestion Speed Speed CommonNumber Depends on Depends on StudentHeight Slope Response a
44 b a a45 b a a46 a c c47 a c c a Most common student responses for the energy cluster discoveredby research reported in Ref. [28]. rect and incorrect responses using a single language ofresource activation.Our clusters (Force Sled, Reversing Direction, ForceGraphs, Acceleration Graphs, Newton III, VelocityGraphs, and Energy) take into account the physics con-tent, the contextual aspects, and the representations usedto ask the questions. We show that students’ incorrectresponses to questions on the FMCE may be indicativeof a variety of mental models that correspond with welldocumented research results. Using the resources frame-work, we can analyze sets of questions within some clus-ters (Reversing Direction and Newton III) to describesome correct student responses as false positives.We have presented interpretations for the most com-mon incorrect responses for each questions, but this is inno way an exhaustive list of the possible mental modelsthat may be used by students while answering questionson the FMCE. Additional patterns of responses shouldbe examined for prevalence among student responses andanalyzed in terms of mental models that may be indi-cated by each. For example, questions 17, 23, and 42 area “coordinate systems” cluster that has not yet been eval-uated but may affect student responses on other clusters.Also, a second tier of mixed-context clusters of questions(such as Bao’s[16]) could be created that “slice” data indifferent ways.Research tools such as the FMCE are most effectiveto educators and researchers only when responses are ex-amined to determine not only whether or not studentshave the correct ideas, but also what ideas they do have(correct or otherwise) and how consistently they use theseideas across similar questions. Our clustering allows suchan analysis, giving insight into both how we model stu-dent thinking and how we could better address studentneeds in the classroom.
Acknowledgments
We thank three reviewers for their valuable commentsand insights on an earlier version of this paper. Sup-ported in part by NSF grants DUE–0510614 and REC–0633951. Portions of this manuscript originally published2in Ref. [28]. [1] D. Hestenes, M. Wells, and G. Swackhamer,
Force Con-cept Inventory , The Physics Teacher , 141 (1992).[2] R. R. Hake, Interactive-engagement versus traditionalmethods: A six-thousand-student survey of mechanicstest data for introductory physics courses , AmericanJournal of Physics , 64 (1998).[3] R. K. Thornton and D. R. Sokoloff, Assessing studentlearning of Newton’s laws: The Force and Motion Con-ceptual Evaluation and the Evaluation of Active LearningLaboratory and Lecture Curricula , American Journal ofPhysics , 338 (1998).[4] M. C. Wittmann, On the dissemination of aproven curriculum: RealTime Physics and In-teractive Lecture Demonstrations , available at http://perlnet.umephy.maine.edu/research/Wittmann2002RTPpaper.pdf .[5] E. F. Redish, J. M. Saul, and R. N. Steinberg,
On theeffectiveness of active-engagement microcomputer-basedlaboratories , American Journal of Physics , 45 (1997).[6] D. Hammer, Student resources for learning introductoryphysics , American Journal of Physics
67 (Physics Ed-ucation Research Supplement) , S45 (2000).[7] J. Minstrell,
Facets of students’ knowledge and relevantinstruction , in
Research in Physics Learning: Theoreti-cal Issues and Empirical Studies, Proceedings of an In-ternational Workshop, Bremen, Germany 1991 , editedby R. Duit, F. Goldberg, and H. Niedderer (IPN, Kiel,1992), pp. 110–128.[8] M. C. Wittmann,
Using resource graphs to representconceptual change , Phys. Rev. ST Phys. Educ. Res. ,020105 (2006).[9] R. J. Beichner, Testing student interpretation of kinemat-ics graphs , American Journal of Physics , 750 (1994).[10] P. B. Kohl and N. D. Finkelstein, Effects of representa-tion on students solving physics problems: A fine-grainedcharacterization , Phys. Rev. ST Phys. Educ. Res. ,010106 (2006).[11] P. B. Kohl and N. D. Finkelstein, Student represen-tational competence and self-assessment when solvingphysics problems , Phys. Rev. ST Phys. Educ. Res. ,010104 (2005).[12] D. J. Dykstra, Why teach kinematics? An examination ofthe teaching of kinematics and force–I (2002), availableat .[13] R. K. Thornton,
Using large-scale classroom research tostudy student conceptual learning in mechanics and todevelop new approaches to learning , in
NATO ASI Series (Springer Verlag, 1993).[14] R. K. Thornton,
Conceptual dynamics: Changing studentviews of force and motion , in
Thinking Physics for Teach-ing , edited by C. Tarsitani, C. Bernardini, and M. Vin-centini (Plenum Publishing, 1995).[15] R. K. Thornton and D. R. Sokoloff,
Learning motionconcepts using real-time microcomputer-based laboratorytools , American Journal of Physics , 858 (1990).[16] L. Bao, unpublished Ph.D. dissertation, available at http://physics.umd.edu/perg/dissertations/ , Uni- versity of Maryland (1999).[17] J. H. Wandersee, J. J. Mintzes, and J. D. Novak, Re-search on alternative conceptions in science , in
Handbookof Research of Science Teaching and Learning (Macmil-lan Publishing, New York, 1994), pp. 177–210.[18] S. Vosniadou and W. F. Brewer,
Mental Models of theEarth: A Study of Conceptual Change in Childhood , Cog-nitive Psychology , 535 (1992).[19] A. diSessa, Towards an epistemology of physics , in
Cog-nition and Instruction (Lawrence Erlbaum Associates,Inc., 1993), vol. 10, pp. 105–225.[20] D. Hammer, A. Elby, R. E. Scherr, and F. Redish, Ed-ward,
Resources, framing, and transfer , in
Transfer ofLearning: Research and Perspectives (Information AgePublishing, Greenwich, CT, 2005).[21] R. E. Scherr,
Modeling student thinking: An examplefrom special relativity , American Journal of Physics ,272 (2007).[22] D. E. Brown and J. Clement, Overcoming misconcep-tions via analogical reasoning: Abstract transfer versusexplanatory model construction , in
Instructional Science (Kluwer Academic Publishers, Dordrecht, Netherlands,1989), vol. 18, pp. 237–261.[23] I. A. Halloun and D. Hestenes,
Common sense conceptsabout motion , American Journal of Physics , 1056(1985).[24] L. Bao, K. Hogg, and D. Zollman, Model analysis of finestructures of student models: An example with Newton’sthird law , American Journal of Physics , 766 (2002).[25] T. I. Smith and M. C. Wittmann, Comparing three meth-ods for teaching Newton’s third law , Phys. Rev. ST Phys.Educ. Res. , 020105 (2007).[26] D. P. Maloney, Rule-governed approaches to physics:Newton’s Third Law , Physics Education , 37 (1984).[27] D. Hammer and A. Elby, Tapping epistemological re-sources for learning physics , Journal of the Learning Sci-ences , 53 (2003).[28] T. I. Smith, Comparing the Effectiveness of Research-based Curricula for Teaching Introductory Mechan-ics (2007), unpublished Masters thesis, available at: .[29] D. E. Trowbridge and L. C. McDermott,
Investigationof student understanding of the concept of accelerationin one dimension , American Journal of Physics , 242(1981).[30] D. E. Trowbridge and L. C. McDermott, Investigationof student understanding of the concept of velocity in onedimension , American Journal of Physics48