Edit Yerushalmi

Grading student problem solutions: The challenge of sending a consistent message

Grading sends a direct message to students about what is expected in class. However, often there is a gap between the assigned grade and the goals of the instructor. In an interview study of faculty teaching calculus-based introductory physics, we verified that this gap exists and identified three themes that appear to shape grading decisions: (1) a desire to see student reasoning, (2) a reluctance to deduct points from a student solution that might be correct, and (3) a tendency to project correct thought processes onto a student solution. When all three themes were expressed by an instructor, the resulting conflict was resolved by placing the burden of proof on either the instructor or the student. The weighting of the themes with the burden of proof criterion explains our finding that although almost all instructors reported telling students to show their reasoning in problem solutions, about half graded problem solutions in a way that would likely discourage students from showing this reasoning.

Physics Learning In The Context Of Scaffolded Diagnostic Tasks (I): The Experimental Setup

For problem solving to serve as an effective learning opportunity, it should involve deliberate reflection, e.g., planning and evaluating the solvers progress toward a solution, as well as self‐diagnosing former steps while elaborating on conceptual understanding. While expert problem solvers employ deliberate reflection, the novices (many introductory physics students) fail to take full advantage of problem solving as a learning opportunity. In this paper we will focus on self‐diagnosis as an instructional strategy to engage students in reflective problem solving. In self‐diagnosis tasks students are explicitly required to carry out self diagnosis activities after being given some feedback on the solution. In this and a companion paper, we will present research exploring the following questions: How well do students self‐diagnose, if at all, their solutions? What are the learning outcomes of these activities? Can one improve the act of self‐diagnosis and the resulting learning outcomes by scaffolding th...

Effect of Self Diagnosis on Subsequent Problem Solving Performance

“Self‐diagnosis tasks” aim at fostering diagnostic behavior by explicitly requiring students to present diagnosis as part of the activity of reviewing their problem solutions. The recitation classes in an introductory physics class (∼200 students) were split into a control group and three experimental groups in which different levels of guidance were provided for performing the self‐diagnosis activities. We have been a) investigating how students in each group performed on subsequent near and far transfer questions given as part of the exams; and b) comparing student’s initial scores on their quizzes with their performance on the exams, as well as comparing student’s self‐diagnosis scores with their performance on the exams. We discuss some hypotheses about the students’ ability to self‐diagnose with different levels of scaffolding support and emphasize the importance of teaching students how to diagnosis their own mistakes. Our findings suggest that struggling with minimal support during in‐class self‐di...

Identifying Differences in Diagnostic Skills between Physics Students: Students’ Self‐Diagnostic Performance Given Alternative Scaffolding

“Self‐diagnosis tasks” aim at fostering diagnostic behavior by explicitly requiring students to present diagnosis as part of the activity of reviewing their problem solutions. We have been investigating the extent to which introductory physics students can diagnose their own mistakes when explicitly asked to do so with different levels of scaffolding support provided to them. In our study in an introductory physics class with more than 200 students, the recitation classes were split into three different experimental groups in which different levels of guidance were provided for performing the self‐diagnosis activities. We present our findings that students’ performance was far from perfect. However, differences in the scaffolding in the three experimental groups (i.e. providing a correct solution and a self‐diagnosis rubric) noticeably affected the resulting diagnosis.

Introductory physics going soft

We describe an elective course on soft matter at the level of introductory physics. Soft matter physics serves as a context that motivates the presentation of basic ideas in statistical thermodynamics and their applications. It also is an example of a contemporary field that is interdisciplinary and touches on chemistry, biology, and physics. We outline a curriculum that uses the lattice gas model as a quantitative and visual tool, initially to introduce entropy, and later to facilitate the calculation of interactions. We demonstrate how free energy minimization can be used to teach students to understand the properties of soft matter systems such as the phases of fluid mixtures, wetting of interfaces, self-assembly of surfactants, and polymers. We discuss several suggested activities in the form of inquiry projects which allow students to apply the concepts they have learned to experimental systems.

Identifying Differences In Diagnostic Skills Between Physics Students: Developing A Rubric

Expert problem solvers are characterized by continuous evaluation of their progress towards a solution. One characteristic of expertise is self‐diagnosis directed towards elaboration of the solvers’ conceptual understanding, knowledge organization or strategic approach. “Self‐diagnosis tasks” aim at fostering diagnostic behavior by explicitly requiring students to present diagnosis as part of the activity of reviewing their problem solutions. We have been investigating how introductory physics students perform in such tasks. Developing a robust rubric is essential for objective evaluation of students’ self‐diagnosis skills. We discuss the development of a grading rubric that takes into account introductory physics students’ content knowledge as well as analysis, planning and presentation skills. Using this rubric, we have found the inter‐rater reliability to be better than 80%. The rubric can easily be adapted to other problems, as will be discussed in a companion paper.

Assessing Reflection on Practice: A Problem Solving Perspective

Reflection on practice (ROP) serves to support teachers that introduce innovative instruction into their classrooms. There is an inherent dilemma between competing goals in ROP workshops: developing teachers’ skills as reflective practitioners (process), vs. developing specific favored practices (result). This dilemma affects the evaluation of such workshops, as evaluation methods should align with the goals. In this paper we will gain insight on how to resolve the dilemma from the perspective of teaching scientific problem solving, where a similar dilemma between process and result is sharply manifested and thoroughly explored. Assessment methods and tools derived from this perspective were applied in a formative evaluation of a workshop for high school physics teachers. We will show how these analysis tools enabled us to identify differences in outcomes between versions of yearlong workshops that used different approaches to guidance of ROP. Our research can contribute to the planning and evaluation of ...

Graduate teaching assistants use different criteria when grading introductory physics vs. quantum mechanics problems

Physics graduate teaching assistants (TAs) are often responsible for grading. Physics education research suggests that grading practices that place the burden of proof for explicating the problem solving process on students can help them develop problem solving skills and learn physics. However, TAs may not have developed effective grading practices and may grade student solutions in introductory and advanced courses differently. In the context of a TA professional development course, we asked TAs to grade student solutions to introductory physics and quantum mechanics problems and explain why their grading approaches were different or similar in the two contexts. TAs expected and rewarded reasoning more frequently in the QM context. Our findings suggest that these differences may at least partly be due to the TAs not realizing that grading can serve as a formative assessment tool and also not thinking about the difficulty of an introductory physics problem from an introductory physics students perspective.

Learning from Mistakes: The Effect of Students' Written Self-Diagnoses on Subsequent Problem Solving

Helping students learn to think like a physicist is an important goal of many introductory physics courses. One characteristic distinguishing more experienced physicists from novice students is that they make better use of problem solving as a learning opportunity. Experts were found to spend more time than novices in monitoring their work, reflecting upon their possibly deficient approach to solving a problem, reconsidering their choices as necessary, and extending and refining their knowledge structure. Moreover, research on worked-out examples suggests that better performing students are those who “self-explain,” that is, elaborate to themselves what they are learning from those examples, generate inferences, acknowledge mismatches between their own approach and that of the example, and attempt to resolve conflicts (self-repair). Indeed, physics instructors often express concern that many students do not make an effort to learn from their mistakes after the graded problems are returned to them. Here, we discuss an investigation focusing on how well introductory physics students self-diagnose their mistakes in their quiz solutions and its effect on subsequent problem solving in different interventions.

Self‐Diagnosis, Scaffolding and Transfer in a More Conventional Introductory Physics Problem

Previously we discussed how well students in an introductory physics course diagnosed their mistakes on a quiz problem with different levels of scaffolding support. In that case, the problem they self‐diagnosed was unusually difficult. We also discussed issues related to transfer, particularly the fact that the transfer problem in the midterm that corresponded to the self‐diagnosed problem was a far transfer problem. Here, we discuss a related intervention in which we repeated the study methodology with the same students in the same intervention groups, using a new quiz problem which was more typical for these students and a near transfer problem. We discuss how these changes affected students’ ability to self‐diagnose and transfer from the self‐diagnosed quiz problem to a transfer problem on the midterm exam.