Adrian Madsen

Gender gap on concept inventories in physics: What is consistent, what is inconsistent, and what factors influence the gap?

We review the literature on the gender gap on concept inventories in physics. Across studies of the most commonly used mechanics concept inventories, the Force Concept Inventory (FCI) and Force and Motion Conceptual Evaluation (FMCE), mens average pretest scores are always higher than womens, and in most cases mens posttest scores are higher as well. The weighted average gender difference on these tests is 13% for pretest scores, 12% for posttest scores, and 6% for normalized gain. This difference is much smaller than the average difference in normalized gain between traditional lecture and interactive engagement (25%), but it is large enough that it could impact the results of studies comparing the effectiveness of different teaching methods. There is sometimes a gender gap on commonly used electromagnetism concept inventories, but it is usually much smaller and sometimes is zero or favors women. The weighted average gender difference on these tests is 3.7% for pretest scores, 8.5% for posttest scores, and 6% for normalized gain. Based on our analysis of 26 published articles, no single factor is sufficient to explain the gap. Several high-profile studies that have claimed to account for or reduce the gender gap have failed to be replicated in subsequent studies, suggesting that isolated claims of explanations of the gender gap should be interpreted with caution. Other factors, such as gender differences in background preparation, scores on different kinds of assessment, and splits between how students respond to test questions when answering for themselves or for a scientist do contribute to a difference between male and female responses, but the size of these differences is smaller than the size of the overall gender gap, suggesting that the gender gap is most likely due to the combination of many small factors rather than any one factor that can easily be modified.

Using ScanMatch scores to understand differences in eye movements between correct and incorrect solvers on physics problems

Using a ScanMatch algorithm we investigate scan path differences between subjects who answer physics problems correctly and incorrectly. This algorithm bins a saccade sequence spatially and temporally, recodes this information to create a sequence of letters representing fixation location, duration and order, and compares two sequences to generate a similarity score. We recorded eye movements of 24 individuals on six physics problems containing diagrams with areas consistent with a novice-like response and areas of high perceptual salience. We calculated average ScanMatch similarity scores comparing correct solvers to one another (C-C), incorrect solvers to one another (I-I), and correct solvers to incorrect solvers (C-I). We found statistically significant differences between the C-C and I-I comparisons on only one of the problems. This seems to imply that top down processes relying on incorrect domain knowledge, rather than bottom up processes driven by perceptual salience, determine the eye movements of incorrect solvers.

Resource Letter RBAI-1: Research-Based Assessment Instruments in Physics and Astronomy

This resource letter provides a guide to Research-Based Assessment Instruments (RBAIs) of physics and astronomy content. These are standardized assessments that were rigorously developed and revised using student ideas and interviews, expert input, and statistical analyses. RBAIs have had a major impact on physics and astronomy education reform by providing a universal and convincing measure of student understanding that instructors can use to assess and improve the effectiveness of their teaching. In this resource letter, we present an overview of all content RBAIs in physics and astronomy by topic, research validation, instructional level, format, and themes, to help faculty find the best assessment for their course. More details about each RBAI available in physics and astronomy are available at PhysPort: physport.org/assessments.

Research-Based Assessment Affordances and Constraints: Perceptions of Physics Faculty.

Citation: Madsen, A., McKagan, S. B., Martinuk, M. S., Bell, A., & Sayre, E. C. (2016). Research-based assessment affordances and constraints: Perceptions of physics faculty. Physical Review Physics Education Research, 12(1), 16. doi:10.1103/PhysRevPhysEducRes.12.010115

Best Practices for Administering Concept Inventories

There is a plethora of concept inventories available for faculty to use, but it is not always clear exactly why you would use these tests, or how you should administer them and interpret the results. These research-based tests about physics and astronomy concepts are valuable because they allow for standardized comparisons among institutions, instructors, or over time. In order for these comparisons to be meaningful, you should use best practices for administering the tests. In interviews with 24 physics faculty, we have identified common questions that faculty members have about concept inventories. We have written this article to address common questions from these interviews and provide a summary of best practices for administering concept inventories.

Scaffolding students’ understanding of force in pulley systems

Recent research results have found that students using virtual manipulatives perform as well or better on measures of conceptual understanding than their peers who used physical equipment. We report on a study with students in a conceptual physics laboratory using either physical or virtual manipulatives to investigate forces in pulley systems. Written materials guided students through a sequence of activities designed to scaffold their understanding of force in pulley systems. The activity sequences facilitated students’ sense making by requiring them to make and test predictions about various pulley systems by building and comparing different systems. We investigate the ways in which students discuss force while navigating the scaffolding activities and how these discussions compare between the physical and virtual treatments.