Katherine L. McNeill

Middle school students’ use of appropriate and inappropriate evidence in writing scientific explanations

Recent science reform efforts and standards documents advocate that students develop scientific inquiry practices, such as the construction and communication of scientific explanations. This paper focuses on 7th grade students’ scientific explanations during the enactment of a project based chemistry unit where the construction of scientific explanations is a key learning goal. During the unit, we make the explanation framework explicit to students and include supports or scaffolds in both the student and teacher materials to facilitate students’ in their understanding and construction of scientific explanations. Results from the enactment show significant learning gains for students for all components of scientific explanation (i.e. claim, evidence, and reasoning). Although students’ explanations were stronger at the end of the instructional unit, we also found that students’ still had difficulty differentiating between appropriate and inappropriate evidence for some assessment tasks. We conjecture that students’ ability to use appropriate data as evidence depends on the wording of the assessment task, students’ content knowledge, and their understanding of what counts as evidence. Having students construct scientific explanations can be an important tool to help make students thinking visible for both researchers and teachers. Appropriate and Inappropriate Evidence Use 2 Middle School Students’ Use of Appropriate and Inappropriate Evidence in Writing Scientific Explanations The National Research Council (1996) and the American Association for the Advancement of Science (1993) call for scientific literacy for all. All students need knowledge of scientific concepts and inquiry practices required for personal decision making, participation in societal and cultural affairs, and economic productivity. Science education should support students’ development toward competent participation in a science infused world (McGinn & Roth, 1999). This type of participation should be obtainable for all students, not just those who are educated for scientific professions. Consequently, we are interested in supporting all students in learning scientific concepts and inquiry practices. By scientific inquiry practices, we mean the multiple ways of knowing which scientists use to study the natural world (National Research Council, 1996). Key scientific inquiry practices called for by national standards documents include asking questions, designing experiments, analyzing data, and constructing explanations (American Association for the Advancement of Science, 1993; National Research Council, 1996). In this study, we focus on analyzing data and constructing explanations. These practices are essential not only for scientists, but for all individuals. On a daily basis, individuals need to evaluate scientific data provided to them in written form such as newspapers and magazines as well spoken through television and radio. Citizens need to be able to evaluate that data to determine whether the claims being made based on the data and reasoning are valid. This type of data evaluation, like other scientific inquiry practices, is dependent both on a general understanding of how to evaluate data as well as an understanding of the science content. Appropriate and Inappropriate Evidence Use 3 In this study we explore when students use appropriate evidence and when they use inappropriate evidence to support their claims. Our work focuses on an 8-week project-based chemistry curriculum designed to support 7 grade students in using evidence and constructing scientific explanations. We examine the characteristics of these students’ explanations, their understanding of the content knowledge, and the assessment tasks to unpack what may be influencing students use of evidence. Our Instructional Model for Scientific Explanations In our work, we examine how students construct scientific explanations using evidence. We use a specific instructional model for evidence-based scientific explanations as a tool for both classroom practice and research. We provide both teachers and students with this model to make the typically implicit framework of explanation, explicit to both teachers and students. Our instructional model for scientific explanation uses an adapted version of Toulmin’s (1958) model of argumentation and builds off previous science educators’ research on students’ construction of scientific explanations and arguments (Bell & Linn, 2000; Jiménez-Aleixandre, Rodríguez, & Duschl, 2000; Lee & Songer, 2004; Sandoval, 2003; Zembal-Saul, et al., 2002). Our explanation framework includes three components: a claim (similar to Toulmin’s claim), evidence (similar to Toulmin’s data), and reasoning (a combination of Toulmin’s warrants and backing). The claim makes an assertion or conclusion that addresses the original question or problem. The evidence supports the student’s claim using scientific data. This data can come from an investigation that students complete or from another source, such as observations, reading material, or archived data. The data need to be both appropriate and sufficient to support the claim. Appropriate data is relevant to the question or problem and relates to the given claim. Data is sufficient when it includes the necessary quantity to convince someone of a claim. The Appropriate and Inappropriate Evidence Use 4 reasoning is a justification that links the claim and evidence and shows why the data counts as evidence to support the claim by using the appropriate scientific principles. Kuhn argues (1993) that argument, or in our case scientific explanation, is a form of thinking that transcends the particular content to which it refers. Students can construct scientific explanations across different content areas. Although an explanation model, such as Toulmin’s, can be used to assess the structure of an explanation, it cannot determine the scientific accuracy of the explanation (Driver, Newton & Osborne, 2000). Instead, both the domain general explanation framework and the domain specific context of the assessment task determine the correctness of the explanation. Consequently, in both teaching students about explanation and assessing students’ construction of explanations we embed the scientific inquiry practice in a specific context. Student Difficulties Constructing Explanations Prior research in science classrooms suggests that students have difficulty constructing high-quality scientific explanations where they articulate and defend their claims (Sadler, 2004). For example, students have difficulty understanding what counts as evidence (Sadler, 2004) and using appropriate evidence (Sandoval, 2003; Sandoval & Reiser, 1997). Instead, students will draw on data that do not support their claim. Consequently, we are interested in whether students use appropriate evidence to support their claim or if they draw on evidence that is not relevant. Students’ claims also do not necessarily relate to their evidence. Instead, students often rely on their personal views instead of evidence to draw conclusions (Hogan & Maglienti, 2001). Students have a particularly difficult time reasoning from primary data, especially when Appropriate and Inappropriate Evidence Use 5 measurement error plays an important role (Kanari & Millar, 2004). Students can recognize variation in data and use characteristics of data in their reasoning, but their ability to draw final conclusions from that data can depend on the context. Masnick, Klahr, and Morris (this volume) concluded that young students who poorly understood the context of the investigation had difficulty interpreting data, particularly when the interpretation of that data contradicted their prior beliefs. Students will likely discount data if the data contradicts their current theory (Chinn & Brewer, 2001) and they will only consider data if they can come up with a mechanism for the pattern of data (Koslowski, 1996). When students evaluate data, more general reasoning strategies interact with domain-specific knowledge (Chinn & Brewer, 2001). Whether students use appropriate and inappropriate evidence may depend on their prior understanding of a particular content area or task. Students also have difficulty providing the backing, or what we refer to as reasoning, for why they chose the evidence (Bell & Linn, 2000) in their written explanations. Other researchers have shown that during classroom discourse, discussions tend to be dominated by claims with little backing to support their claims (Jiménez-Aleixandre, Rodríguez & Duschl, 2000). Our previous work supports these ideas. We found that middle school students’ had the most difficulty with the reasoning component of scientific explanations (McNeill, Lizotte, Krajcik & Marx, in review; McNeill, et al., 2003). Although students’ reasoning improved over the course of the 6-8 week instructional unit, it was consistently of lower quality than their claims or evidence. Students’ reasoning often just linked their claim and evidence and less frequently articulated the scientific principles that allowed them to make that connection. Similar to students ability to evaluate and use data, providing accurate reasoning is related to students understanding of the content. Students with stronger content knowledge Appropriate and Inappropriate Evidence Use 6 provide stronger reasoning in their scientific explanations (McNeill et al., in review). Previous research with students has found that their success at completing scientific inquiry practices is highly dependent on their understanding of both the content and the scientific inquiry practices (Metz, 2000). Both domain specific and general reasoning are essential for students’ effective evaluation of data and construction of scientific explanations. Although previous work has shown that students have difficulty with components of scientific explanations, there has been little research unpacking exactly when

Use of First‐hand and Second‐hand Data in Science: Does data type influence classroom conversations?

In this paper, we examine how students discuss and interpret data and whether these actions vary depending on the type of data they analyse. More specifically, we are interested in whether students perform differently when analysing first‐hand data, which they collect themselves, compared with second‐hand data provided to them. Our data analysis focused on two classrooms using two different curriculum units, chemistry in Grade 7 and biology in Grade 8, collected during the 2002/03 school year from a Mid‐western urban middle school in the USA. We analysed classroom videotape associated with lessons in which students discussed first‐hand and second‐hand data both in small group settings and full class discussions. We found the two types of data offer different benefits and limitations, suggesting that both types of data are important for students to work with as they develop skills in scientific inquiry practices. We discuss the characteristics of classroom discussions around different data sources as well as implications for the design of curriculum materials, instructional environments, and student learning in science.

The Impact of High School Science Teachers’ Beliefs, Curricular Enactments and Experience on Student Learning During an Inquiry-based Urban Ecology Curriculum

Inquiry-based curricula are an essential tool for reforming science education yet the role of the teacher is often overlooked in terms of the impact of the curriculum on student achievement. Our research focuses on 22 teachers’ use of a year-long high school urban ecology curriculum and how teachers’ self-efficacy, instructional practices, curricular enactments and previous experience impacted student learning. Data sources included teacher belief surveys, teacher enactment surveys, a student multiple-choice assessment focused on defining and identifying science concepts and a student open-ended assessment focused on scientific inquiry. Results from the two hierarchical linear models indicate that there was significant variation between teachers in terms of student achievement. For the multiple-choice assessment, teachers who spent a larger percentage of time on group work and a smaller percentage of time lecturing had greater student learning. For the open-ended assessment, teachers who reported a higher frequency of students engaging in argument and sharing ideas had greater student learning while teachers who adapted the curriculum more had lower student learning. These results suggest the importance of supporting the active role of students in instruction, emphasising argumentation, and considering the types of adaptations teachers make to curriculum.

Multimedia Educative Curriculum Materials (MECMs): Teachers’ Choices in Using MECMs Designed to Support Scientific Argumentation

ABSTRACT In this article, we describe the development process for designing multimedia educative curriculum materials (MECMs) focused on supporting teachers in argumentation. We also describe results from a study with 46 teachers. We were interested in whether teachers’ backgrounds or characteristics of the MECMs impacted MECM use. Overall, 89% of the teachers used the MECMs. Teachers’ use was not related to their background, such as years teaching; however, placement of the MECMs and the type of support impacted use. Teachers were more likely to access MECMs embedded in lesson plans or reflective self-assessment prompts compared to a separate library. In addition, teachers were more likely to watch videos earlier in the curriculum and the 1st time a new activity structure was introduced.

‘Does it answer the question or is it French fries?’: an exploration of language supports for scientific argumentation

ABSTRACT Reform initiatives around the world are reconceptualising science education by stressing student engagement in science practices. Yet, science practices are language-intensive, requiring students to have strong receptive and productive language proficiencies. It is critical to address these rigorous language demands to ensure equitable learning opportunities for all students, including English language learners (ELLs). Little research has examined how to specifically support ELL students’ engagement in science practices, such as argumentation. Using case-study methodology, we examined one middle school science teachers instructional strategies as she taught an argumentation-focused curriculum in a self-contained ELL classroom. Findings revealed that three trends characterized the teacher’s language supports for the structural and dialogic components of argumentation: (1) more language supports focused on argument structure, (2) dialogic interactions were most often facilitated by productive language supports, and (3) some language supports offered a rationale for argumentation. Findings suggest a need to identify and develop supports for the dialogic aspects of argumentation. Furthermore, engaging students in argumentation through productive language functions could be leveraged to support dialogic interactions. Lastly, our work points to the need for language supports that make the rationale for argumentation explicit since such transparency could further increase access for all students.

Negotiating Competing Goals in the Development of an Urban Ecology Practitioner Inquiry Community

Teacher learning communities are hailed by many as vehicles for reforming and elevating the professional status of teaching. While much research explores teacher community as a venue for measurable gains, our research examines the orientation of practitioner inquiry toward critical debate about effective instruction. Specifically, our study focuses on a group of middle and high school teachers who worked with a nonprofit organization to engage students in urban environmental field investigations. Teachers met regularly as a community with the common goal of teaching urban ecology in an outdoor setting. We collected interview data from members of the teacher community, and we observed teacher interaction during a meeting of the practitioner inquiry group. Interview results indicated that while the nonprofit aimed to support collaborative dialogue and self-critique, participants saw the community mainly as a venue for pursuing short-term goals, such as receiving new resources or socializing with colleagues. Observation data, however, suggested that the community was taking early steps toward building an environment oriented toward critical discussion. Juxtaposing results from our interviews and observations, we discuss the challenges communities face when they seek to develop shared beliefs and deal openly with conflict. Ultimately, we suggest that organizers of collaborative learning environments should work to actively develop structures for building the organizational trust necessary to support civil critique.

Assessment at the Intersection of Science and Literacy

The authors of this article, all of whom have been a part of this effort to assess argumentation in literacy-rich science curriculum, have struggled with our attempts to build 3 argument-related assessments—understanding, critiquing, and constructing arguments about scientific phenomena in both oral and written modes. Loosely affiliated with the Seeds of Science/Roots of Reading Project at Lawrence Hall of Science, this effort focused on creating a suite of assessments as models for how middle school science teachers might create their own school-based, curriculum-embedded assessments of science. After reviewing the broad scope and insights derived from a 10-year history of assessments that operate at the intersection of science and literacy, we zoom in on 3 vexing but informative challenges they encountered—and addressed (if not resolved)—as they tried to assess the comprehension, critique, and construction of oral and written arguments.

Developing Educative Materials to Support Middle-School Science Teachers’ PCK for Argumentation: Comparing Multimedia to Text-Based Supports

Science teachers need support in developing pedagogical content knowledge (PCK) around science practices such as argumentation. Educative curriculum materials (ECMs) have great potential as a tool to support teachers’ PCK development, particularly for in-service teachers. In this chapter, we describe an intervention in which we developed and researched web-based educative curriculum materials focused on supporting teachers’ PCK of argumentation. Two different versions of the materials—one with text-based educative supports and one with text-based supports plus multimedia elements—were tested with a total of 90 middle-school science teachers. Participants completed pre- and post-surveys designed to measure their PCK of argumentation. In addition, back-end data was collected on teachers’ use of the web-based materials. Findings indicated that teachers’ PCK did increase from pre to post, although there was no significant difference between the two versions of the curriculum. Teachers’ use of the materials varied widely. Our findings suggest that educative curriculum materials are a promising avenue for supporting teacher PCK. Future efforts should take into consideration teachers’ use patterns in order to increase the likelihood that teachers will use and benefit from educative elements.

Subject-Specific Instructional Leadership in K8 Schools: The Supervision of Science in an Era of Reform

ABSTRACT Purpose: In U.S. public schools, principals must implement reforms that require instructional leadership across subjects, though little is known about subject-specific supervision. Methods: Through interviews with 26 K–8 principals, we examine instructional leadership for science. Findings: Our findings showed that science supervision occurred rarely; principals used a “content-neutral” approach that did not emphasize science-specific aspects of instruction. Principals explained this in terms of external accountability pressures in literacy and mathematics, as well as their own lack of science knowledge. Implications: We argue for subject-specific resources for principal supervision. For classrooms to change, principals must provide subject-specific support for teachers.

Teachers’ enactments of curriculum: Fidelity to Procedure versus Fidelity to Goal for scientific argumentation

ABSTRACT Fidelity of implementation (FOI) has received attention in calls for funding and research; however, there are numerous ways of conceptualising and measuring this construct. We argue that this conceptualisation is important for recent reform efforts focused on science practices. Consequently, we explored FOI in the context of the enactment of a middle-school curriculum focused on one particular science practice, argumentation. We coded videos of five teachers’ enactments of argumentation lessons using two different fidelity coding schemes. First, Fidelity to Procedure targeted teachers’ adherence to the order and types of procedures. Second, Fidelity to Goal examined teachers’ adherence to the overarching argumentation goals. This analysis resulted in case studies that illustrate distinct patterns in the teachers’ curriculum enactments. One case in particular, Ms Newbury, received a low score for Fidelity to Procedure, but a high score for Fidelity to Goal. She altered procedures to provide her students, all of whom were English Language Learners, with different linguistic supports, but maintained the overarching argumentation goals. Consequently, we argue that FOI for goals may better capture whether teachers’ enactments are supporting students in the science practices. Furthermore, the results suggest the importance of educative curriculum including rationales for the curricular goals.