Knut Neumann

Evaluating Instrument Quality in Science Education: Rasch‐based analyses of a Nature of Science test

Given the central importance of the Nature of Science (NOS) and Scientific Inquiry (SI) in national and international science standards and science learning, empirical support for the theoretical delineation of these constructs is of considerable significance. Furthermore, tests of the effects of varying magnitudes of NOS knowledge on domain‐specific science understanding and belief require the application of instruments validated in accordance with AERA, APA, and NCME assessment standards. Our study explores three interrelated aspects of a recently developed NOS instrument: (1) validity and reliability; (2) instrument dimensionality; and (3) item scales, properties, and qualities within the context of Classical Test Theory and Item Response Theory (Rasch modeling). A construct analysis revealed that the instrument did not match published operationalizations of NOS concepts. Rasch analysis of the original instrument—as well as a reduced item set—indicated that a two‐dimensional Rasch model fit significantly better than a one‐dimensional model in both cases. Thus, our study revealed that NOS and SI are supported as two separate dimensions, corroborating theoretical distinctions in the literature. To identify items with unacceptable fit values, item quality analyses were used. A Wright Map revealed that few items sufficiently distinguished high performers in the sample and excessive numbers of items were present at the low end of the performance scale. Overall, our study outlines an approach for how Rasch modeling may be used to evaluate and improve Likert‐type instruments in science education.

Competence in Science Education

The term science competence describes the results of educational work regarding science and is therefore linked to many different fields of research. This makes competence one of the central issues in science education, on the one hand, and causes difficulties defining science competence in a useful way for research and teaching on the other hand. This chapter aims to define science competence with a rather cognitive focus, concentrating on three central aspects of being competent in science. First, the aspect of cognitive ability is discussed that underlines the idea of defining competence as an ability to solve problems. Secondly, the importance of content is detailed to argue that competence is domain specific. Third, the scientific literacy aspect is illustrated to prove that competence is represented by a decontextualized knowledge structure that is applied to specific and contextualized problems. These aspects lead to a definition of competence which is finally discussed from the perspective of measurement.

Framing students’ progression in understanding matter: a review of previous research

This manuscript presents a systematic review of the research on how students conceptualise matter. Understanding the structure and properties of matter is an essential part of science literacy. Over the last decades the number of studies on students’ conceptions of matter published in peer-reviewed journals has increased significantly. These studies investigated how students conceptualise matter, to what extent students are able to explain everyday phenomena or how students develop an understanding of matter over time. In order to understand how students progress in their understanding of matter, what they understand easily and where they have difficulties, there is a need to identify common patterns across the available studies. The first substantial review of research on students’ conception was provided in the 1990s with the aim to organise students’ understanding of matter into four categories: students’ conceptions about (1) chemical reactions, (2) physical states and their changes, (3) atoms, molecules and particle systems and (4) conservation. The aim of this review and analysis is to identify how subsequent research on students’ conceptions of matter adds to this framework. The last comprehensive review of research on students’ understanding of matter was carried out in the early 2000s. Thus, we analysed studies on students’ conceptions of matter published within the last decade in five peer-reviewed journals of science education. Our findings suggest that research has moved from categorising students’ conceptions to analysing students’ progression in understanding matter. Based on our findings, we also identified typical pathways by which students may develop over time related to the four categories identified in previous reviews. As a conclusion, we present a model describing students’ progression in understanding matter which may contribute to the development of a K-12 learning progression of matter.

Quality of Instruction in Science Education

Instructional quality has been a central issue in educational research for a long time now. Models of school learning were proposed, a vast number of correlational studies were carried out, and lately large-scale video studies were undertaken in order to discern the factors that render one type of instruction superior to another. Although no single explanatory variable emerged, a suite of relevant factors can be identified. This chapter provides an overview of the research undertaken and specifically establishes five dimensions of characteristics that collectively define high quality instruction.

Video Analysis As A Tool For Understanding Science Instruction

Research on science instruction has revealed complex and nontrivial relations between instructional variables – including school system characteristics, teacher cognition and beliefs, teachers’ and students’ activities during instruction and last but not least, learning outcomes. To further investigate these relations, the development of respective models as well as appropriate research designs and methodologies are required. This will allow for tracing effects to the instructional level, shedding light on the well-known gap between teachers’ demands and students’ efforts as well as for the creation of interventions to overcome this gap. To this end, variables of teaching and learning have to be investigated using low- and high-inferent video analyses. Students’ and teachers’ behaviour holds valuable information for identifying cause-effect relations between what happens in the classroom and targeted outcomes.

Structure and development of pre-service physics teachers’ professional knowledge

ABSTRACT Teachers’ professional knowledge is considered one of the most important predictors of instructional quality. According to Shulman, such professional knowledge includes content, pedagogical content and pedagogical knowledge. Although recent research shed some light on the structure of the dimensions of professional knowledge, little is known how teacher education impacts pre-service physics teachers’ professional knowledge. In an effort to address this issue, we examined the content, pedagogical content and pedagogical knowledge of N = 200 pre-service physics teachers enrolled in different years of teacher education at 12 major teacher education universities in Germany. We used structural equation modelling (1) to examine the relations amongst pre-service physics teachers’ content, pedagogical content and pedagogical knowledge, (2) to explore how the three kinds of knowledge and their relations differ across different stages of teacher education and (3) to identify factors affecting the level of each component of professional knowledge. Our findings suggest that content, pedagogical content and pedagogical knowledge represent distinct types of knowledge. Furthermore, our findings show that in the first years of professional education, pedagogical content knowledge is more closely related with general pedagogical knowledge while in later years, it is more closely related with content knowledge, suggesting that it develops from a general knowledge about teaching and learning into knowledge about the teaching and learning of specific content. Finally, beyond school achievement and years of enrolment as predictors, we find in particular the amount of classroom observations to have a positive impact on the professional knowledge of pre-service physics teachers.

Introduction: Why Focus on Energy Instruction?

Energy is one of the most important ideas in all of science and is useful for predicting and explaining phenomena within every scientific discipline. Yet, there are substantive differences in how the energy concept is used across disciplines. While a particle physicist relies heavily on the idea that energy is conserved during interactions between subatomic particles, an ecologist is typically more concerned with the idea energy transfers across system boundaries.

Rasch-Analyse naturwissenschaftsbezogener Leistungstests

Empirische naturwissenschaftsdidaktische Forschung beruht wie die Forschung in den Naturwissenschaften zu einem betrachtlichen Teil auf der Auswertung von Messdaten. Typische Messgrosen sind kognitive Merkmale, wie z. B. das Wissen uber die Natur der Naturwissenschaften, aber auch affektive Merkmale, wie das Interesse an den Naturwissenschaften. Im Gegensatz zu vielen Messgrosen bei naturwissenschaftlichen Untersuchungen entzieht sich die uberwiegende Zahl der Messgrosen in der naturwissenschaftsdidaktischen Forschung einer direkten Messung. So lasst sich das Wissen uber Mechanik als solches nicht messen, sondern nur anhand der Bearbeitung entsprechender Aufgaben abschatzen. Daher spricht man in Anlehnung an die Sozialwissenschaften statt von Messgrosen auch von latenten Konstrukten. Als Instrumente zur Messung kognitiver Konstrukte (z. B. Fachwissen) werden ublicherweise Tests verwendet und zur Erfassung affektiver Konstrukte (z. B. Interesse) Fragebogen. Bei der inhaltlichen Entwicklung von Tests und Fragebogen geht die naturwissenschaftsdidaktische Forschung mit auserster Sorgfalt vor. Die Prufung der psychometrischen Qualitat der Instrumente findet allerdings erst in jungerer Zeit mehr Beachtung. Dieser Beitrag beschreibt, wie sich die Rasch-Analyse nutzen lasst, um einen vorliegenden Leistungstest zu analysieren, Verbesserungsmoglichkeiten zu identifizieren und zu einem standardisierten Instrument weiterzuentwickeln.

Refining a learning progression of energy

ABSTRACT This paper presents a revised learning progression for the energy concept and initial findings on diverse progressions among subgroups of sample students. The revised learning progression describes how students progress towards an understanding of the energy concept along two progress variables identified from previous studies – key ideas about energy and levels of conceptual development. To assess students understanding with respect to the revised learning progression, we created a specific instrument, the Energy Concept Progression Assessment (ECPA) based on previous work on assessing students’ understanding of energy. After iteratively refining the instrument in two pilot studies, the ECPA was administered to a total of 4550 students (Grades 8–12) from schools in two districts in a major city in Mainland China. Rasch analysis was used to examine the validity of the revised learning progression and explore factors explaining different progressions. Our results confirm the validity of the four conceptual development levels. In addition, we found that although following a similar progression pattern, students’ progression rate was significantly influenced by environmental factors such as school type. In the discussion of our findings, we address the non-linear and complex nature of students’ progression in understanding energy. We conclude with illuminating our researchs implication for curriculum design and energy teaching.

Conclusion and Summary Comments: Teaching Energy and Associated Research Efforts

The Energy Summit and the chapters in this book started with the premise that energy is both a critical disciplinary idea as well as a crosscutting concept, as elaborated in the Framework for K-12 Science Education (National Research Council 2012). Energy serves a central role in our everyday lives, as well as in all science disciplines. We were influenced by the argument presented in Framework for K-12 Science Education that energy is a critical concept that cuts across the disciplines and as such all learners need a solid understanding of this idea. However, the general population and many professionals, including K-12 science teachers, many science graduate students and scientists, lack a solid understanding of energy across all disciplines. Many of the challenges learners face in understanding the energy concept result not only because energy is a challenging concept but also because energy is seldom taught as a unifying idea; it is more likely taught using different language in different disciplines. For example, most learners never develop a rich conceptual understanding of what is meant by “energy is stored in chemical bonds.” This problematic situation most likely arises because there are substantive differences in how the energy concept is used across disciplines that result from shorthand usage of language. Although many scientists can translate between the various shorthand ways of using energy, this language is never clearly explained to students and practitioners, including teachers and curriculum developers. In fact, many graduate students do not fully understand the idea of energy. This has led to many misunderstandings of energy including “energy being stored in chemical bonds” as meaning “energy is released when bonds break.” As such, throughout the globe, we face challenges in teaching the energy concept, both because energy is such a challenging, misunderstood concept and different language is used to express different manifestations of it.