Rüdiger Tiemann

Factors of problem-solving competency in a virtual chemistry environment: The role of metacognitive knowledge about strategies

The ability to solve complex scientific problems is regarded as one of the key competencies in science education. Until now, research on problem solving focused on the relationship between analytical and complex problem solving, but rarely took into account the structure of problem-solving processes and metacognitive aspects. This paper, therefore, presents a theoretical framework, which describes the relationship between the components of problem solving and strategy knowledge. In order to assess the constructs, we developed a virtual environment which allows students to solve interactive and static problems. 162 students of grade 10 and the upper secondary level completed the tests within a cross-sectional survey. In order to investigate the structure of problem-solving competency, we established measurement models representing different theoretical assumptions, and evaluated model fit statistics by using confirmatory factor analyses. Results show that problem-solving competency in virtual environments comprises to three correlated abilities: achieving a goal state, systematical handling of variables, and solving analytical tasks. Furthermore, our study provides empirical evidence on the distinction between analytical and complex problem solving. Additionally, we found significant differences between students of grades 10 and 12 within the problem-solving subscales, which could be explained by gaming experience and prior knowledge. These findings are discussed from a measurement perspective. Implications for assessing complex problem solving are given.

Assessing students’ abilities in processes of scientific inquiry in biology using a paper-and-pencil test

The aim of the study was to describe, categorise and analyse students’ (aged 14–16) processes of scientific inquiry in biology and chemistry education. Therefore, a theoretical structure for scientific inquiry for both biology and chemistry, the VerE model, was developed. This model consists of nine epistemological acts, which combine processes of scientific thinking and inquiry methods. Based on the theoretical structure, a paper-and-pencil test was developed to investigate the students’ abilities in the acts of scientific inquiry. Each of the nine acts was operationalised to generate multiple-choice items. For each act, ten items were constructed. In total, ninety items per subject were tested in a field study to evaluate their psychometric quality. The article focuses on the outcomes for testing in biology. In biology, 537 students were tested with a paper-and-pencil test, following a multi-matrix design in which each student solved twenty-seven items. Data from 260 students have been analysed so far. Seventy-five items showed satisfactory item characteristics. The distribution of the items’ difficulties fits the students’ abilities appropriately. We conclude that theory-driven epistemological acts can be operationalised in tasks that assess students’ abilities in scientific inquiry.

Predicting Students’ Skills in the Context of Scientific Inquiry with Cognitive, Motivational, and Sociodemographic Variables

Research on predictors of achievement in science is often targeted on more traditional content-based assessments and single student characteristics. At the same time, the development of skills in the field of scientific inquiry constitutes a focal point of interest for science education. Against this background, the purpose of this study was to investigate to which extent multiple student characteristics contribute to skills of scientific inquiry. Based on a theoretical framework describing nine epistemological acts, we constructed and administered a multiple-choice test that assesses these skills in lower and upper secondary school level (n = 780). The test items contained problem-solving situations that occur during chemical investigations in school and had to be solved by choosing an appropriate inquiry procedure. We collected further data on 12 cognitive, motivational, and sociodemographic variables such as conceptual knowledge, enjoyment of chemistry, or language spoken at home. Plausible values were drawn to quantify students’ inquiry skills. The results show that students’ characteristics predict their inquiry skills to a large extent (55%), whereas 9 out of 12 variables contribute significantly on a multivariate level. The influence of sociodemographic traits such as gender or the social background becomes non-significant after controlling for cognitive and motivational variables. Furthermore, the performance advance of students from upper secondary school level can be explained by controlling for cognitive covariates. We discuss our findings with regard to curricular aspects and raise the question whether the inquiry skills can be considered as an autonomous trait in science education research.

Heuristic chemistry—a qualitative study on teaching domain-specific strategies for the six-electron case

We investigate the efficiency of domain-specific heuristic strategies in mastering and predicting pericyclic six-electron rearrangements. Based on recent research findings on these types of reactions a new concept has been developed that should help students identify and describe six-electron rearrangements more readily in complex molecules. The goal of this qualitative study with chemistry majors is to understand the way students cope with this new concept depending on their prior knowledge, and to reveal the merits and limitations of this approach. The results suggest that the use of domain-specific heuristic strategies provides the students with process-orientated thinking skills to identify six-electron rearrangements and to determine as well as predict reaction mechanisms and outcomes. The explicit emphasis on recurrent patterns and structure–property relationship fosters the conceptual thinking of the learner.

Assessing scientific reasoning: a comprehensive evaluation of item features that affect item difficulty

The aim of this study was to improve the criterion-related test score interpretation of a text-based assessment of scientific reasoning competencies in higher education by evaluating factors which systematically affect item difficulty. To provide evidence about the specific demands which test items of various difficulty make on pre-service teachers’ scientific reasoning competencies, we applied a general linear mixed model which allows estimation of the impact of item features on the response observations. The item features had been identified during a standard setting process. Results indicate important predictive potential of one formal item feature (length of response options), two features based on cognitive demands (processing data from tables, processing abstract concepts) and one feature based on solid knowledge (specialist terms). The revealed predictive potential of item features was in accordance with the cognitive demands operationalised in our competence model. Thus, we conclude that the findings support the validity of our interpretation of the test scores as measures of scientific reasoning competencies.

The Development of Scientific Strategy Knowledge Across Grades

In this study, we developed a new test on scientific strategy knowledge and investigated the construct validity of the resulting test scores. Moreover, measurement invariance across grade levels has been analyzed to ensure the generalizability of the assessment. Furthermore, convergent and discriminant validity were investigated. A total of N = 1,182 German high school students of Grade Levels 8, 10, and 12 completed tasks on strategy knowledge, fluid intelligence, content knowledge, interest in science, and scientific self-concept within a cross-sectional study. Multigroup confirmatory factor analysis was used to check for measurement invariance. Our results show that scalar invariance holds across grades and that there are significant differences in performance favoring students of higher grade levels. Furthermore, fluid intelligence and content knowledge are relevant predictors of strategy knowledge, whereas gender and motivational constructs do not show significant effects. Implications for developmental studies on strategy knowledge and assessment practice are discussed.

Die Fragebogenmethode, ein Klassiker der empirischen didaktischen Forschung

Die Entwicklung von Fragebogen zur Erhebung von Personlichkeitsmerkmalen, wie Uberzeugungen oder Interessen, stellt einen wichtigen Bestandteil der empirischen didaktischen Forschung dar. Sollen objektive, reliable und valide Fragebogen entwickelt werden, sind Kenntnisse uber adaquate Vorgehensweisen der Fragebogenkonstruktion und geeignete statistische Auswertungsmethoden unabdingbar. Dieser Beitrag beschreibt ein mogliches Vorgehen bei der Entwicklung und Auswertung von Fragebogen am Beispiel eines in der Fachdidaktik erprobten Instruments zum Thema „Standards im Chemieunterricht“. Das grundsatzliche methodische Vorgehen ist auf andere Fragebogenkonstruktionen ubertragbar.

Chemistry Student Teachers' Scientific Inquiry Competencies

We constructed paper-and-pencil test-items to assess chemistry student teachers’ scientific inquiry competencies. We focused on the inquiry method experiment, divided into three reasoning steps (question & hypothesis; planning & performing; analysis & reflection). Test revision resulted in a test of 20 items, which was applied to a sample of undergraduate and graduate chemistry student teachers. The first explorative assessment ( N = 89) based on that instrument (1) revealed acceptable item characteristics, such as an appropriate range of item difficulties (.48 < P i < .84) and only slightly unsatisfactory item discrimination parameters (.19 < r it < .65). The study (2) obtained preliminary information on the high prognostic potential of the planned degree.

Covariates of complex problem solving competency in chemistry

The ability to solve complex and real-life problems is one of the key competencies in science education. Different studies analyzed the relationships between complex problem solving (CPS) and covari ates such as intelligence, prior knowledge, and motivational constructs on a manifest level. Additionally, research findings indicate that intelligence and prior knowledge are substantial predictors of CPS. Due to the interconnections between covariates, the relationships between CPS and covariates are quite complex. Therefore, we propose a model which describes these relations by taking direct and indirect effects into account. All analyses are based on structural equation modeling. Results show that the proposed model represents the data with substantial goodness-of-fit statistics and explanation of variance. Intelligence, domain-specific prior knowledge, computer familiarity, and attendance in advanced chemistry courses are direct predictors of CPS, while interest and scientific self-concept show indirect effects.