Moritz Krell

Testing Models: A Key Aspect to Promote Teaching Activities Related to Models and Modelling in Biology Lessons?

This study investigated biology teachers’ (N = 148) understanding of models and modelling (MoMo), their model-related teaching activities and relations between the two. A framework which distinguishes five aspects of MoMo in science (nature of models, multiple models, purpose of models, testing models and changing models) served as a theoretical background. Teachers’ understanding of MoMo was assessed using constructed-response items which were analysed qualitatively based on a coding scheme. The biology teachers mainly expressed a limited understanding of models as copies or idealised depictions used to show or to explain something. Model-related teaching activities were assessed through rating-scale items. The findings propose that models are primarily generated in biology lessons to show or to explain something but are rarely contrasted with other models, evaluated and modified. Significant correlations between teachers’ understanding of the aspect testing models and their intensity of model-related teaching activities were found. This suggests that the aspect testing models is a key aspect of promoting teaching activities related to MoMo in biology lessons. The findings are discussed with respect to relevant literature about MoMo in science education and educational implications are provided.

Using polytomous IRT models to evaluate theoretical levels of understanding models and modeling in biology education

Forced choice-tasks have been developed to assess students’ (N = 901) understanding of models and modeling in biology based on a theoretical structure differentiating five aspects and three levels of understanding (Upmeier zu Belzen & Kruger, 2010). The data have been analyzed by using the partial credit and the rating scale model to evaluate the assumption of an increasing degree of difficulty from level I to level III in each aspect. The findings suggest (1) that the levels of understanding are not equi distant across all aspects and (2) that the theoretically developed levels of understanding are in fact ordered by difficulty. However the latter issue needs further investigations since the present findings are not clear for all five aspects of the theoretical structure.

Evaluating an instrument to measure mental load and mental effort using Item Response Theory

Measurement of cognitive load (CL) is seen as a problematic issue since no consensus about appropriate instruments has been reached. In this study, a rating scale instrument to measure mental load (ML; 6 items) and mental effort (ME; 6 items) is evaluated using Item Response Theory. N=506 students self-reported their amount of ML and ME after working on a standardised multiple choice-test. The findings propose to separately measure ML and ME instead of CL in general. Furthermore, the 7-point rating scale had to be reduced post-hoc to a 3-point scale in order to reach consistent information. Finally, there was a significant (negative) correlation between ML and test performance, but not be tween ME and test performance.

Fostering pre-service teachers’ views about nature of science: evaluation of a new STEM curriculum

Background: An elaborated understanding of Nature of Science (NOS) is seen as an important part of scientific literacy. In order to enable teachers to adequately discuss NOS in their lessons, various approaches have recently been employed to improve teachers’ understanding of NOS. Purpose: This study investigated the effect of participating in a newly developed Science, Technology, Engineering and Mathematics (STEM) curriculum at the Freie Universität Berlin (Germany) on pre-service teachers’ NOS views. Program description: In the new STEM curriculum, two versions of explicitly teaching NOS, which are discussed in the literature, have been adopted: the pre-service teachers explicitly reflect upon nature and history of science (version one) as well as conduct own scientific investigations (version two). Sample: N = 76 pre-service teachers from different semester levels (cross-sectional study) who participated in the new STEM curriculum took part in this study (intervention group). As control groups, students who did not partake in the new curriculum participated (pre-service primary (N = 134), science (N = 198), and no-science (N = 161) teachers). Design and methods: In order to allow an economic assessment, a testing instrument with closed-item formats was developed to assess the respondents’ views about six NOS aspects. Results: The intervention group shows significantly more elaborated NOS views than a relevant control group (p < .01, g = .48). Additionally, a one-way ANOVA reveals a positive effect of semester level on NOS views for the intervention group (p < .01; η² = .16) but not for the control groups. Conclusion: The findings support evidence suggesting that explicit approaches are effective when fostering an informed understanding of NOS. More specifically, a sequence of both versions of explicitly teaching NOS discussed in the literature seems to be a way to successfully promote pre-service teachers’ NOS understanding.

Evaluating an instrument to measure mental load and mental effort considering different sources of validity evidence

Abstract This study evaluates a 12-item instrument for subjective measurement of mental load (ML) and mental effort (ME) by analysing different sources of validity evidence. The findings of an expert judgement (N = 8) provide evidence based on test content that the formulation of the items corresponds to the meaning of ML and ME. An empirical study was conducted in which secondary school students (N = 602) worked on multiple choice (mc)-tasks and thereafter using the developed instrument to self-report ML and ME. The findings show that the instrument reliably measures the two positively correlated constructs ML and ME (evidence based on internal structure). Students working on mc-tasks with high complexity self-reported higher amounts of ML and ME than students working on mc-tasks with low complexity, and there is a negative relation between test performance and ML (evidence based in relation to other variables). Implications for educational assessment and limitations of the study are discussed.

University students’ meta-modelling knowledge

Abstract Background: As one part of scientific meta-knowledge, students’ meta-modelling knowledge should be promoted on different educational levels such as primary school, secondary school and university. This study focuses on the assessment of university students’ meta-modelling knowledge using a paper–pencil questionnaire. Purpose: The general purpose of this study was to assess and to describe university students’ meta-modelling knowledge. More specifically, it was analysed to what extent the meta-modelling knowledge, as expressed in a questionnaire, depends on the scientific discipline to which university students relate their answer and the concrete model to which they refer while answering. Sample: N = 184 students from one German university voluntarily participated in this study. Design and methods: The questionnaire was developed based on a theoretical framework for model competence and includes constructed response items asking about the purpose of models, ways for testing models and reasons for changing models. Students written answers were first analysed qualitatively based on the theoretical framework to decide whether they expressed advanced understandings or not. Further analyses then were conducted quantitatively. Results: Findings suggest that only few university students possess an advanced meta-modelling knowledge. However, significant more students who relate their answers to the STEM-disciplines expressed advanced understandings than those who referred to social sciences or linguistics/philology. Furthermore, university students who expressed an advanced meta-modelling knowledge referred to rather abstract kinds of models in order to explain their view. Conclusions: The present study supports the assumption that meta-modelling knowledge may be situated and contextualised. Both the scientific discipline and the concrete model to which university students refer seem to be relevant item features influencing university students’ expressed meta-modelling knowledge. Implications for assessment and teaching are discussed in the article.

Methodical challenges concerning the Draw-A-Scientist Test: a critical view about the assessment and evaluation of learners’ conceptions of scientists

ABSTRACT Students’ and pre-service teachers’ conceptions of scientists have been assessed in a variety of studies. One of the most commonly used instruments is the Draw-A-Scientist Test (DAST) which offers the advantage that no verbal skills are needed by the participants. In some studies, methodical challenges related to the DAST have been discussed; for example, the lack of drawing abilities among the subjects or the impact of the prompt itself on the drawings which lead to invalid interpretations. This study aims to evaluate the eligibility of the DAST or similar instruments to validly assess conceptions of scientists and their work. Pre-service science teachers in two cohorts (N1 = 79, N2 = 101) were prompted to draw their conceptions of a scientist, the location at which the scientist is working, and his/her scientific activity. The participants of cohort 2 were also asked to give written descriptions of their drawings and information on the conditions while they were drawing. From the results, several points of methodical criticism can be made which question the valid interpretation of DAST-drawings. Instead of using the DAST, we suggest developing and evaluating other formats, such as open-ended or closed-ended instruments which could produce a more validly interpretable assessment of individuals’ conceptions in this field.