Øystein Guttersrud

An empirical-mathematical modelling approach to upper secondary physics

In this paper we describe a teaching approach focusing on modelling in physics, emphasizing scientific reasoning based on empirical data and using the notion of multiple representations of physical phenomena as a framework. We describe modelling activities from a project (PHYS 21) and relate some experiences from implementation of the modelling approach in Norwegian upper secondary physics classrooms.

Effects of school-based educational interventions for enhancing adolescents' abilities in critical appraisal of health claims: a systematic review

Background and Objective Adolescents are frequent media users who access health claims from various sources. The plethora of conflicting, pseudo-scientific, and often misleading health claims in popular media makes critical appraisal of health claims an essential ability. Schools play an important role in educating youth to critically appraise health claims. The objective of this systematic review was to evaluate the effects of school-based educational interventions for enhancing adolescents’ abilities in critically appraising health claims. Methods We searched MEDLINE, Embase, PsycINFO, AMED, Cinahl, Teachers Reference Centre, LISTA, ERIC, Sociological Abstracts, Social Services Abstracts, The Cochrane Library, Science Citation Index Expanded, Social Sciences Citation Index, and sources of grey literature. Studies that evaluated school-based educational interventions to improve adolescents’ critical appraisal ability for health claims through advancing the students’ knowledge about science were included. Eligible study designs were randomised and non-randomised controlled trials, and interrupted time series. Two authors independently selected studies, extracted data, and assessed risk of bias in included studies. Due to heterogeneity in interventions and inadequate reporting of results, we performed a descriptive synthesis of studies. We used GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) to assess the certainty of the evidence. Results Eight studies were included: two compared different teaching modalities, while the others compared educational interventions to instruction as usual. Studies mostly reported positive short-term effects on critical appraisal-related knowledge and skills in favour of the educational interventions. However, the certainty of the evidence for all comparisons and outcomes was very low. Conclusion Educational interventions in schools may have beneficial short-term effects on knowledge and skills relevant to the critical appraisal of health claims. The small number of studies, their heterogeneity, and the predominantly high risk of bias inhibit any firm conclusions about their effects. None of the studies evaluated any long-term effects of interventions. Future intervention studies should adhere to high methodological standards, target a wider variety of school-based settings, and include a process evaluation. Systematic Review Registration PROSPERO no. CRD42015017936.

Measuring ability to assess claims about treatment effects: the development of the ‘Claim Evaluation Tools’

Objectives To describe the development of the Claim Evaluation Tools, a set of flexible items to measure peoples ability to assess claims about treatment effects. Setting Methodologists and members of the community (including children) in Uganda, Rwanda, Kenya, Norway, the UK and Australia. Participants In the iterative development of the items, we used purposeful sampling of people with training in research methodology, such as teachers of evidence-based medicine, as well as patients and members of the public from low-income and high-income countries. Development consisted of 4 processes: (1) determining the scope of the Claim Evaluation Tools and development of items; (2) expert item review and feedback (n=63); (3) cognitive interviews with children and adult end-users (n=109); and (4) piloting and administrative tests (n=956). Results The Claim Evaluation Tools database currently includes a battery of multiple-choice items. Each item begins with a scenario which is intended to be relevant across contexts, and which can be used for children (from age 10  and above), adult members of the public and health professionals. People with expertise in research methods judged the items to have face validity, and end-users judged them relevant and acceptable in their settings. In response to feedback from methodologists and end-users, we simplified some text, explained terms where needed, and redesigned formats and instructions. Conclusions The Claim Evaluation Tools database is a flexible resource from which researchers, teachers and others can design measurement instruments to meet their own requirements. These evaluation tools are being managed and made freely available for non-commercial use (on request) through Testing Treatments interactive (testingtreatments.org). Trial registration numbers PACTR201606001679337 and PACTR201606001676150; Pre-results.

Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the 'Claim Evaluation Tools' database using Rasch modelling

Background The Claim Evaluation Tools database contains multiple-choice items for measuring people’s ability to apply the key concepts they need to know to be able to assess treatment claims. We assessed items from the database using Rasch analysis to develop an outcome measure to be used in two randomised trials in Uganda. Rasch analysis is a form of psychometric testing relying on Item Response Theory. It is a dynamic way of developing outcome measures that are valid and reliable. Objectives To assess the validity, reliability and responsiveness of 88 items addressing 22 key concepts using Rasch analysis. Participants We administrated four sets of multiple-choice items in English to 1114 people in Uganda and Norway, of which 685 were children and 429 were adults (including 171 health professionals). We scored all items dichotomously. We explored summary and individual fit statistics using the RUMM2030 analysis package. We used SPSS to perform distractor analysis. Results Most items conformed well to the Rasch model, but some items needed revision. Overall, the four item sets had satisfactory reliability. We did not identify significant response dependence between any pairs of items and, overall, the magnitude of multidimensionality in the data was acceptable. The items had a high level of difficulty. Conclusion Most of the items conformed well to the Rasch model’s expectations. Following revision of some items, we concluded that most of the items were suitable for use in an outcome measure for evaluating the ability of children or adults to assess treatment claims.

Validating the functional, communicative and critical health literacy scale using rasch modeling and confirmatory factor analysis

Background and Purpose: The functional, communicative, and critical health literacy (FCCHL) scale is widely used for assessing health literacy (HL) in people with chronic diseases, such as type 2 diabetes (T2DM). Despite related subscales, researchers continue to apply a consecutive modeling approach, treating the three subscales as independent. This article studies the psychometric characteristics of the FCCHL by applying multidimensional modeling approaches. Methods: Rasch modeling and confirmatory factor analyses were applied to responses (paper-and-pencil) from 386 adults with T2DM. Results: Using a six-point rating scale and a three-dimensional Rasch model, this study found that a 12-item version of the FCCHL reduced within-item bias and improved subscale reliability indexes. Conclusion: This study suggests a parsimonious 12-item version of the FCCHL. The data fit a three-dimensional Rasch model best.

Observing the World through Your Own Lenses – The Role of Perceived Adaptability for Epistemological Beliefs about the Development of Scientific Knowledge

Students are exposed to vast amounts of information and knowledge that is rapidly changing. This exposure requires them to be adaptive, that is, to constantly adjust their thinking, behavior, and even their affect to successfully solve information-rich and knowledge-lean problems. Considering these developments, the purpose of the present study is twofold: First, it is aimed at exploring the link between students’ beliefs about their adaptability in an ever-changing world and their beliefs about the changing nature of scientific knowledge, thus linking two educationally relevant belief systems. Second, this study further explores validity issues related to the well-established and commonly used “Epistemological Beliefs about the Development of Scientific Knowledge (EBDE)” scale. Performing structural equation modeling on a large-scale data set of 1,662 Norwegian tenth-grade students, we estimated the correlations among different aspects of adaptability (i.e., cognitive-behavioral and affective-emotional adaptability) and EBDE. Moving beyond these correlations, we tested whether students’ perceived adaptability had an impact on the functioning of EBDE items by means of moderated factor analysis. Our analyses revealed that adaptability was associated with sophisticated EBDE in science, and the EB scale functioned differently with respect to different adaptability scores. The results of this study indicate that students perceive the development of scientific knowledge through the lenses of their own adaptability. Furthermore, the differential functioning of the EBDE scale challenges its validity.

Establishing the HLS-Q12 short version of the European Health Literacy Survey Questionnaire: latent trait analyses applying Rasch modelling and confirmatory factor analysis

BackgroundThe European Health Literacy Survey Questionnaire (HLS-EU-Q47) is widely used in assessing health literacy (HL). There has been some controversy whether the comprehensive HLS-EU-Q47 data, reflecting a conceptual model of four cognitive domains across three health domains (i.e. 12 subscales), fit unidimensional Rasch models. Still, the HLS-EU-Q47 raw score is commonly interpreted as a sufficient statistic. Combining Rasch modelling and confirmatory factor analysis, we reduced the 47 item scale to a parsimonious 12 item scale that meets the assumptions and requirements of objective measurement while offering a clinically feasible HL screening tool. This paper aims at (1) evaluating the psychometric properties of the HLS-EU-Q47 and associated short versions in a large Norwegian sample, and (2) establishing a short version (HLS-Q12) with sufficient psychometric properties.MethodsUsing computer-assisted telephone interviews during November 2014, data were collected from 900 randomly sampled individuals aged 16 and over. The data were analysed using the partial credit parameterization of the unidimensional polytomous Rasch model (PRM) and the ‘between-item’ multidimensional PRM, and by using one-factorial and multi-factorial confirmatory factor analysis (CFA) with categorical variables.ResultsUsing likelihood-ratio tests to compare data-model fit for nested models, we found that the observed HLS-EU-Q47 data were more likely under a 12-dimensional Rasch model than under a three- or a one-dimensional Rasch model. Several of the 12 theoretically defined subscales suffered from low reliability owing to few items. Excluding poorly discriminating items, items displaying differential item functioning and redundant items violating the assumption of local independency, a parsimonious 12-item HLS-Q12 scale is suggested. The HLS-Q12 displayed acceptable fit to the unidimensional Rasch model and achieved acceptable goodness-of-fit indexes using CFA.ConclusionsUnlike the HLS-EU-Q47 data, the parsimonious 12-item version (HLS-Q12) meets the assumptions and the requirements of objective measurement while offering clinically feasible screening without applying advanced psychometric methods on site. To avoid invalid measures of HL using the HLS-EU-Q47, we suggest using the HLS-Q12. Valid measures are particularly important in studies aiming to explain the variance in the latent trait HL, and explore the relation between HL and health outcomes with the purpose of informing policy makers.

Teaching and Learning Representations in Upper Secondary Physics

This study aimed at developing teaching material and strategies emphasising representations and modelling in upper secondary physics teaching. Representations are tools for modelling and understanding representations and how these can be manipulated is therefore believed to be a prerequisite for developing modelling skills. The study, accordingly, wanted mathematical representations and modelling to be an underlying focus in physics lessons. The study worked with physics teachers over 2 years to develop and trial teaching material and strategies, and in a third year tested this material in an intervention with experimental and control classes. Research questions asked what effects the material had on teaching, how teacher conceptualised the physics curriculum and teaching from using the material, and what influence the teaching had on students’ learning. The methodology included classroom observations with video recordings and note taking, questionnaires to teachers and students, analysis of teachers’ lesson plans, notes from discussions with teachers, interviews with teachers and an achievement test to measure learning. Student questionnaire data showed increased focus on representation in the experimental classrooms as compared to the control group, but no better effect in students’ achievement score in test for using representations. Significant differences were found between schools independently of being in the intervention or not, which makes us believe teacher and teaching characteristics were more important than the teaching material. Classroom observations, for example, showed that teachers varied in doing explicit or implicit teaching of representations, having a dialogical or authoritative style of teaching, and having an interactive or no-interactive communicative approach.