Trude Nilsen

The quest for comparability: Studying the invariance of the teachers' sense of self-efficacy (TSES) measure across countries

Teachers’ self-efficacy is an important motivational construct that is positively related to a variety of outcomes for both the teachers and their students. This study addresses challenges associated with the commonly used ‘Teachers’ Sense of Self-Efficacy (TSES)’ measure across countries and provides a synergism between substantive research on teachers’ self-efficacy and the novel methodological approach of exploratory structural equation modeling (ESEM). These challenges include adequately representing the conceptual overlap between the facets of self-efficacy in a measurement model (cross-loadings) and comparing means and factor structures across countries (measurement invariance). On the basis of the OECD Teaching and Learning International Survey (TALIS) 2013 data set comprising 32 countries (N = 164,687), we investigate the effects of cross-loadings in the TSES measurement model on the results of measurement invariance testing and the estimation of relations to external constructs (i.e., working experience, job satisfaction). To further test the robustness of our results, we replicate the 32-countries analyses for three selected sub-groups of countries (i.e., Nordic, East and South-East Asian, and Anglo-Saxon country clusters). For each of the TALIS 2013 participating countries, we found that the factor structure of the self-efficacy measure is better represented by ESEM than by confirmatory factor analysis (CFA) models that do not allow for cross-loadings. For both ESEM and CFA, only metric invariance could be achieved. Nevertheless, invariance levels beyond metric invariance are better achieved with ESEM within selected country clusters. Moreover, the existence of cross-loadings did not affect the relations between the dimensions of teachers’ self-efficacy and external constructs. Overall, this study shows that a conceptual overlap between the facets of self-efficacy exists and can be well-represented by ESEM. We further argue for the cross-cultural generalizability of the corresponding measurement model.

School emphasis on academic success: exploring changes in science performance in Norway between 2007 and 2011 employing two-level SEM

We study whether changes in school emphasis on academic success (SEAS) and safe schools (SAFE) may explain the increased science performance in Norway between TIMSS 2007 and 2011. Two-level structural equation modelling (SEM) of merged TIMSS data was used to investigate whether changes in levels of SEAS and SAFE mediate the changes in science performance. Two mediation models were fitted, one using subdomain scores of science as manifest dependent variables and one in which these scores are indicators of a latent science performance variable. The change in the latent science variable was fully mediated by SEAS, but this model did not explain changes in earth science performance, which increased more than the other subdomains. In the model with subdomain scores as manifest dependent variables, SEAS mediated the increased performance of all 4 subdomains of science. SAFE did not explain increased science performance but did have a positive impact on SEAS.

The Impact of School Climate and Teacher Quality on Mathematics Achievement: A Difference-in-Differences Approach

The aim of the study was to investigate causal effects of aspects of teacher quality and school climate on mathematics achievement through use of country-level longitudinal data. By investigating within-country change over time, biasing influence from omitted variables in the form of fixed country characteristics is avoided, thereby increasing the likelihood of making correct causal inferences. Data from 38 countries participating in both TIMSS 2007 and TIMSS 2011 were analyzed with structural equation modeling techniques, using both latent and manifest variables. The analyses focused aspects of teacher quality (educational level, teaching experience and major academic discipline studied, professional development, and self-efficacy) and an aspect of school climate referred to as school emphasis on academic success (SEAS). Results showed that the teachers’ attained level of education had effects on mathematics achievement. Quite substantial effects of professional development on student achievement were also identified. Teacher self-efficacy, as assessed by self-reports of preparedness for teaching in different domains, showed a weakly positive, but insignificant relation to student achievement. The teacher characteristics years of teaching experience and major academic discipline studied had no effect on student achievement. SEAS did not satisfy ideals of unidimensionality, and only items reflecting parental support for student achievement and students’ desire to perform well were significantly related to student achievement. OECD and non-OECD countries showed similar results and could not be differentiated.

The Relations Among School Climate, Instructional Quality, and Achievement Motivation in Mathematics

Instructional quality is considered to be an important classroom variable, as it is significantly related to student achievement and motivation in mathematics. Existing studies in educational effectiveness furthermore identified a positive relation between instructional quality and school climate, suggesting that the school environment plays a significant role in teachers’ instructional practices. In order to bring together these two core findings, the relations among different aspects of school climate, instructional quality, and students’ achievement motivation for the TIMSS 2011 grade eight mathematics data sets comprising 50 countries are investigated. In particular, the role of instructional quality as a potential mediator between school climate and student motivation is examined, thereby focusing on three aspects of school climate (emphasis on academic success, safety, and order in schools) and three aspects of achievement motivation (self-concept, intrinsic value, and extrinsic value). In general, there was a significant positive relation between instructional quality and achievement motivation at the classroom level in mathematics; in some countries, a partial mediation of instructional quality between school climate and achievement motivation was apparent. Four main patterns of relations occurred. These findings are discussed with respect to implications for educational effectiveness research.

Conceptual Framework and Methodology of This Report

In this volume, five separate studies examine differing aspects of relations between teacher quality, instructional quality and learning outcomes across countries, taking into account context characteristics such as school climate. The 2007 and 2011 TIMSS (Trends in Mathematics and Science Study) cycles provided the research data. These five studies cover grade four or grade eight students and their teachers, including cognitive or affective-motivational learning outcomes. This introductory chapter describes the overall conceptual framework and the research questions posed by each chapter, and outlines the general design features of TIMSS. Key constructs, and common methodological issues among the five studies are discussed, and this introduction concludes with an overview of all chapters.

Exploring Content Knowledge: Country Profile of Science Strengths and Weaknesses in TIMSS. Possible Implications for Educational Professionals and Science Research

ABSTRACT This study offers curriculum developers, teachers, and science education researchers a fine-grained profile on strengths and weaknesses in specific science domains and topics. The study involved a representative sample of 3844 Norwegian pupils in grade 8. Their responses on 216 TIMSS items in 18 topics from 4 science domains were modelled in a hierarchical item response model. An internal comparison identified topics that were relatively harder or easier compared to other topics for Norwegian pupils, and an external comparison identified topics that were relatively harder or easier for the Norwegian pupils compared with the average of TIMSS participating countries. Interpretation of the profile necessitates contextualisation; hence, these strengths and weaknesses, as well as their plausible explanations and possible implications, are discussed from the perspectives of curriculum development, teacher training, and science education research.

Standard Setting in PISA and TIMSS and How These Procedures Can Be Used Nationally

In this chapter, we compare and discuss similarities and differences in the way the two large-scale international studies, PISA and TIMSS, formulate and set descriptions of standards. Although the studies use similar methods, different decisions have been made regarding the nature and properties of the final descriptions of student achievement. In addition to this overview, we treat PISA and TIMSS as case studies in order to illustrate an under-researched area in standard setting: the nature of and empirical basis for the development of performance level descriptors (PLDs). We conclude by discussing how these procedures may be relevant for formulating useful standards in tests and assessments in the Norwegian context.