Beatrijs De Fraine

A New Study on Educational Effectiveness in Secondary Schools in Flanders: An Introduction

As an introduction to the articles of Opdenakker, Van Damme, De Fraine, Van Landeghem, and Onghena (2002) and Van Landeghem, Van Damme, Opdenakker, De Fraine, and Onghena (2002) in this issue, we give some background information on a new study on educational effectiveness in secondary schools, and on the variables measured in that study that are relevant to the 2 articles mentioned. We conclude with some information on the system of secondary education in Flanders.

The effect of schools and classes on mathematics achievement

In this article, the effects of secondary schools, teachers, and classes on mathematics achievement are explored. The results of multilevel analyses indicate important recruitment differences between schools and classes within schools. After controlling for this, the group composition at the class level remained very important for the explanation of differences in mathematics achievement. Also the learning climate in the class, which correlates positively with group composition, explained an additional part of the variance. The effect of other educational process variables disappeared when group composition variables were entered into the model. At the school level, the proportion of girls was positively related to mathematics achievements. Indications of differential effectiveness of classes and schools and of heteroscedasticity related to SES, average class SES, average class SES, sex, and learning climate were found.

The Effect of Schools and Classes on Language Achievement.

This study addresses the effects of secondary schools and classes on language achievement in Flanders, Belgium. The results of a three-level analysis (students within classes within schools) indicate that the group composition at the class level is very important. In classes with a high average initial cognitive ability or a large proportion of girls, the language achievement is higher. These compositional effects are discussed with reference to type ‘A’ and type ‘B’ effects. The analyses show that group composition is more important than educational practices in accounting for differences in language achievement. With whom one is taught has a larger impact than how one is taught. Indications of differential effectiveness of classes related to prior achievement were found, with greater variations in effectiveness between classes for pupils of low prior achievement.

The Effect of Schools and Classes on Noncognitive Outcomes

This article reports on the effects of individual background characteristics, classes, and schools on 4 noncognitive outcomes at the end of the 1st cycle secondary education in Flanders. The outcomes are: the degree to which the student feels at home in the school environment, the extent to which the student does his/her best for the school work, the academic self-concept, and the social integration in the class. Firstly, the raw class and school effects are estimated. Secondly, a broad set of background variables is introduced to isolate the net class and school effects. Thirdly, class and school composition variables are introduced. Finally, the additional predictive power of a number of other student, class, and school characteristics is documented.

Season of birth and school success in the early years of primary education

Several studies have reported significant relationships between children’s season of birth and measures of their academic success (i.e., the ‘season of birth effect’). Whereas most of these studies were cross‐sectional, the current study uses growth curve modelling to analyse longitudinal data on 3,187 children in Flemish primary education. The results indicate season of birth effects on both grade retention and mathematics achievement during the first two years of primary school. Because the Flemish cut‐off date is 31 December, children born in the fourth quarter (October‐November‐December) invariably are among the youngest in their grade age group. Almost 20% of these children were found to have been retained or referred to special education by the end of Grade 2, whereas for children born in the first quarter (January‐February‐March), this was only 6.34%. First quarter‐born children also showed moderately higher mathematics achievement at the start of first grade. During the next two school years, this achievement gap between children born in the first and the fourth quarter narrowed significantly. Finally, differentiated instruction was not found to be related to the decrease of the season of birth effect.

The Consequence of Ignoring a Level of Nesting in Multilevel Analysis: A Comment.

This short contribution is a comment on M. Moerbeeks exploration of consequences of ignoring a level of clustering in a multilevel model, which was published in the first issue of the 2004 volume of Multivariate Behavioral Research. After having recapitulated the framework and extended the results of Moerbeeks study, we formulate two critical notes. First, we point at the incompleteness of the conclusions drawn by Moerbeek from the analytical work. The second note is concerned with the limitations of the framework itself.

School effects on the development of motivation toward learning tasks and the development of academic self-concept in secondary education: a multivariate latent growth curve approach

The main objective of the present study is to gain more insight into the school effects on the development of 2 noncognitive student outcomes, namely, the motivation toward learning tasks and the academic self-concept, and, more specifically, on the consistency of the school effects on these 2 outcomes. Data were drawn from the “Longitudinaal Onderzoek Secundair Onderwijs” (LOSO)-project and consisted of 2,687 students in 50 schools that were tested repeatedly at 4 different time points during secondary education (Grades 7–12). A multivariate multilevel latent growth curve model was used to analyze the data. The results showed that the school effects on the development of the noncognitive outcomes were considerable, and, more importantly, the school effects were larger on growth than on student status. Schools that were effective for the development of the motivation toward learning tasks also proved to be effective for the development of academic self-concept. However, the consistency of the school effects resulted largely from intake differences between schools.

Multilevel Design Efficiency in Educational Effectiveness Research.

In educational effectiveness research, multilevel data analyses are often used because research units (most frequently, pupils or teachers) are studied that are nested in groups (schools and classes). This hierarchical data structure complicates designing the study because the structure has to be taken into account when approximating the accuracy of estimation and the power for statistical testing, which should be sufficient to reach meaningful conclusions. Accuracy and power, both referred to as efficiency, can be optimized by carefully choosing the number of units to sample at each of the levels, taking into account the available resources and costs of sampling at these levels. We complement the findings that are found in the literature with regard to designing multilevel studies and propose a simulation approach that can be used to help making study-specific decisions.

Disentangling instructional roles: the case of teaching and summative assessment

While in some higher education contexts a separation of teaching and summative assessment is assumed to be self-evident, in other contexts the opposite is regarded to be obvious. In this article the different arguments supporting either position are analyzed. Based on a systematic literature review, arguments for and against are classified at the micro-, meso- and macro-level. Articles specifically discuss the way in which making use of an external examiner influences the learning environment, the quality assurance policy and the policy of an institution with respect to the outside world (users, other institutions, etc.). In addition, the literature reveals that the suggested arguments and positions for and against disentangling instructional roles are highly context specific. Strikingly, empirical evidence for these arguments is rather scarce. There is a need, therefore, for a systematic research program which investigates the influence of the large and diverse set of assumptions and presuppositions on the disentangling of instructional roles.