Anneke Timmermans

Conceptual and empirical differences among various value-added models for accountability

Accountability systems in education generally include indicators of student performance. However, these indicators often differ considerably among the various systems. More and more countries try to include value-added measures, mainly because they do not want to hold schools accountable for differences in their initial intake of students. This study presents a conceptual framework of these value-added measures, resulting in an overview of 5 different types. Using data from Dutch secondary schools, we empirically provide estimates of these different measures. Our analyses show that the correlation between the different types of school effects estimated is rather high, but that the different models implicate different results for individual schools. Based on theoretical considerations, arguments are given to use the following indicators in the value-added accountability models: prior achievement, student-level background characteristics, and compositional characteristics of the student population.

Accurate, inaccurate, or biased teacher expectations: Do Dutch teachers differ in their expectations at the end of primary education?

BACKGROUND In several tracked educational systems, realizing optimal placements in classes in the first year of secondary education depends on the accuracy of teacher expectations. AIMS The aim of this study was to investigate between-teacher differences in their expectations regarding the academic aptitude of their students. SAMPLE The sample consisted of 500 teachers (classes) who provided their expectations of 7,550 students in the final grade of Dutch primary education. METHODS We analysed the extent to which teachers differed in their expectations and in what contexts their expectations were biased, using multilevel random slope models. RESULTS Multilevel analysis showed teacher expectation bias to be related to gender and socio-ethnic background of students. The differences among teachers in expectations for Turkish, Moroccan, and other foreign students with low-educated parents were larger than the average teacher expectation bias for these groups in the sample. When student characteristics were controlled for, we found that the teachers in our sample had higher expectations for students in high-performing classes or classes with only a small proportion of students from low-SES families. Teacher expectation bias for demographic groups, however, was found to be independent of the class population. CONCLUSIONS The results indicate the importance of the teacher and the necessity of using multilevel models when investigating the complex nature of between-teacher differences in expectations of student performance.

The impact of student composition on schools’ value-added performance: a comparison of seven empirical studies

In many countries, policy makers struggle with the development of value-added indicators of school performance for educational accountability purposes and in particular with the choice whether school context measured in the form of student composition variables should be included. This study investigates differences between 7 empirical studies that have looked at changes in value-added indicators through the inclusion of school composition variables. Differences were found in the strength of the association between Type A (student-level covariates) and Type B (student- and school-level covariates) school effect models in various regions, varying between 0.74 and 0.99 for secondary education. Important differences were found between the estimate of a school’s effectiveness by using a Type A or a Type B value-added model in secondary education in The Netherlands, Belgium, Chile, and Australia and in US and UK primary education. However, few differences were found in other cases.

In Search of Value Added in the Case of Complex School Effects

In traditional studies on value-added indicators of educational effectiveness, students are usually treated as belonging to those schools where they took their final examination. However, in practice, students sometimes attend multiple schools and therefore it is questionable whether this assumption of belonging to the last school they attended can be made. Furthermore, the schools attended earlier by students might have long-term effects on their subsequent performance. Using data from Dutch primary and secondary schools, multiple membership models and cross classification multilevel models were estimated to explore the effects of student mobility and long-term primary school effects on the estimated value added of secondary schools. Long-term effects of primary schools did not change the estimated value added of secondary schools. On the other hand, allowing students to be a member of multiple secondary schools changed the estimated effectiveness of these schools especially for schools in the middle range of effectiveness.

Value added as an indicator of educational effectiveness in Dutch senior secondary vocational education

This study investigates the possibilities of estimating value added as a performance indicator in senior secondary vocational education. Value added is interesting in this context because it is considered as a reliable tool for comparing the effectiveness of educational institutions. Although value added indicators have been developed since the 1980s for both primary and secondary educations, the research on school effectiveness has largely neglected vocational education because of its complexity. For estimating value added in this study, data concerning almost 90,000 students in Dutch senior secondary vocational education are used. Factors such as ethnicity, living in problematic neighbourhoods and students’ prior educational attainment appear to be significant predictors of student outcomes. The results indicate considerable differences in the effectiveness among clusters of training programmes, whereas there are hardly any differences between the educational institutions. Of the total variance among the student outcomes, 14% is related to the training programme clusters.

Dominant achievement goals across tracks in high school

The dominant achievement goals (DAGs) of 7008 students in the third grade of Dutch secondary education (US grade 9) were investigated, based on Elliot & McGregors’ 2 × 2 framework (2001), in relation to track-level and motivational variables. We found the mastery-approach goal and the performance-approach goal, generally considered adaptive, to be more prominent among students in lower tracks. In contrast, avoidance goals were more common in higher tracks. Most notably, in the highest track, the mastery-avoidance goal was the most prominent. Additionally, we found that students with a dominant performance-approach goal scored highest on almost all motivational variables examined; students without a DAG scored mostly second highest. The implications of these findings are discussed.

Accounting for previous performance of students by means of growth curves analyses to estimate the size, stability, and consistency of school effects

ABSTRACT The current study explored the size, stability, and consistency of school effects, using 2 effectiveness indicators: achievements of students at the end of primary school and growth in achievement across 3 years of schooling. The sample consisted of the scores of 25,269 students on 3 subjects, taken in Grades 4 to 6 among 3 cohorts in 319 primary schools. The results showed that (a) for students’ growth of achievement the relative proportion of variance at the school level seemed larger compared to achievement at the end of primary school; (b) the total variance in growth was substantially smaller compared to variance in achievement at the end of primary school; (c) school effects for growth are less stable across different cohorts than school effects established at a particular moment; and (d) school effects for growth are less consistent across multiple subject domains than school effects as indicated by students’ achievement at a particular moment.

Dominant achievement goals and academic outcomes across tracks in high school

Abstract The self-reported grades Dutch, English and Math of 13,970 students in the third grade of Dutch secondary education (US grade 9) were investigated with regard to educational track-level and dominant achievement goal (DAG). The performance approach goal group scored significantly higher on all three subjects than the performance avoidance group, the mastery approach group, the mastery avoidance group and the group without a DAG. In addition, the differences between the performance approach group and the other groups with regard to the three school subjects were of the same size, suggesting that the DAG is associated with the same processes across various school subjects. The magnitude of the differences between the other DAG groups, however, decreased with decreasing track level, suggesting that the DAGs’ adaptive value varies systematically with ability level.

Effects of the design of written music on the readability for children with dyslexia

This study investigates the relationship between the design of written music and the number of mistakes dyslexic and non-dyslexic children make in reading music is investigated in this study. Previous research shows that children with dyslexia experience difficulties with reading music, especially discerning pitch. Common mistakes of dyslexic students are third-transpositions. Based on insights from previous studies, a questionnaire was developed with several design adaptations of written music. The questionnaire was filled out by 72 Dutch children aged between 8 and 13 years. The results show that children with dyslexia did make significantly more mistakes in reading pitch than non-dyslexic children. The number of mistakes was reduced by enlarging the staves and also by writing all the stems of the notes in the same direction. Using differently-coloured lines in the staff did not reduce the amount of mistakes and was in some cases counterproductive.