Leonidas Kyriakides

The dynamics of educational effectiveness : a contribution to policy, practice and theory in contemporary schools

Part 1: The Background to Educational Effectiveness Research 1. Towards the Development of the Theoretical Framework of Educational Effectiveness Research 2. Essential Characteristics of Educational Effectiveness Research 3. The Comprehensive Model of Educational Effectiveness 4. Empirical Testing of the Comprehensive Model of Educational Effectiveness: Review of Research Part 2: A Dynamic Model of Educational Effectiveness 5. The Essential Features of The Dynamic Model 6. The Dynamic Model: Factors Operating at Student and Classroom Level 7. The Dynamic Model: Factors Operating at School and Context Level Part 3: Empirical Support for the Dynamic Model of Educational Effectiveness 8. Testing the Dynamic Model: A Study Investigating the Impact of Classroom-Level Factors on Student Achievement Gains 9. Testing The Dynamic Model: A Synthesis of Studies Investigating the Impact of School Factors on Student Achievement Part 4: Implications for Research, Policy and Practice 10. Implications for Research and Evaluation 11. Implications for Policy 12. Implications for the Improvement of Educational Practice

An analysis of the Revised Olweus Bully/Victim Questionnaire using the Rasch measurement model

BACKGROUND Bullying is a problem in schools in many countries. There would be a benefit in the availability of a psychometrically sound instrument for its measurement, for use by teachers and researchers. The Olweus Bully/Victim Questionnaire has been used in a number of studies but comprehensive evidence on its validity is not available. AIMS To examine the conceptual design, construct validity and reliability of the Revised Olweus Bully/Victim Questionnaire (OBVQ) and to provide further evidence on the prevalence of different forms of bullying behaviour. SAMPLE All 335 pupils (160 [47.8%] girls; 175 [52.2%]) boys, mean age 11.9 years [range 11.2-12.8 years]), in 21 classes of a stratified sample of 7 Greek Cypriot primary schools. METHOD The OBVQ was administered to the sample. Separate scales were created comprising (a) the items of the questionnaire concerning the extent to which pupils are being victimized; and (b) those concerning the extent to which pupils express bullying behaviour. Using the Rasch model, both scales were analysed for reliability, fit to the model, meaning, and validity. Both scales were also analysed separately for each of two sample groups (i.e. boys and girls) to test their invariance. RESULTS Analysis of the data revealed that the instrument has satisfactory psychometric properties; namely, construct validity and reliability. The conceptual design of the instrument was also confirmed. The analysis leads also to suggestions for improving the targeting of items against student measures. Support was also provided for the relative prevalence of verbal, indirect and physical bullying. As in other countries, Cypriot boys used and experienced more bullying than girls, and boys used more physical and less indirect forms of bullying than girls. CONCLUSIONS The OBVQ is a psychometrically sound instrument that measures two separate aspects of bullying, and whose use is supported for international studies of bullying in different countries. However, improvements to the questionnaire were also identified to provide increased usefulness to teachers tackling this significant problem facing schools in many countries.

Critical analysis of the current approaches to modelling educational effectiveness: The importance of establishing a dynamic model

Researchers in the area of educational effectiveness should attempt to develop a new theoretical framework. A critical analysis of the current models of educational effectiveness research is provided and reveals that a dynamic model of effectiveness must: (a) be multilevel in nature, (b) be based on the assumption that the relation of some effectiveness factors with achievement may be curvilinear, (c) illustrate the dimensions upon which the measurement of each effectiveness factor should be based, and (d) define relations among the effectiveness factors. In principle, each factor that refers to the classroom, school, and system, can be measured by taking into account five dimensions: frequency, focus, stage, quality, and differentiation. Examples of measuring effectiveness factors operating at different levels using these 5 dimensions are given. More attention in describing in detail factors associated with teacher behaviour in the classroom is given, since this is seen as the starting point for the development and the testing of the dynamic model. Finally, suggestions for the next steps in the development of other parts of the model are provided.

Differential Teacher Effectiveness: Towards a model for research and teacher appraisal

The article reviews the research on teacher effectiveness and develops the case for a model of teacher effectiveness in which differential effectiveness is incorporated. Five problems with current concepts of teacher effectiveness are identified: undue influence of available techniques upon the concept; emphasis on school, to the detriment of teacher, effectiveness; tenuous relationship to teacher improvement; narrowness of operational definitions in research; and the development of generic, rather than differentiated, models. In addition the failure of existing models to explain variance in pupil outcome at the classroom level, the neglect of teacher self-evaluation, and the restricted measures of pupil outcomes are noted. A differential model is proposed incorporating five dimensions of difference. These refer to teacher activity, outside as well as inside the classroom; curriculum subject; pupil background factors; pupil personal characteristics; cultural and organisational contexts of teaching. The developmental functions of such a model for research and for teacher appraisal are explored. Four problems for implementing a differentiated model are raised: complexity, stakeholder expectations, values, and policy acceptability. These are considered in the light of the controversial Hay McBer model in England and of models developed in Europe and the USA in the early decades of the last century.

Using a multidimensional approach to measure the impact of classroom-level factors upon student achievement: a study testing the validity of the dynamic model

The dynamic model does not only refer to different effectiveness factors and groupings of factors operating at different levels but also supports that each factor can be defined and measured using 5 dimensions: frequency, focus, stage, quality, and differentiation. The importance of taking each dimension into account is raised in this paper. Moreover, empirical support to the model and the use of this measurement framework is provided. Specifically, the paper refers to the methods and results of a study conducted in Cyprus which investigates the validity of the model at the classroom level by measuring teacher effectiveness in mathematics, language, and religious education. It is shown that the proposed measurement framework can be used to describe each classroom-level factor. The added value of using these 5 dimensions of the classroom-level factors to explain variation on student achievement is also identified. Finally, implications for the development of the dynamic model are drawn.

State of the art – teacher effectiveness and professional learning

One of the key findings from decades of educational effectiveness research is the importance of the classroom level as a predictor of pupil outcomes. In this review, we therefore look at synthesising our best evidence from research on effective teaching, and its corollary, teacher development. In the 1st section, we will look at key findings from 35 years of research on effective teaching using a process-product research that has led to the identification of a range of behaviours which are positively related to student achievement. A key limitation of this research, however, is its focus on basic skills in English and maths. Therefore, in the 2nd section we review research on “new learning” and teaching for metacognitive and thinking skills. While in these 2 sections we have discussed key findings from research on teaching, including emerging knowledge on metacognition, it is important to continue to take into account ongoing developments in theories of learning. In the 3rd section of this paper, we develop the argument that a major contributing factor to this situation is that “state-of-the-art” understandings about processes and conditions that promote student learning are typically not used to construct appropriate learning environments for their teachers.

A synthesis of studies searching for school factors: implications for theory and research

This paper reports the results of a meta-analysis in which the dynamic model of educational effectiveness is used as a framework to search for school factors associated with student achievement. The methods and results of a synthesis of 67 studies are presented. Findings reveal that effective schools are able to develop policies and take actions in order to improve their teaching practice and learning environment. Factors excluded from the dynamic model were found to be only weakly associated with outcomes. Implications for research on school effectiveness and for improvement of practice are drawn. It is illustrated that this approach of conducting meta-analysis helps us interpret the findings by providing support to the validity of the dynamic model and suggestions for its further development.

Generating Criteria for Measuring Teacher Effectiveness Through a Self-Evaluation Approach: A Complementary Way of Measuring Teacher Effectiveness

The article argues that the traditional conception of teacher effectiveness, focused on the teaching performance of individual teachers in relation to student cognitive outcomes, has limitations primarily because it does not recognise broader roles and responsibilities. A case study of a primary school staff attempting to generate criteria for effectiveness was conducted in order to develop a complementary approach. The criteria generated cover most of the characteristics of the effective teacher in the literature. Also the criteria generated in the case study school correlated strongly with those of a nationally representative sample. Implications for teacher effectiveness research are drawn.

A longitudinal study on the stability over time of school and teacher effects on student outcomes

This paper reviews educational effectiveness theory, concentrating on the time stability of the teacher and school effect. The contribution of longitudinal studies investigating the long‐term effect of schools and teachers to modelling educational effectiveness is discussed. Findings of a longitudinal study on the progress of students (N=1681) in mathematics during their first four years at the primary school are presented. Results of this study reveal that traditional approaches of measuring educational effectiveness tend to overestimate the short‐term effects of teachers and student background factors and underestimate the long‐term effects of teachers and schools. Implications of findings for the theory of educational effectiveness and especially for the concepts of teacher and school effects and their stability are drawn. Finally, suggestions for establishing evaluation mechanisms to improve practice are provided.

The power of Interactive Groups: how diversity of adults volunteering in classroom groups can promote inclusion and success for children of vulnerable minority ethnic populations

Despite the limited success of grouping students by attainment in enhancing educational achievement for all, this practice is still widely followed in European schools. Aiming at identifying successful educational actions that promote high academic achievement and social inclusion and cohesion, part of the EU-sponsored Europe-wide INCLUD-ED project analysed different ways of grouping students in classrooms. A classification was developed that distinguishes between three ways according to two dimensions – homogeneity/heterogeneity and use of human resources. This classification differentiates among mixture, streaming, and inclusion. In this paper, an example of inclusive action, Interactive Groups (IGs), is explored in depth. Based on grouping students heterogeneously and including adults from the community in the classroom, IGs address educational inequalities and enhance learning for students participating in them. Empirical data obtained from three case studies in schools in Spain indicate that IGs are one of the most successful inclusive actions implemented in these schools.