Kenneth J. Rowe

Multilevel Modelling in School Effectiveness Research

ABSTRACT Fifteen years ago, Burstein (1980) argued that the key to methodological progress in studies of classroom and school effects depended on the development of appropriate models and methods for the analysis of multilevel data. Considerable progress has been made in the intervening years such that anyone familiar with the growing school effectiveness research literature will have encountered the methodological imperative: ‘Pay attention to the multilevel organisational structure in which schooling occurs’ (i.e., students within classes within schools). Results are now available from a number of studies that have employed multilevel modelling to investigate school and teacher effectiveness. In the main, these results suggest that variation between classes is far more significant than variation between schools, although in detail the evidence often appears to be contradictory and open to a variety of interpretations. This article considers why different studies generate different findings, identifies s...

Assessing, Recording and Reporting Students’ Educational Progress: the case for ‘subject profiles’

Abstract In response to persistent concerns about limitations endemic to the exclusive use of standardised tests and examinations for student assessment and standards monitoring, one approach to addressing these concerns in Victoria (Australia) since 1986 has been the development and use of ‘subject profiles’ as frameworks’ for assessing, recording and reporting students’ educational progress. Following a brief context review and account of their origins, this paper defines what is meant by subject profiles and indicates why and how they have been developed and used. In arguing the case for the use of subject profiles in both monitoring and explanatory research, attention is given to both their utility and limitations. For illustrative purposes, descriptive data are presented from recent studies involving 34 000 students in Reception to Year 11, drawn from 650 government, Catholic, independent primary and secondary schools, using the ‘reading’ strand from the Victorian English Profiles and the ‘number’ st...

Methodological Issues in Educational Performance and School Effectiveness Research: A Discussion with Worked Examples

There has been a growing awareness among educational researchers of the consequences of using data-analytic models that fail to account for the inherent clustered or hierarchical sampling structure of the data typically obtained. Such clustering poses special analytic problems related to levels of analysis, aggregation bias, heterogeneity of regression and parameter mis-estimation, with important implications for the correct interpretation of effects. This paper compares the results obtained from fitting single-level and multilevel models to two hierarchically structured data sets designed to explain variation in student achievement. Emphasis is given to the crucial importance of fitting models commensurate with the sampling structure of the data to which they are applied.

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 1 Introduction: Effects and context

The co-occurrence of inattentiveness and poor literacy achievement is problematic for a significant number of children and adolescents. Such problems are resistant to intervention and constitute substantial management difficulties for parents, educators and health professionals. Due to conceptual and methodological deficiencies endemic to inquiry in this field, findings from prevailing explanatory models are inadequate and misleading. However, the identification of salient contributing factors is difficult for two key reasons. First, indicators of poor literacy achievement and inattentiveness are heterogeneous and overlap. Second, there has been a persistent failure of researchers to fit cross-validated models that adequately reflect the complex inter-relationships that operate among multivariate, multilevel factors affecting child behavior and educational outcomes. The chapter concludes with an acceptance of Hinshaw’s (1992, Journal of Consulting and Clinical Psychology, 60, 893–903) “challenge,” namely: “The challenge for the field is to derive explanatory models with sufficient rigor and complexity to handle the diversity of causal factors” (p. 151).

Symptom patterns of children and adolescents with chronic fatigue syndrome

Publisher Summary This chapter discusses the symptom patterns of children and adolescents with chronic fatigue syndrome (CFS). It documents the symptoms reported, in terms of presence and severity, by 189 young people presenting for assessment of chronic fatigue, and compares them with responses from a matched sample of 68 normal, healthy controls. To estimate the magnitude of the differences between the clinical and control groups on the means of all composite factor scores, a multiple analysis of variance (MANOVA) was undertaken using STATISTICA. CFS is a condition of unknown etiology, characterized by extreme fatigue exacerbated by minimal physical activity. In addition, symptoms such as difficulty with concentration, headache and sleep disturbance, muscle aches and pains and recurrent sore throat are common. CFS is predominantly an illness of young adults. Although the differentiation of CFS from a somatization disorder has been debated, and the role of psychological factors in “perpetuating” the illness has been discussed, it is generally agreed that patients consistently report the symptom complex and that it does differ from depression and other common fatiguing conditions.

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 6 Epistemological and methodological issues: A synthesis

Abstract An overview of the investigation is provided within the framework of a discussion of epistemological and methodological issues related to the status of claims to knowledge in the field. Future research must confront the inherent complexity and substantive richness of the phenomena being investigated by employing conceptual orientations and investigative methodologies that are commensurate with that complexity. The challenge is to refocus the prevailing psycho-behavioral models accounting for the overlap between inattentive behavior problems and poor academic achievement — together with their related intervention emphases — to educational ones. Indeed, It is the considered view of the authors that the personal, social and financial costs of failure to meet this challenge will be both unsustainable and unbearable.

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 4 Model specifications, results and comments: fitting multi–level, non-recursive structural equation models

Abstract The models fitted in this chapter are consistent with the major proposition of the investigation. The overlap between students’ inattentive behaviors in the classroom and their literacy achievement is reciprocal and mediated by the dynamic, interdependent effects of prior and concurrent inattentive behaviors and literacy achievements, as well as being subject to background and contextual influences — both at the student level and at the class/teacher level. To this end, the results of fitting two bi-directional explanatory models to the cross-validated data are compared, the findings from which are examined as a basis for fitting a third multilevel, non-recursive, structural equation model. In sum, the findings indicate that whereas students’ inattentive behaviors have significant negative effects on their literacy progress, literacy achievement has notably stronger effects on decreasing their early and subsequent inattentive behaviors.

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 5 Implications of the findings

Abstract The findings are discussed in terms of their methodological and substantive implications. This includes evaluations of the methods used to: (1) assess students’ attentive–inattentive behaviors in the classroom, (2) assess their literacy progress, and (3) analyze the data. A critical evaluation of the fitted explanatory models is provided, suggesting that prevailing unidirectional, single-level, explanatory models purporting to “account” for the overlap are inadequate. Two major substantive implications for current and future intervention policies and practices are discussed. First, early success in literacy is vital for students since it has significant positive effects on both their early and subsequent attentive behaviors and achievement hence reducing the need for subsequent behavioral management interventions. Second, since teachers make a difference, the need to provide them with structured approaches to the teaching of early literacy is paramount.

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 2 Purpose, data and analytic methods for the investigation

Abstract The major purpose of the investigation is to address the conceptual and methodological difficulties outlined in Chapter 1. For cross-validation purposes, the data from two longitudinal studies are described. The first was a controlled intervention study of early literacy progress of 3586 students (Grades K–1) in 202 classes in 51 elementary schools; the second, a longitudinal cohort study of 4527 students (Grades K–5) in 339 classes in 52 elementary schools. The chapter also includes a description of the data-analytic procedures used to treat missing data, determine construct measurement and reliability, examine the distributional and structural properties of the data, and to fit explanatory models designed to account for covariation among multivariate, multilevel factors.

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 3 Model specifications, results and comments: Fitting conditional models

Abstract The fitting of simple, three-level, variance-components models to estimate the proportion of variance in each of the continuous variables due to the inherent multilevel structure of the data (i.e., students within class/teacher groups clustered within schools), identified substantial residual variation at the class/teacher-level for several variables. To explain variation in students’ literacy progress, and their attentive–inattentive behaviors, respectively, two traditional, unidirectional models are then specified and fitted to each of the data sets. The results of fitting both single-level and multi-level versions of these models are compared and evaluated critically in terms of their explanatory power and their links with the research literature. As a basis for subsequently specifying and fitting three multilevel, non-recursive, structural equation models to the data in Chapter 4, the findings from fitting multivariate, multilevel models to each data set are presented. This is done to estimate the variances and covariances among the variables — both at the student-level and at the class/teacher-level.